Module 8 Practice: Applications of Linear Algebra
10 problems on change of basis, SVD, PCA, and computer graphics transformations.
Problem 1
B = {(1, 2), (3, 1)}. Find [x]_B for x = (7, 8).
Solution
Solve c1(1,2) + c2(3,1) = (7,8). System: c1 + 3c2 = 7 and 2c1 + c2 = 8.
From eq2: c1 = (8-c2)/2. Sub into eq1: (8-c2)/2 + 3c2 = 7 → 8 - c2 + 6c2 = 14 → 5c2 = 6, c2 = 6/5.
c1 = (8 - 6/5)/2 = (34/5)/2 = 17/5. [x]_B = (17/5, 6/5).
Problem 2
If A and B are similar matrices, and A has eigenvalues 2, 3, 5, what are the eigenvalues of B?
Solution
Similar matrices share eigenvalues. B has eigenvalues 2, 3, 5.
Problem 3
Find the singular values of A = [3 0; 4 0].
Solution
A^T A = [3 4; 0 0][3 0; 4 0] = [25 0; 0 0]. Eigenvalues: 25 and 0.
Singular values: sigma_1 = sqrt(25) = 5, sigma_2 = 0. Rank = 1.
Problem 4
A covariance matrix has eigenvalues 10, 3, 1, 0.5. What proportion of variance does PC1 explain? How many components for 90%?
Solution
Total = 14.5. PC1: 10/14.5 = 69.0%. PC1+PC2: 13/14.5 = 89.7%. PC1+PC2+PC3: 14/14.5 = 96.6%.
Need 3 components for over 90%.
Problem 5
Write the 3x3 homogeneous matrix for translating by (4, -2) then scaling by 3.
Solution
Translate: T = [1 0 4; 0 1 -2; 0 0 1]. Scale: S = [3 0 0; 0 3 0; 0 0 1].
Combined (scale after translate): M = S * T = [3 0 12; 0 3 -6; 0 0 1].
Problem 6
Apply a 90-degree rotation to the point (2, 3). Then apply it again. What transformation is two 90-degree rotations?
Solution
R90 = [0 -1; 1 0]. R90*(2,3) = (-3, 2). R90*(-3,2) = (-2, -3).
Two 90-degree rotations = 180-degree rotation = [-1 0; 0 -1]. Sends (x,y) to (-x,-y). Check: R90^2 = [0 -1; 1 0]^2 = [-1 0; 0 -1].
Problem 7
Find the perspective projection of (8, 6, 4) onto the z=1 plane.
Solution
(x/z, y/z) = (8/4, 6/4) = (2, 1.5).
Problem 8
The SVD of A gives singular values 7, 3, 1. What is the rank-1 approximation error (in Frobenius norm)?
Solution
||A - A_1||_F = sqrt(sigma_2^2 + sigma_3^2) = sqrt(9 + 1) = sqrt(10).
Problem 9
P_B = [2 1; 1 1]. Find P_B^{-1} and use it to find [x]_B for x = (5, 3).
Solution
det(P_B) = 2-1 = 1. P_B^{-1} = [1 -1; -1 2].
[x]_B = P_B^{-1}(5,3) = (5-3, -5+6) = (2, 1). Check: 2(2,1) + 1(1,1) = (4,2)+(1,1) = (5,3).
Problem 10
Center the data {(2, 4), (4, 8), (6, 6)} and compute the covariance matrix.
Solution
Means: (4, 6). Centered: (-2, -2), (0, 2), (2, 0). X_c = [-2 -2; 0 2; 2 0].
X_c^T X_c = [8 4; 4 8]. S = (1/2)[8 4; 4 8] = [4 2; 2 4].