Matrix representations and transformations

From: 3blue1brown

Linear transformations can be understood by considering how they affect space and, in particular, how they act on specific vectors within that space [00:00:59]. This perspective is crucial for grasping concepts like eigenvectors and eigenvalues [00:00:51].

Matrices as Linear Transformations

Matrices provide a representation of linear transformations [00:01:00]. For a matrix representation of a transformation, its columns show where the basis vectors land after the transformation [00:01:25]. For example, a transformation that moves the basis vector i-hat to (3, 0) and j-hat to (1, 2) is represented by a matrix with columns (3, 0) and (1, 2) [00:01:31].

Eigenvectors and Eigenvalues

While most vectors are “knocked off” their original span (the line passing through their origin and tip) during a linear transformation, some special vectors remain on their span [00:01:44]. The effect of the matrix representation of the transformation on such vectors is merely to stretch or squish them, as if by a scalar factor [00:02:00].

These special vectors are called eigenvectors of the transformation [00:03:22]. Each eigenvector has an associated eigenvalue, which is the factor by which it is stretched or squished [00:03:27]. An eigenvalue can be positive (stretching), negative (flipping and stretching/squishing), or even zero (collapsing onto the origin) [00:03:40]. The key property is that the vector stays on its span [00:03:56].

Examples of Eigenvectors and Eigenvalues

• Example 1 (2D Transformation): For a transformation where i-hat moves to (3,0) and j-hat to (1,2) (matrix columns are 3,0 and 1,2):
  • The basis vector i-hat is an eigenvector [00:02:09]. Its span is the x-axis, and it moves to 3 times itself, remaining on the x-axis [00:02:14]. The eigenvalue is 3 [00:03:05].
  • The vector (-1, 1) is also an eigenvector for this transformation [00:02:38]. It gets stretched by a factor of 2, so its eigenvalue is 2 [00:02:44].
• Example 2 (3D Rotation): For a three-dimensional rotation, an eigenvector represents the axis of rotation [00:04:11]. Since rotations don’t stretch or squish, the corresponding eigenvalue is 1 [00:04:37].
• Example 3 (Rotation by 90 degrees): A 90-degree rotation has no real eigenvectors because every vector is rotated off its span [00:10:50]. When calculating its eigenvalues, the solutions are imaginary numbers (i and -i), indicating no real eigenvectors [00:11:22].
• Example 4 (Shear Transformation): A shear transformation fixes the x-axis in place (eigenvalue 1) [00:11:48]. These are the only eigenvectors for a shear, and the only eigenvalue is 1 [00:11:55].
• Example 5 (Scaling Matrix): A matrix that scales everything by a factor (e.g., 2) has only one eigenvalue (e.g., 2), but every vector in the plane is an eigenvector with that eigenvalue [00:12:29].

Significance of Eigenvectors and Eigenvalues

Understanding a linear transformation by finding its eigenvectors and eigenvalues can be more insightful and less dependent on the specific coordinate system than merely reading off the columns of its matrix representation of a transformation [00:05:00].

Computing Eigenvectors and Eigenvalues

Symbolically, the idea of an eigenvector v and its eigenvalue λ for a transformation represented by matrix A is expressed as: A * v = λ * v [00:05:31]

To find v and λ that satisfy this:

1. Rewrite the right side using a matrix that scales any vector by λ. This is λ times the identity matrix (λI) [00:06:11]. A * v = (λI) * v [00:06:36]
2. Rearrange the equation: (A - λI) * v = 0 [00:06:48]
3. For a non-zero eigenvector v to satisfy this, the matrix (A - λI) must “squish” space into a lower dimension [00:07:14]. This condition corresponds to the determinant of (A - λI) being zero [00:07:29]. det(A - λI) = 0 [00:07:58]

Solving det(A - λI) = 0 for λ yields the possible eigenvalues. Once an eigenvalue is found, it can be plugged back into (A - λI) * v = 0 to solve for the corresponding eigenvectors v [00:10:09].

Eigenbasis and Diagonal Matrices

A matrix where all entries are zero except for the diagonal is called a diagonal matrix [00:13:38]. In a diagonal matrix, the basis vectors themselves are eigenvectors, and their corresponding eigenvalues are the entries on the diagonal [00:13:45].

Diagonal matrices are computationally convenient. For instance, computing a high power of a diagonal matrix (e.g., A^100) simply involves raising each diagonal entry (eigenvalue) to that power [00:14:09].

Changing to an Eigenbasis

If a transformation has enough eigenvectors to span the full space (i.e., you can choose a set of eigenvectors that form a basis), you can change your coordinate system to this eigenbasis [00:14:42] [00:15:55].

To do this, use a change of basis matrix (whose columns are the coordinates of the new basis vectors, which are your eigenvectors) [00:15:08]. By “sandwiching” the original transformation matrix A between the inverse of the change of basis matrix P⁻¹ and the change of basis matrix P (P⁻¹AP), the resulting matrix will be a diagonal matrix representing the same transformation from the perspective of the eigenbasis [00:15:20]. The diagonal entries of this new matrix will be the eigenvalues [00:15:41].

This transformation to an eigenbasis simplifies complex matrix operations, such as computing high powers of a matrix [00:16:02]. However, not all transformations have enough eigenvectors to form an eigenbasis (e.g., a shear transformation) [00:16:16].