Matrixvector multiplication

From: 3blue1brown

Understanding Coordinate Systems and Basis Vectors

A vector in 2D space can be described using coordinates, for example, a vector with coordinates 3,2 means moving three units to the right and two units up from its tail to its tip [00:00:18]. In linear algebra, these numbers are thought of as scalars that stretch or squish vectors [00:00:31].

This description is tied to a standard coordinate system defined by special vectors called basis vectors [00:01:23]:

• i-hat: The vector with length 1 pointing to the right, scaled by the first coordinate [00:00:37].
• j-hat: The vector with length 1 pointing straight up, scaled by the second coordinate [00:00:43].

The tip-to-tail sum of these two scaled vectors is what the coordinates describe [00:00:49]. These basis vectors encapsulate assumptions about the coordinate system, such as which direction is rightward or upward, and how far a unit of distance is [00:01:02].

Alternative Coordinate Systems

Different sets of basis vectors define different coordinate systems [00:01:30]. For example, a friend named Jennifer might use basis vectors b1 and b2 [00:01:39]:

• b1: Points up and slightly to the right (described as 2,1 in our standard system) [00:01:44] [00:02:49].
• b2: Points left and up (described as -1,1 in our standard system) [00:01:48] [00:02:53].

From Jennifer’s perspective, these vectors have coordinates 1,0 and 0,1, respectively, defining the meaning of those coordinates in her system [00:02:59]. While we observe the same vectors in space, Jennifer uses different numbers to describe them [00:03:15]. The origin (0,0) is consistent across all systems, but the direction of axes and spacing of grid lines will differ based on the chosen basis vectors [00:03:53].

Translating Between Coordinate Systems

A natural question arises: how do we translate vector descriptions between different coordinate systems [00:04:11]?

From Jennifer’s System to Ours

If Jennifer describes a vector with coordinates -1,2, this means the vector is -1 times b1 plus 2 times b2 [00:04:29]. To translate this into our system:

1. Represent Jennifer’s basis vectors (b1 and b2) in our coordinates: b1 = (2,1) and b2 = (-1,1) [00:04:39].
2. Perform the scalar multiplication and vector addition using our coordinates: (-1) * (2,1) + (2) * (-1,1) = (-2,-1) + (-2,2) = (-4,1) [00:04:49]. So, the vector Jennifer describes as -1,2 is described as -4,1 in our system [00:05:03].

This process is precisely matrix-vector multiplication [00:05:18]. The matrix used for this translation, known as the change of basis matrix, has Jennifer’s basis vectors as its columns (in our language) [00:05:18]:

$$\left(\begin{array}{cc}2& -1\\ 1& 1\end{array}\right)\left(\begin{array}{c}-1\\ 2\end{array}\right)=\left(\begin{array}{c}(-1\times 2)+(2\times -1)\\ (-1\times 1)+(2\times 1)\end{array}\right)=\left(\begin{array}{c}-4\\ 1\end{array}\right)$$

Interpreting Matrix-Vector Multiplication as a Linear Transformation

Understanding matrix-vector multiplication as applying a linear transformation provides intuition [00:05:25]. A matrix whose columns represent Jennifer’s basis vectors can be viewed as a transformation that moves our basis vectors (i-hat and j-hat) to Jennifer’s basis vectors (b1 and b2) [00:05:39].

When this transformation is applied to a vector represented in our system (e.g., -1 times i-hat plus 2 times j-hat), a key property of linear transformations ensures the resulting vector will be the same linear combination of the new basis vectors (the landing spots of i-hat and j-hat) [00:06:17]. In essence, this matrix transforms our “misconception” of what Jennifer means (if we use her coordinates in our system) into the actual vector she is referring to [00:06:33].

Interpretation

Geometrically, this matrix transforms our grid into Jennifer’s grid. Numerically, it translates a vector described in Jennifer’s language to our language [00:06:47].

From Our System to Jennifer’s

To go the other way – translating a vector from our system (e.g., 3,2) into Jennifer’s coordinates (5/3, 1/3) [00:07:18] – we use the inverse of the change of basis matrix [00:07:32].

The inverse of a transformation effectively “plays it backwards” [00:07:43]. For the example where Jennifer’s basis matrix is $\left(\begin{array}{cc}2& -1\\ 1& 1\end{array}\right)$, its inverse is $\left(\begin{array}{cc}1/3& 1/3\\ -1/3& 2/3\end{array}\right)$ [00:07:58].

To find the coordinates of vector (3,2) in Jennifer’s system, we multiply it by this inverse matrix [00:08:11]:

$$\left(\begin{array}{cc}1/3& 1/3\\ -1/3& 2/3\end{array}\right)\left(\begin{array}{c}3\\ 2\end{array}\right)=\left(\begin{array}{c}(1/3\times 3)+(1/3\times 2)\\ (-1/3\times 3)+(2/3\times 2)\end{array}\right)=\left(\begin{array}{c}1+2/3\\ -1+4/3\end{array}\right)=\left(\begin{array}{c}5/3\\ 1/3\end{array}\right)$$

So, the vector (3,2) in our system is (5/3, 1/3) in Jennifer’s system [00:08:16].

Translation Summary

• The matrix whose columns represent Jennifer’s basis vectors (in our coordinates) translates vectors from her language into our language [00:08:35].
• Its inverse matrix does the opposite, translating from our language to hers [00:08:46].

Translating Linear Transformations Between Coordinate Systems

Matrices are also used to represent linear transformations [00:08:55]. Our matrix for a 90-degree counterclockwise rotation, for instance, has columns (0,1) and (-1,0), representing where i-hat and j-hat land [00:09:12]. This representation is specific to our choice of basis vectors [00:09:36].

To find how Jennifer would describe this same 90-degree rotation, we need a matrix that operates on vectors in her language and outputs transformed vectors in her language [00:09:50].

This is achieved by a composition of three transformations, often seen as an expression like $A^{-1}MA$:

1. Translate to our language (A): Take any vector written in Jennifer’s language and multiply it by her change of basis matrix (A). This converts the vector into our coordinate system [00:10:23].
2. Apply the transformation (M): Apply our standard transformation matrix (M, e.g., the 90-degree rotation matrix) to the vector now expressed in our language [00:10:43]. The result is the transformed vector, still in our language [00:10:49].
3. Translate back to Jennifer’s language (A^-1): Apply the inverse of the change of basis matrix ($A^{-1}$) to get the final transformed vector back in Jennifer’s language [00:10:53].

This composition of three matrices ($A^{-1}MA$) results in the transformation matrix as seen in Jennifer’s coordinate system [00:11:04]. When Jennifer uses this matrix with a vector from her system, it returns the transformed version of that vector, also expressed in her system [00:11:44].

Mathematical Empathy

The expression $A^{-1}MA$ suggests a “mathematical empathy” [00:12:03]. The middle matrix (M) represents a transformation as we see it, while the outer matrices ($A^{-1}$ and A) represent a shift in perspective. The full matrix product represents that same transformation, but as someone else sees it [00:12:07].