Dot products and vectors

From: 3blue1brown

Dot products are a fundamental concept in linear algebra, often introduced early in a course [00:00:20]. While a basic understanding of vectors is sufficient for an initial grasp, a deeper understanding of dot products emerges when viewed through the lens of linear transformations [00:00:35].

Numerical Definition

Numerically, the dot product of two vectors of the same dimension (lists of numbers with the same length) is calculated by:

1. Pairing up all corresponding coordinates [00:00:59].
2. Multiplying those pairs together [00:01:00].
3. Adding the results [00:01:03].

Examples:

• The dot product of vector (1, 2) and (3, 4) is (1 * 3) + (2 * 4) = 3 + 8 = 11 [00:01:06].
• The dot product of vector (6, 2, 8, 3) and (1, 8, 5, 3) is (6 * 1) + (2 * 8) + (8 * 5) + (3 * 3) = 6 + 16 + 40 + 9 = 71 [00:01:14].

Geometric Interpretation

The numerical calculation of the dot product has a distinct geometric interpretation [00:01:24].

To understand the dot product between two vectors, v and w:

1. Imagine projecting w onto the line that passes through the origin and the tip of v [00:01:33].
2. Multiply the length of this projection by the length of v [00:01:38]. This product is v dot w [00:01:42].

Directional Significance

The sign of the dot product indicates the general direction of the vectors:

• If the projection of w is pointing in the opposite direction from v, the dot product is negative [00:01:46].
• When two vectors generally point in the same direction, their dot product is positive [00:01:53].
• When two vectors are perpendicular, meaning the projection of one onto the other is the zero vector, their dot product is zero [00:01:59].
• If they point in generally the opposite direction, their dot product is negative [00:02:05].

Order Invariance

Despite this seemingly asymmetric geometric interpretation, the order of vectors in a dot product does not matter [00:02:16]. Projecting v onto w and multiplying by the length of w yields the same result as projecting w onto v and multiplying by the length of v [00:02:20].

This symmetry can be intuited:

• If v and w have the same length, the processes of projecting w onto v and v onto w are mirror images, resulting in the same dot product [00:02:38].
• If one vector, say v, is scaled by a constant (e.g., 2v), the dot product 2v dot w will be twice v dot w [00:02:57]. This holds true regardless of which vector is considered for projection:
  • Projecting w onto 2v: The length of w’s projection doesn’t change, but the length of the vector 2v is doubled [00:03:20].
  • Projecting 2v onto w: The length of 2v’s projection is scaled by 2, while the length of w remains constant [00:03:30]. In both cases, the overall dot product is doubled [00:03:47].

Duality: Connecting Numerical Computation and Geometric Projection

A deeper understanding of why the numerical dot product calculation relates to geometric projection involves the concept of linear transformations from multiple dimensions to a single dimension (the number line) [00:04:10].

Linear Transformations to the Number Line

These are functions that take a 2D vector as input and output a single number [00:04:32]. Like other linear transformations, they maintain even spacing of dots on a line when mapped to the output space (the number line) [00:04:59].

A linear transformation is completely determined by where it takes the basis vectors i-hat and j-hat [00:05:22]. Since the output space is a single number, the landing spots of i-hat and j-hat are recorded as single numbers, forming the columns of a 1x2 matrix [00:05:26].

Example: If a linear transformation maps i-hat to 1 and j-hat to -2 [00:05:46], then for a vector (4, 3):

• It’s expressed as 4 times i-hat plus 3 times j-hat [00:05:56].
• After transformation, it lands at (4 * 1) + (3 * -2) = 4 - 6 = -2 [00:06:01]. This calculation is identical to matrix-vector multiplication [00:06:18].

The Connection: Duality

The numerical operation of multiplying a 1x2 matrix by a vector strongly resembles taking the dot product [00:06:25]. This suggests an association between 1x2 matrices and 2D vectors by simply “tipping” one on its side to get the other [00:06:37]. This seemingly simple numerical correspondence hints at a profound geometric connection:

Consider a linear transformation defined by projecting 2D vectors onto a diagonal copy of the number line embedded in 2D space, with 0 at the origin [00:07:28]. Let u-hat be the unit vector in 2D space whose tip is at the number 1 on this diagonal number line [00:07:36]. This projection defines a linear transformation from 2D vectors to numbers [00:07:50].

To find the 1x2 matrix describing this projection:

• Projecting i-hat onto the line passing through u-hat is symmetric to projecting u-hat onto the x-axis [00:09:03].
• Projecting u-hat onto the x-axis simply means taking its x-coordinate [00:09:22].
• Therefore, the number where i-hat lands when projected onto the diagonal number line is the x-coordinate of u-hat [00:09:29].
• Similarly, the number where j-hat lands is the y-coordinate of u-hat [00:09:39].

Thus, the entries of the 1x2 matrix describing this projection transformation are the coordinates of u-hat [00:10:00]. Calculating this projection transformation for any vector involves multiplying that matrix by the vector, which is computationally identical to taking a dot product with u-hat [00:10:08]. This explains why dotting with a unit vector can be interpreted as projecting a vector onto that unit vector’s span and taking the length [00:10:21].

For non-unit vectors, if u-hat is scaled by a factor (e.g., 3), its components are also scaled by 3 [00:10:36]. The associated matrix will take i-hat and j-hat to three times their previous values [00:10:44]. This means the new matrix projects any vector onto the number line and then multiplies the result by 3 [00:10:55]. This explains why the dot product with a non-unit vector is interpreted as first projecting onto that vector, then scaling the length of that projection by the length of the vector [00:11:05].

This revelation highlights a concept called duality [00:12:12]. Duality describes a natural, often surprising, correspondence between two types of mathematical objects [00:12:22]. In this case, the dual of a vector is the linear transformation it encodes, and the dual of a linear transformation from a space to one dimension is a specific vector in that space [00:12:34].

Conclusion

On the surface, the dot product is a valuable geometric tool for understanding projections and determining if vectors point in similar directions [00:12:46]. More profoundly, dotting two vectors together serves as a method to translate one of them into the realm of transformations [00:13:01]. This signifies that a vector can sometimes be best understood not just as an arrow, but as the physical embodiment of a linear transformation [00:13:18]. Vectors, in this sense, act as a conceptual shorthand for certain transformations [00:13:30].