Visualizing matrix exponents

From: 3blue1brown

Matrix exponentiation, symbolized as ‘E’ to the power ‘A’ ‘t’ where ‘A’ is a matrix, initially appears as “total nonsense” when taken out of context [00:00:22]. However, it represents an extremely beautiful and useful mathematical operation [00:00:29]. It is primarily used to solve an important class of differential equations [00:00:33]. Given that the universe is often described by differential equations, matrix exponents frequently appear in physics, particularly in quantum mechanics [00:00:44].

A key aspect of understanding matrix exponents is the ability to visualize what they are doing using “flow” [00:01:08].

Defining Matrix Exponentiation

The term ‘E’ to the power ‘A’ ‘t’ does not involve multiplying the constant ‘E’ by itself multiple times [00:01:27]. Instead, its definition is based on the infinite polynomial known as the Taylor series for describing real number powers of ‘E’ [00:01:33]. This polynomial is: e^x = 1 + x + x^2/2! + x^3/3! + ... [00:01:36]

Mathematicians became interested in plugging various objects, such as complex numbers and matrices, into this polynomial, even when these objects do not immediately make sense as exponents [00:02:14]. While this equation is a theorem for real numbers, it serves as a definition for more complex inputs [00:02:58]. Some authors use the name ‘exp’ when plugging in “exotic inputs” like matrices, as a nod to its connection to exponential functions [00:02:26].

To plug a matrix into this polynomial:

• The matrix must have the same number of rows and columns (be a square matrix) [00:03:16].
• Matrix powers (squaring, cubing, etc.) mean repeated matrix multiplication [00:03:22]. For example, squaring means multiplying the matrix by itself [00:03:25], and cubing means multiplying the result by the original matrix again [00:03:28].
• Raising a matrix to the zeroth power results in the identity matrix [00:04:27].
• Each term in the polynomial is scaled by one divided by a factorial, which means multiplying each component of the matrix by that number [00:03:53].
• Matrices are added term by term [00:04:03].

As terms are added in the infinite sum, the result approaches a stable value [00:04:51]. For instance, exponentiating a 2x2 matrix with negative pi and pi on its off-diagonal entries results in approximately negative one times the identity matrix [00:04:20]. This is essentially the matrix version of Euler’s famous identity [00:05:07].

Motivation for Matrix Exponents

Mathematicians and physicists are interested in matrix exponents because they help solve specific types of problems [00:06:29]. Two examples that motivate matrix exponents are:

1. Relationships: A system of differential equations describing the changing affections between two lovers, Romeo and Juliet [00:06:43].
2. Quantum Mechanics: Schrodinger’s equation, which describes how quantum systems evolve over time [00:13:46].

The Romeo and Juliet Example

This system involves two changing values: Juliet’s love for Romeo (x) and Romeo’s love for Juliet (y) [00:06:46]. Their rates of change are defined as:

• dx/dt = -y (Juliet’s love increases when Romeo shows disinterest) [00:07:03]
• dy/dt = x (Romeo’s love decreases when Juliet is angry, grows when she loves him) [00:07:27]

This is a system of differential equations [00:08:04]. The values can be packaged as coordinates of a single point in a 2D space, (x,y), representing their relationship as a column vector [00:08:53]. The rate of change of this state is a velocity vector [00:09:28].

The system can be written as a matrix-vector multiplication: d/dt [x y] = [[0 -1] [1 0]] * [x y] [00:09:53]

The matrix [[0 -1] [1 0]] is a 90-degree rotation matrix [00:11:20], which can be understood by observing how it transforms the basis vectors [00:11:02]. For this system, the velocity vector is always perpendicular to the position vector [00:11:36]. This implies circular motion around the origin, rotating at one radian per unit time [00:12:12]. A more general rotation matrix, [[cos t -sin t] [sin t cos t]], describes this kind of rotation [00:12:25]. To predict where Romeo and Juliet end up after ‘t’ units of time, one multiplies this rotation matrix by their initial state [00:12:49].

Generalizing to Solve Differential Equations

The general form of the differential equation that matrix exponents solve is where the rate of change of a vector is equal to a constant matrix times the vector itself [00:10:07].

One-Dimensional Case Analogy: The simplest case is a single changing value x, where its rate of change equals some constant r times itself: dx/dt = r*x [00:14:45]. This describes phenomena like compound interest or population growth [00:15:16]. The solution is x(t) = x0 * e^(rt), where x0 is the initial condition [00:15:51]. The exponential term acts on the initial condition to give a solution [00:16:33].

Higher Dimensions: In higher dimensions, for a changing vector whose rate of change is d/dt (vector) = M * (vector), the solution looks like an exponential term acting on a given initial condition vector [00:16:46]. The exponential part e^(Mt) produces a matrix that changes with time [00:16:58]. The definition of matrix exponentiation is motivated by ensuring this fact is true [00:17:06].

For the Romeo and Juliet system, the solution is e^(Mt) * initial_vector, where M = [[0 -1] [1 0]] [00:17:19]. Computing e^(Mt) using the Taylor series:

• Successive powers of M follow a cycling pattern every four iterations [00:18:00].
• When summing the infinite series, each component of the resulting matrix becomes a Taylor series for either sine or cosine (or negative sine) [00:18:35].
• This computation yields exactly the rotation matrix [[cos t -sin t] [sin t cos t]] [00:18:58].

This demonstrates that two different ways of reasoning—geometric analysis and matrix exponentiation computation—lead to the same solution for the system [00:19:09]. The initial example of exponentiating the matrix with negative pi and pi on its off-diagonals to produce the negative identity matrix is explained: it’s e^(M*pi), effectively rotating 180 degrees [00:19:35]. This is analogous to imaginary number exponents, where e^(it) describes rotation [00:20:06].

Schrodinger’s Equation

Schrodinger’s famous equation is a fundamental equation in quantum mechanics [00:13:46]. It shares the form d/dt (state_vector) = (matrix) * (state_vector) [00:14:17]. The relevant matrix in Schrodinger’s equation describes a kind of rotation, often in a “function space” [00:21:19]. The imaginary unit i in Schrodinger’s equation plays a role similar to the 90-degree rotation matrix in the Romeo-Juliet example, communicating that the rate of change is perpendicular to the state, leading to oscillation over time [00:21:44].

Visualizing Matrix Exponents with Vector Fields

Systems of differential equations can be visualized using a vector field [00:22:15]. The equation d/dt (vector) = M * (vector) means that at every point v in space, the velocity of a state passing through that point is M times v [00:22:35].

To intuitively understand how an initial condition evolves, one can “flow” along this vector field, with the velocity always matching the vector at the current point [00:22:54]. The matrix e^(Mt) describes this transition from the starting point to the final point after t units of time [00:23:10].

For the 90-degree rotation matrix example, the vector field causes circular flow, which lines up with the rotation described by e^(Mt) [00:23:33]. In other examples, such as a different Romeo and Juliet system where feelings feed off each other, the vector field shows states either growing towards infinity or decreasing [00:24:19]. In this case, e^(Mt) would describe a transformation that squishes along one diagonal while stretching along another [00:24:50].

The property that e^(Mt) times an initial condition solves these systems can be demonstrated by taking the derivative of the Taylor series: applying the power rule cancels factorial terms, showing that the derivative of the expression is M times the original expression [00:25:03].

Further Considerations

Understanding the properties of matrix exponentiation, especially its relationship with eigenvectors and eigenvalues, can lead to more concrete methods for computation [00:26:21].