Convolution theorem and its mathematical significance

From: 3blue1brown

The Convolution Theorem is a fundamental concept in mathematics, particularly in the fields of Fourier analysis and signal processing. It provides a crucial link between the operations of multiplication and convolution when functions are transformed between different domains via the Fourier Transform. This theorem offers a powerful method for solving complex problems by shifting perspective, often making intricate calculations simpler [00:13:50].

The Sinc Function Integral Mystery

The practical significance of the Convolution Theorem can be illustrated by a peculiar mathematical phenomenon involving integrals of the sinc function. The sinc function is defined as sine(x)/x [00:00:41]. While x=0 appears to be an issue, the function approaches 1 as x approaches 0, allowing for a continuous definition where sinc(0) = 1 [00:01:15].

Consider a sequence of integrals involving sinc functions:

1. The integral of sinc(x) from negative infinity to infinity equals exactly pi [00:01:42].
2. Multiplying sinc(x) by sinc(x/3) and integrating also yields pi [00:02:08], [00:02:24].
3. Continuing this pattern, multiplying by sinc(x/5), sinc(x/7), sinc(x/9), sinc(x/11), and sinc(x/13) still results in pi [00:02:31], [00:02:44].
4. However, when the next factor, sinc(x/15), is included, the integral surprisingly breaks the pattern, yielding a value just barely less than pi [00:00:27], [00:03:16]. This exact value is a complex fraction of pi [00:03:27].

This pattern, described by Jonathan and David Borwein [00:03:40], was initially mistaken for a bug in computer algebra systems due to its unexpected stability and minute deviation [00:03:49]. The phenomenon becomes even stranger when an additional factor of 2 cos(x) is included in the integrals. In this case, the pattern of equaling pi continues much longer, breaking only when the factor sinc(x/113) is introduced, again by an incredibly subtle amount [00:04:09], [00:04:15].

The Analogous Moving Average Process

To understand this, consider a seemingly unrelated process involving a “rectangle” function, rect(x), defined as 1 for x between -1/2 and 1/2, and 0 otherwise [00:04:52]. Let this be f1(x).

A sequence of new functions (f2(x), f3(x), etc.) is generated by taking a moving average of the previous function [00:05:11]:

• f2(x) is the moving average of f1(x) using a window of width 1/3 [00:05:15]. This results in a new function f2(x) that also has a plateau where its value is 1 [00:06:00], but this plateau is narrower, specifically 1 - 1/3 in width [00:06:33]. The value f2(0) is 1 [00:06:44].
• f3(x) is the moving average of f2(x) using a window of width 1/5 [00:06:49]. Its plateau width becomes 1 - 1/3 - 1/5 [00:07:11], and f3(0) is still 1 [00:07:36].
• This process continues with window widths of 1/7, 1/9, and so on [00:07:38]. The plateau continually shrinks [00:07:48].

The values f_n(0) remain exactly 1 as long as the cumulative sum of the reciprocals of the odd numbers (1/3 + 1/5 + 1/7 + ...) remains less than the initial plateau width (which is 1). The point where this pattern breaks is when the sum becomes greater than 1, which occurs when 1/15 is added to the sum [00:08:29]. At this point, the plateau “disappears” (becomes thinner than the moving average window), and f_n(0) becomes slightly less than 1 [00:08:10], [00:08:17].

This mirrors the sinc integral pattern [00:09:00]. In the case where the 2 cos(x) term is added, it corresponds to starting with a rect function with an initial plateau of width 2 (from -1 to 1) [00:09:36]. In this scenario, it takes much longer for the sum of reciprocals to exceed the initial plateau width, specifically until the sum exceeds 2, which occurs when 1/113 is added [00:09:51], [00:10:03].

The Role of Fourier Transforms and the Convolution Theorem

The crucial insight is that these two seemingly disparate phenomena are, in fact, “two distinct ways of computing exactly the same thing” [00:12:10]. The connection is made through the Fourier Transform and the Convolution Theorem [00:10:48].

The Fourier Transform converts a function into a representation in the “frequency domain” [00:12:32]. For example, the sinc(pi*x) function (a normalized version of sinc(x) with an area of 1 [00:11:45]) is related to the rect(x) function via a Fourier Transform [00:12:21], [00:13:01]. Stretching sinc(x) by a factor k corresponds to stretching and squishing rect(x) in the Fourier domain [00:13:17].

Two key facts from Fourier analysis bridge the gap:

1. Integral to Evaluation at Zero: The integral of a function from negative infinity to infinity is equivalent to evaluating its Fourier transformed version at the input zero [00:14:00], [00:14:07].

   • This explains why the integral of sinc(pi*x) is 1: because rect(0) = 1 [00:14:39].

2. The Convolution Theorem: When two functions are multiplied in the original domain, their Fourier transforms are combined through an operation called convolution in the frequency domain [00:15:11].

   • Crucially, for the specific rectangular functions discussed, a convolution operation is mathematically equivalent to the moving average process described earlier [00:15:31], [00:15:35].

Therefore, multiplying successive sinc functions (e.g., sinc(x) * sinc(x/3) * sinc(x/5)) in the original domain corresponds to performing successive convolutions (which are like moving averages) of their respective rect function equivalents in the Fourier domain [00:15:55]. The initial integral value of pi (or 1 for sinc(pi*x)) is maintained as long as the “plateau” of the convolved rect function remains wide enough to encompass zero [00:16:07]. Once the cumulative “eating away” by the moving averages (convolutions) causes the plateau around zero to shrink past a critical point, the value at zero (and thus the integral) drops slightly [00:16:07].

Broader Significance

The Convolution Theorem, often mentioned alongside Fourier Transforms, offers a systematic way to shift perspective on mathematical problems [00:16:37]. It allows hard problems involving multiplication in one domain to be simplified by transforming them into easier problems involving convolution in another domain [00:13:50].

Beyond this specific esoteric integral problem, the Convolution Theorem is widely applicable in various fields:

• Signal Processing: Crucial for understanding filtering, blurring, and other signal modifications.
• Image Processing: Used in tasks like image sharpening, blurring, and edge detection, which often involve convolution operations.
• Probability Theory: In probability, the probability distribution of the sum of two independent random variables is the convolution of their individual probability distributions.
• Polynomial Multiplication: It enables algorithms, such as the Fast Fourier Transform (FFT) based multiplication, to compute the product of two large numbers or polynomials significantly faster than traditional methods, which can be thought of as a discrete convolution algorithm [00:16:53].

The elegant connection between the sinc function integrals and the moving average process serves as a compelling example of the power and beauty of the Convolution Theorem and Fourier analysis in revealing hidden mathematical structures [00:16:46].