Fourier Series and Fourier Transforms

When I was studying differential equations, I didn’t think that Fourier analysis would be that useful, seeing how out-there it seemed at the time. Well, it turns out that they are really useful, especially in signal processing and materials science. Let’s talk about them!

Linear Algebra Review

The inner product (dot product) of two vectors is the sum of the pairwise products of their components:

a = [a_1, \dots, a_n], b = [b_1, \dots, b_n] \implies \langle a, b \rangle = a^Tb = \sum_{k = 1}^n a_kb_k

The inner product of $a$ and $b$ can be thought of as the component of $a$ projected onto $b$, multiplied by $b$’s length. From this, we can derive two important properties of the inner product:

  • If the inner product of two vectors is 0, then they’re orthogonal to each other in $\mathbb{R}^n$.
  • The inner product of a vector with itself is its length squared. If $\langle v, v \rangle = 1$, then we say that $v$ is normalized

Two normalized vectors that are orthogonal are called orthonormal. If we have $n$ orthonormal vectors $q_1, \dots, q_n$ in $\mathbb{R}^n$, then we say that those $n$ vectors form an orthonormal basis. That is, any vector $v$ in $\mathbb{R}^n$ can be written as a linear combination of those basis vectors: $v = \sum_{k = 1}^n c_kq_k$.

The coefficients $c_k$ can be obtained by projecting $v$ onto $q_k$, which is an inner product, so we get $v = \sum_{k = 1}^n \langle v, q_k \rangle q_k$. If the basis vectors are just orthogonal but not normalized, then we also have to divide $c_k$ by $\langle q_k, q_k \rangle$.

Extending it to Functions

A lot of the techniques that we use in linear algebra can be extended to work for functions and even infinite dimensions. For example, operators (used in quantum mechanics) are to functions as matrices are to vectors. As such, we can define the inner product of two functions $f$ and $g$ on the interval $[a, b]$ as:

\langle f, g \rangle = \int_a^b f(x)g(x)dx

(Note that this is just one definition of the inner product – we’ll use this one because it’s convenient).

Real Fourier Series

Now, consider the interval $[-L, L]$ and functions of the form $\cos\left(\frac{k\pi x}{L}\right)$ and $\sin\left(\frac{k\pi x}{L}\right)$, where $k$ is a non-negative integer. If we plug these functions into our above definition of the inner product, we can see that they’re all orthogonal to each other. In fact, they form a basis for all continuous functions on the interval $[-L, L]$. How convenient!

Because of this, we can approximate any such function $f$ as:

f(x) = \frac{a_0}{2} + \sum_{k = 1}^\infty a_k \cos\left(\frac{k\pi x}{L}\right) + b_k \sin\left(\frac{k\pi x}{L}\right)

Where the coefficients are calculated as:

a_k = \frac{1}{L} \int_{-L}^L f(x)\cos\left(\frac{k\pi x}{L}\right) dx\\
b_k = \frac{1}{L} \int_{-L}^L f(x)\sin\left(\frac{k\pi x}{L}\right) dx

The reason why we divide $a_0$ by $2$ is that $\langle \cos\left(\frac{k\pi x}{L}\right), \cos\left(\frac{k\pi x}{L}\right) \rangle$ is $2L$ if $k = 0$ and $L$ if $k \neq 0$. Also, $b_0 = 0$, so we don’t show it in the equation.

This works very well for all periodic functions with period $2L$, and it converges quite quickly too.

Animation of a partial Fourier series, courtesy of WikipediaAnimation of a partial Fourier series, courtesy of Wikipedia

In the context of differential equations, this is useful for turning ugly input functions into sinusoids, which we can then solve using undetermined coefficients. Such a solution is called the formal Fourier series solution of the differential equation.

Complex Fourier Series

Since Euler’s formula ($e^{i\theta} = \cos \theta + i\sin \theta$) tells us how to relate sinusoids to complex exponentials, we can also express the Fourier series in terms of complex exponentials:

f(x) = \sum_{k = -\infty}^\infty c_k e^{i\omega kx}

Where $\omega = \frac{\pi}{L}$ is the angular frequency and $c_k$ is calculated as:

c_k = \frac{1}{2L} \int_{-L}^L f(x)e^{-i\omega kx} dx

This form is more compact and is used a lot in electrical engineering.

Fourier Transforms

You may have noticed that in real Fourier series, the sinusoid coefficients are the same, while in complex Fourier series, we multiply by $e^{-i\omega kx}$ to get $c_k$ but then $e^{i\omega kx}$ to reconstruct $f$. This leads us to our next idea – the Fourier transform.

The 1D Fourier transform $\hat f(k)$ of a function $f(x)$ is defined as:

\hat f(k) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty f(x)e^{-ikx}dx

While the inverse Fourier transform (i.e., getting from $\hat f(k)$ to $f(x)$) is defined as:

\hat f(x) = \frac{1}{\sqrt{2\pi}} \int_{-\infty}^\infty \hat f(k)e^{ikx}dk

Note the difference in variables between the two functions.

Essentially, the Fourier transform takes an input signal and tells us what frequencies it’s composed of. In other words, it converts the function from the time domain to the frequency domain (and the inverse does the opposite). It’s a bit like constructing the Fourier series, except with continuous frequencies instead of discrete ones.

Fourier transform of two unit pulses, courtesy of WikipediaFourier transform of two unit pulses, courtesy of Wikipedia

I still don’t have much experience with Fourier transforms, but I know that it’s useful for:

  • Quantum computing (Shor’s algorithm involves the quantum Fourier transform).
  • Fast polynomial multiplication (FFTs are a whole class of algorithms).
  • X-ray diffraction (reciprocal space is the 3D Fourier transform of real space).
  • Nuclear magnetic resonance imaging (Fourier transform spectroscopy).
  • Probably a lot of stuff in signal processing.

Hopefully, I’ll learn more about applying this technique to solving problems soon!