# Day 37:

More Properties of the Fourier Transform

and convolutions

### Circulant Matrices

Definition. Let $$N\in\N$$, and let $$P$$ be the $$N\times N$$ upward shift matrix. A matrix $$C$$ is called circulant if there are numbers $$c_{0},c_{1},c_{2},\ldots,c_{N-1}$$ such that

$C = c_{0}I + c_{1}P + c_{2}P^{2} + \cdots + c_{N-1}P^{N-1}.$

$= \begin{bmatrix} c_{0} & c_{1} & c_{2} & c_{3} & \cdots & c_{N-1}\\ c_{N-1} & c_{0} & c_{1} & c_{2} & \cdots & c_{N-2}\\ c_{N-2} & c_{N-1} & c_{0} & c_{1} & \cdots & c_{N-3}\\ c_{N-3} & c_{N-2} & c_{N-1} & c_{0} & \cdots & c_{N-4}\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ c_{1} & c_{2} & c_{3} & c_{4} & \cdots & c_{0}\end{bmatrix}$

### Circulant Matrices

Definition. Let $$N\in\N$$, and let $$P$$ be the $$N\times N$$ upward shift matrix. Given a vector $$x=[a_{0}\ \ a_{1}\ \ a_{2}\ \ \cdots\ \ a_{N-1}]^{\top}$$, define the circulant matrix

$C(x) = a_{0} + a_{1}P + a_{2}P^{2} + \cdots + a_{N-1}P^{N-1}.$

Lemma. Given $$x,y\in\mathbb{C}^{n}$$ we have

$C(x)C(y) = C(x\circledast y)$

### Eigenvalues/vectors of the upward shift

Lemma. If $$P$$ is the $$N\times N$$ upward shift matrix, and $$f_{k}$$ is the $$k$$th column of the Fourier Transform matrix $$F_{N}$$, then

$Pf_{k} = e^{-2k\pi i/N}f_{k} = \omega^{k}f_{k}.$

Proof.  $f_{k} = \frac{1}{\sqrt{N}}\begin{bmatrix} 1\\ \omega^k\\ \omega^{2k}\\ \omega^{3k}\\ \vdots\\ \omega^{(N-1)k}\end{bmatrix}$

$Pf_{k} = \frac{1}{\sqrt{N}}\begin{bmatrix} \omega^k\\ \omega^{2k}\\ \omega^{3k}\\ \vdots\\ \omega^{(N-1)k}\\ 1\end{bmatrix} = \frac{\omega^{k}}{\sqrt{N}}\begin{bmatrix} \omega^0\\ \omega^{k}\\ \omega^{2k}\\ \vdots\\ \omega^{(N-2)k}\\ \omega^{-k}\end{bmatrix} = \omega^{k}\frac{1}{\sqrt{N}}\begin{bmatrix} 1\\ \omega^{k}\\ \omega^{2k}\\ \vdots\\ \omega^{(N-2)k}\\ \omega^{Nk}\omega^{-k}\end{bmatrix} = \omega^{k}f_{k}$

### Eigenvalues/vectors of the upward shift

By the lemma, the columns of $$F_{N}$$ are an orthonormal basis of eigenvectors of the $$N\times N$$ upward shift matrix $$P$$, thus

$P = F_{N}\begin{bmatrix} \omega^{0} & 0 & 0 & 0 & \cdots & 0\\ 0 & \omega & 0 & 0 & \cdots & 0\\ 0 & 0 & \omega^2 & 0 & \cdots & 0\\ 0 & 0 & 0 & \omega^3 & \cdots & 0\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 & \cdots & \omega^{N-1}\end{bmatrix}F_{N}^{\ast}$

Set $W_{N} := \begin{bmatrix} \omega^{0} & 0 & 0 & 0 & \cdots & 0\\ 0 & \omega & 0 & 0 & \cdots & 0\\ 0 & 0 & \omega^2 & 0 & \cdots & 0\\ 0 & 0 & 0 & \omega^3 & \cdots & 0\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 & \cdots & \omega^{N-1}\end{bmatrix}$

then we have $$P = F_{N}W_{N}F_{N}^{\ast}.$$

### Powers of the upward shift

Note that $W_{N}^{k} = \begin{bmatrix} \omega^{0k} & 0 & 0 & 0 & \cdots & 0\\ 0 & \omega^{k} & 0 & 0 & \cdots & 0\\ 0 & 0 & \omega^{2k} & 0 & \cdots & 0\\ 0 & 0 & 0 & \omega^{3k} & \cdots & 0\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 & \cdots & \omega^{(N-1)k}\end{bmatrix},$

and hence

$P^{k} = \underbrace{(F_{N}W_{N}F_{N}^{\ast})(F_{N}W_{N}F_{N}^{\ast})(F_{N}W_{N}F_{N}^{\ast})\cdots (F_{N}W_{N}F_{N}^{\ast})}_{k\text{ factors}}$

$$= F_{N}W_{N}(F_{N}^{\ast}F_{N})W_{N}(F_{N}^{\ast}F_{N})W_{N}(F_{N}^{\ast}F_{N})\cdots W_{N}F_{N}^{\ast}$$

$$= F_{N}W_{N}(I)W_{N}(I)W_{N}(I)\cdots W_{N}F_{N}^{\ast} = F_{N}W_{N}^{k}F_{N}^\ast$$

(Note that $$W_{N}^{N}$$ is the identity, and hence $$P^{N} = F_{N}I F_{N}^{\ast} = I$$ as we have already observed.)

### Eigenvalues of a Circulant Matrix

$$= F_{N}\big(a_{0}W_{N}^{0} + a_{1}W_{N}^{1} + a_{2}W_{N}^2 + \cdots + a_{N-1}W_{N}^{N-1}\big)F_{N}^{\ast}$$

$$= a_{0}F_{N}W_{N}^{0}F_{N}^{\ast} + a_{1}F_{N}W_{N}^{1}F_{N}^{\ast} + a_{2}F_{N}W_{N}^{2}F_{N}^{\ast} + \cdots + a_{N-1}F_{N}W_{N}^{N-1}F_{N}^{\ast}$$

Given a vector $$x=[a_{0}\ \ a_{1}\ \ a_{2}\ \ \cdots\ \ a_{N-1}]^{\top}$$. consider the circulant matrix:

$C(x) = a_{0} + a_{1}P + a_{2}P^{2} + \cdots + a_{N-1}P^{N-1}$

If we define the polynomial

$f(x) = a_{0} + a_{1}x+a_{2}x^2+a_{3}x^3+\cdots+a_{N-1}x^{N-1},$

then $C(x) = F_{N}\begin{bmatrix} f(\omega^{0}) & 0 & 0 & 0 & \cdots & 0\\ 0 & f(\omega) & 0 & 0 & \cdots & 0\\ 0 & 0 & f(\omega^2) & 0 & \cdots & 0\\ 0 & 0 & 0 & f(\omega^3) & \cdots & 0\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 & \cdots & f(\omega^{N-1})\end{bmatrix}F_{N}^{\ast}$

### Eigenvalues of a Circulant Matrix

Example. Consider the vector $$[1\ \ 3\ \ 2]^{\top}$$, then

$=F_{3}\left(\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{bmatrix} + 3\begin{bmatrix} 1 & 0 & 0\\ 0 & \omega & 0\\ 0 & 0 & \omega^2\end{bmatrix} + 2\begin{bmatrix} 1 & 0 & 0\\ 0 & \omega^2 & 0\\ 0 & 0 & \omega^4\end{bmatrix}\right)F_{3}^{\ast}$

$=F_{3}\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{bmatrix}F_{3}^{\ast} + 3F_{3}\begin{bmatrix} 1 & 0 & 0\\ 0 & \omega & 0\\ 0 & 0 & \omega^2\end{bmatrix}F_{3}^{\ast} + 2F_{3}\begin{bmatrix} 1 & 0 & 0\\ 0 & \omega^2 & 0\\ 0 & 0 & \omega^4\end{bmatrix}F_{3}^{\ast}$

$=F_{3}\begin{bmatrix} 1+3(1)+2(1)^2 & 0 & 0\\ 0 & 1+3\omega+2\omega^2 & 0\\ 0 & 0 & 1+3(\omega^{2}) + 2(\omega^{2})^{2} \end{bmatrix} F_{3}^{\ast}$

$C(\begin{bmatrix}1\\ 3\\ 2\end{bmatrix}) = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\end{bmatrix} + 3\begin{bmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0\end{bmatrix} + 2\begin{bmatrix} 0 & 0 & 1\\ 1 & 0 & 0\\ 0 & 1 & 0\end{bmatrix} = \begin{bmatrix} 1 & 3 & 2\\ 2 & 1 & 3\\ 3 & 2 & 1\end{bmatrix}$

### Eigenvalues of a Circulant Matrix

Theorem. Given a vector $$x=[a_{0}\ \ a_{1}\ \ a_{2}\ \ \cdots\ \ a_{N-1}]^{\top}$$, define the polynomial

$g(t) = a_{0} + a_{1}t + a_{2}t^{2} + a_{3}t^{3} + \cdots + a_{N-1}t^{N-1}.$

If $$f_{k}$$ is the $$k$$th column of the Fourier Transform matrix $$F_{N}$$, then

$C(x)f_{k} = g(\omega^{k})f_{k}.$

That is, $$\{f_{0},f_{1},f_{2},\ldots,f_{N-1}\}$$ is an orthonormal basis of eigenvectors of $$C(x)$$ with associated eigenvalues $$\{g(1),g(\omega),g(\omega^{2}),\ldots,g(\omega^{N-1})\}$$.

Example.

$C(\begin{bmatrix}1\\ 3\\ 2\end{bmatrix}) = \begin{bmatrix} 1 & 3 & 2\\ 2 & 1 & 3\\ 3 & 2 & 1\end{bmatrix}$

$f_{1} = \frac{1}{\sqrt{3}}\begin{bmatrix} 1\\ \omega\\ \omega^2\end{bmatrix} = \frac{1}{\sqrt{3}}\begin{bmatrix} 1\\ -\frac{1}{2}-\frac{\sqrt{3}}{2}i\\ -\frac{1}{2}+\frac{\sqrt{3}}{2}i\end{bmatrix}$

Set $$g(x) = 1+3x+2x^2$$.

is an eigenvector with eigenvalue

$$g(\omega) = 1+3\omega+2\omega^2$$

$$=1+3(-\frac{1}{2}-\frac{\sqrt{3}}{2}i) + 2(-\frac{1}{2}+\frac{\sqrt{3}}{2}i)$$

$$=-\frac{3}{2} -\frac{\sqrt{3}}{2}i$$

### Fourier Transform of a Cyclic Convolution

Example. If

$x=\begin{bmatrix} 1\\ 0\\ 2\end{bmatrix}\quad\text{and}\quad y = \begin{bmatrix} 1\\ 5\\ 3\end{bmatrix}$

$x\circledast y = \begin{bmatrix} 11 & 11 & 5\end{bmatrix}^{\top}$

Now, we take the Fourier Transform of $$x\circledast y$$ and we get

$F_{3}(x\circledast y) = \frac{1}{\sqrt{3}}\begin{bmatrix} 1 & 1 & 1\\ 1 & \omega & \omega^2\\ 1 & \omega^2 & \omega^4\end{bmatrix} \begin{bmatrix} 11\\ 11\\ 5\end{bmatrix} = \frac{1}{\sqrt{3}} \begin{bmatrix} 11+ 11+ 5\\ 11+ 11\omega+ 5\omega^2\\ 11+ 11\omega^2+ 5\omega^4\end{bmatrix}$

$=\frac{1}{\sqrt{3}} \begin{bmatrix} 27\\ 11+ 11\omega+ 5\omega^2\\ 11+ 11\omega^2+ 5\omega\end{bmatrix} = \frac{1}{\sqrt{3}} \begin{bmatrix} 27\\ -6\omega^2\\ -6\omega \end{bmatrix}$

Remember:

• $$\omega^3=1$$
• $$1+\omega+\omega^2=0$$

Example continued. Now, we take the Fourier Transform of $$x\circledast y$$ and we get $F_{3}(x\circledast y) = \frac{1}{\sqrt{3}} \begin{bmatrix} 27\\ -6\omega^2\\ -6\omega \end{bmatrix}$

$F_{3}x = \frac{1}{\sqrt{3}} \begin{bmatrix} 3\\ 1+2\omega^2\\ 1+2\omega \end{bmatrix}\ \text{and}\ F_{3}y = \frac{1}{\sqrt{3}} \begin{bmatrix} 9\\ 1+5\omega+3\omega^2\\ 1+5\omega^2+3\omega \end{bmatrix} = \frac{1}{\sqrt{3}} \begin{bmatrix} 9\\ 4\omega+2\omega^2\\ 4\omega^2+2\omega \end{bmatrix}$

$$(1+2\omega^2)(4\omega+2\omega^2) = 4\omega+2\omega^2+8\omega^3+4\omega^4 = 8+8\omega+2\omega^2 = -6\omega^2$$

$$(1+2\omega)(4\omega^2+2\omega) = 4\omega^2+2\omega+8\omega^3+4\omega^2 = 8+2\omega+8\omega^2 = -6\omega$$

$F_{3}x\odot F_{3}y$

### Fourier Transform of a Cyclic Convolution

This is the entrywise product of two matrices of the same size. This is sometimes called the Hadamard product.

$= \frac{1}{3}\begin{bmatrix} 3\cdot 9\\ (1+2\omega^2)(4+2\omega^2)\\ (1+2\omega)(4+2\omega)\end{bmatrix} = \frac{1}{3}\begin{bmatrix} 27\\ -6\omega^2\\ -6\omega\end{bmatrix} = \frac{1}{\sqrt{3}}F_{3}(x\circledast y)$

Lemma. Let

$x=\begin{bmatrix} a_{0} & a_{1} & a_{2} & \cdots & a_{N-1}\end{bmatrix}^{\top}$

$y = \begin{bmatrix} b_{0} & b_{1} & b_{2} & \cdots & b_{N-1}\end{bmatrix}^{\top}$

Define the polynomials

$f(t) = a_{0} + a_{1}t + a_{2}t^2+\cdots + a_{N-1}t^{N-1}$

$g(t) = b_{0} + b_{1}t + b_{2}t^2+\cdots + b_{N-1}t^{N-1}$

Then,

$f(t)g(t) = c_{0} + c_{1}t+c_{2}t^2+\cdots + c_{2N-2}t^{2N-2}$

where

$[c_{0}\ \ c_{1}\ \ c_{2}\ \ \cdots\ \ c_{2N-2}]^{\top} = x\ast y.$

If $$\omega = e^{-2\pi i/N}$$, then

$f(\omega^{k})g(\omega^{k}) = d_{0} + d_{1}\omega^{k} + d_{2}\omega^{2k} + \cdots + d_{N-1}\omega^{(N-1)k}$

where

$[d_{0}\ \ d_{1}\ \ d_{2}\ \ \cdots\ \ d_{N-1}]^{\top} = x\circledast y.$

Theorem. If $$x,y\in\mathbb{C}^{N}$$, then

$F_{N}(x\circledast y) = \sqrt{N}\big(F_{N}x\odot F_{N}y\big).$

### Fourier Transform of a Cyclic Convolution

Proof. Let

$x=\begin{bmatrix} a_{0} & a_{1} & a_{2} & \cdots & a_{N-1}\end{bmatrix}^{\top}$

$y = \begin{bmatrix} b_{0} & b_{1} & b_{2} & \cdots & b_{N-1}\end{bmatrix}^{\top}$

$x\circledast y = \begin{bmatrix} d_{0} & d_{1} & d_{2} & \cdots & d_{N-1}\end{bmatrix}^{\top}$

Define the polynomials

$f(t) = a_{0} + a_{1}t + a_{2}t^2+\cdots + a_{N-1}t^{N-1}$

$g(t) = b_{0} + b_{1}t + b_{2}t^2+\cdots + b_{N-1}t^{N-1}.$

Notice that

$F_{N}x = \frac{1}{\sqrt{N}}\begin{bmatrix} f(1) & f(\omega) & f(\omega^{2}) & \cdots & f(\omega)^{N-1}\end{bmatrix}^{\top}$

$F_{N}y = \frac{1}{\sqrt{N}}\begin{bmatrix} g(1) & g(\omega) & g(\omega^{2}) & \cdots & g(\omega)^{N-1}\end{bmatrix}^{\top}$

### Fourier Transform of a Cyclic Convolution

Proof continued. Hence, if we define

$h(t) = d_{0} + d_{1}t + d_{2}t^2 + \cdots + d_{N-1}t^{N-1},$

then by the lemma we have

$F_{N}x\odot F_{N}y=$

$= \frac{1}{N}\begin{bmatrix} f(1)g(1) & f(\omega)g(\omega) & f(\omega^{2})g(\omega^{2}) & \cdots & f(\omega^{N-1})g(\omega^{N-1})\end{bmatrix}^{\top}$

$= \frac{1}{\sqrt{N}}\left(\frac{1}{\sqrt{N}}\begin{bmatrix} h(1) & h(\omega) & h(\omega)^{2} & \cdots & h(\omega^{N-1})\end{bmatrix}^{\top}\right)$

$= \frac{1}{\sqrt{N}}\left(F_{N}(x\circledast y)\right).\ \Box$

By John Jasper

• 310