# Day 36:

Properties of the Fourier Transform

### The Fourier Transform

Definition. Given a vector $$v\in\mathbb{C}^{N}$$, the Fourier transform of $$v$$ is the vector $$F_{N}v$$.

(Important note: In some places, including Matlab, the Fourier transform of $$v$$ is defined to be $$\sqrt{N}F_{N}v$$.)

Example. If $$v = [1\ \ 1\ \ 1\ \ 1]^{\top}$$, then the Fourier transform of $$v$$ is the vector

$F_{4}v = \frac{1}{2}\left[\begin{array}{rrrr} 1 & 1 & 1 & 1\\ 1 & -i & -1 & i\\ 1 & -1 & 1 & -1\\ 1 & i & -1 & -i\end{array}\right]\begin{bmatrix}1\\ 1\\ 1\\ 1\end{bmatrix} = \begin{bmatrix}2\\ 0\\ 0\\ 0\end{bmatrix}.$

Note that $$\|F_{4}v\| = \|v\|.$$ This always happens!

### Convolution

Motivation. Given two vectors

$x=\begin{bmatrix} a_{0} & a_{1} & a_{2} & \cdots & a_{N-1}\end{bmatrix}^{\top}$

$y = \begin{bmatrix} b_{0} & b_{1} & b_{2} & \cdots & b_{N-1}\end{bmatrix}^{\top}$

Define the polynomials

$f(x) = a_{0} + a_{1}x + a_{2}x^2+\cdots + a_{N-1}x^{N-1}$

$g(x) = b_{0} + b_{1}x + b_{2}x^2+\cdots + b_{N-1}x^{N-1}$

Then,

$f(x)g(x) = c_{0} + c_{1}x+c_{2}x^2+\cdots + c_{2N-2}x^{2N-2}$

where

$c_{0} = a_{0}b_{0},\quad c_{1} = a_{0}b_{1} + a_{1}b_{0},\quad c_{2} = a_{0}b_{2} + a_{1}b_{1} + a_{2}b_{0},\ldots$

Following this pattern:

$c_{N} = a_{0}b_{N} + a_{1}b_{N-1} + a_{2}b_{N-2} + \cdots + a_{N-1}b_{1} + a_{N}b_{0}$

If we take $$a_{i}=b_{i}=0$$ for $$i\geq N$$, then we have $c_{k} = \sum_{i=0}^{k}a_{i}b_{k-i}$

### Convolution

Definition. Given two vectors

$x=\begin{bmatrix} a_{0}\\ a_{1}\\ a_{2}\\ \vdots\\ a_{N-1}\end{bmatrix}\quad\text{and}\quad y = \begin{bmatrix} b_{0}\\ b_{1}\\ b_{2}\\ \vdots\\ b_{N-1}\end{bmatrix}$

The convolution, denoted $$x\ast y$$, is the vector

$x\ast y = \begin{bmatrix} c_{0}\\ c_{1}\\ c_{2}\\ \vdots\\ c_{2N-2}\end{bmatrix}\quad\text{where }c_{k} = \sum_{i=0}^{k}a_{i}b_{k-i}$

Note that if $$x,y\in\R^{N}$$, then $$x\ast y\in\R^{2N-1}$$.

### Convolution

Example. If

$x=\begin{bmatrix} 1\\ 0\\ 2\end{bmatrix}\quad\text{and}\quad y = \begin{bmatrix} 1\\ 5\\ 3\end{bmatrix}$

Since

$(1+2x^2)(1+5x+3x^2) = (1+2x^2)(1) + (1+2x^2)(5x)+(1+2x^2)(3x^2)$

we see that $x\ast y = \begin{bmatrix} 1 & 5 & 5 & 10 & 6\end{bmatrix}^{\top}$

$\begin{array}{ccc} 1 & 0 & 2\\ 1 & 5 & 3\\ \hline\end{array}$

$\begin{array}{ccccc} & & 3 & 0 & 6\\ & 5 & 0 & 10 & \\ 1 & 0 & 2 & & \\ \hline 1 & 5 & 5 & 10 & 6\end{array}$

$$1+0x+2x^2$$

$$5x+0x^2+10x^3$$

$$3x^2+0x^3+6x^4$$

$$= (1+2x^2) + (5x+10x^3) + (3x^2+6x^4)$$

$= 1+5x+5x^2+10x^3+6x^4$

To quickly compute $$x\ast y$$:

### Cyclic Convolution

Example. If

$x=\begin{bmatrix} 1\\ 0\\ 2\end{bmatrix}\quad\text{and}\quad y = \begin{bmatrix} 1\\ 5\\ 3\end{bmatrix}$

$x\ast y = \begin{bmatrix} 1 & 5 & 5 & 10 & 6\end{bmatrix}^{\top}$

$\begin{array}{ccc} 1 & 0 & 2\\ 1 & 5 & 3\\ \hline\end{array}$

$\begin{array}{ccccc} & & 3 & 0 & 6\\ & 5 & 0 & 10 & \\ 1 & 0 & 2 & & \\ \hline 1 & 5 & 5 & 10 & 6\end{array}$

Convolution:

$x\circledast y = \begin{bmatrix} 11 & 11 & 5\end{bmatrix}^{\top}$

$\begin{array}{ccc} 1 & 0 & 2\\ 1 & 5 & 3\\ \hline\end{array}$

$\begin{array}{ccc} 0 & 6 & 3\\ 10 & 5 & 0\\ 1 & 0 & 2\\\hline 11 & 11 & 5\end{array}$

Cyclic convolution:

### Cyclic Convolution

Definition. Given two vectors $$x,y\in\mathbb{R}^{n}$$, let

$x\ast y = \begin{bmatrix} c_{0}\\ c_{1}\\ c_{2}\\ \vdots\\ c_{2N-2}\end{bmatrix}$

Note that if $$x,y\in\R^{N}$$, then $$x\circledast y\in\R^{N}$$.

The cyclic convolution, denoted $$x\circledast y$$, is the vector

$x\circledast y = \begin{bmatrix} c_{0}+c_{N}\\ c_{1}+c_{N+1}\\ c_{2}+c_{N+2}\\ \vdots\\ c_{N-2}+c_{2N-2}\\ c_{N-1}\end{bmatrix}.$

### Upward Shift Matrix

Definition.  Let $$N\in\N$$. For each $$i=1,2,\ldots,N$$ let $$e_{i}$$ be the $$i$$th column of the $$N\times N$$ identity matrix.

The upward shift matrix is the matrix

$P = \begin{bmatrix} \vert & \vert & \vert & \vert & & \vert\\ e_{N} & e_{1} & e_{2} & e_{3} & \cdots & e_{N-1}\\ \vert & \vert & \vert & \vert & & \vert\end{bmatrix}$

Example. If $$N=4$$, then the upward shift matrix is

$\begin{bmatrix} 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\end{bmatrix}$

Importantly, $$\begin{bmatrix} 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\end{bmatrix} = \begin{bmatrix} x_{2}\\ x_{3}\\ x_{4}\\ x_{1}\end{bmatrix}$$ and hence the name.

### Upward Shift Matrix

Proposition. If $$P$$ is the $$N\times N$$ upward shift matrix, then $$P^{N} = I$$.

Proof. Let $$x = [x_{1}\ \ x_{2}\ \ x_{3}\ \ \cdots\ \ x_{N}]^{\top}$$, then

$Px = \begin{bmatrix} x_{2}\\ x_{3}\\ x_{4}\\ \vdots\\ x_{N}\\ x_{1}\end{bmatrix},\ P^{2}x = \begin{bmatrix} x_{3}\\ x_{4}\\ \vdots\\ x_{N}\\ x_{1}\\ x_{2}\end{bmatrix},\cdots, P^{N-1}x = \begin{bmatrix} x_{N}\\ x_{1}\\ x_{2}\\ x_{3}\\ \vdots \\ x_{N-1}\end{bmatrix},\ P^{N}x = \begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ \vdots \\ x_{N-1}\\ x_{N}\end{bmatrix}.\Box$

### Upward Shift Matrix

Exercise. Let $$P$$ be the $$4\times 4$$ upward shift matrix.

1. Compute $$P^{2}, P^{3},$$ and $$P^{4}$$.
2. Find the matrix $$3I + 2P+5P^2+7P^3$$.

$P = \begin{bmatrix} 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\end{bmatrix}$

$P^2 = \begin{bmatrix} 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\end{bmatrix},\ P^3 = \begin{bmatrix} 0 & 0 & 0 & 1\\ 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\end{bmatrix},\ \text{and}\ \ P^4 = \begin{bmatrix} 1 & 0 & 0 & 0\\ 0 & 1 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\end{bmatrix}$

$3I + 2P+5P^2+7P^3 = \begin{bmatrix} 3 & 2 & 5 & 7\\ 7 & 3 & 2 & 5\\ 5 & 7 & 3 & 2\\ 2 & 5 & 7 & 3\end{bmatrix}$

### Circulant Matrices

Definition. Let $$N\in\N$$, and let $$P$$ be the $$N\times N$$ upward shift matrix. A matrix $$C$$ is called circulant if there are numbers $$c_{0},c_{1},c_{2},\ldots,c_{N-1}$$ such that

$C = c_{0}I + c_{1}P + c_{2}P^{2} + \cdots + c_{N-1}P^{N-1}.$

$= \begin{bmatrix} c_{0} & c_{1} & c_{2} & c_{3} & \cdots & c_{N-1}\\ c_{N-1} & c_{0} & c_{1} & c_{2} & \cdots & c_{N-2}\\ c_{N-2} & c_{N-1} & c_{0} & c_{1} & \cdots & c_{N-3}\\ c_{N-3} & c_{N-2} & c_{N-1} & c_{0} & \cdots & c_{N-4}\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ c_{1} & c_{2} & c_{3} & c_{4} & \cdots & c_{0}\end{bmatrix}$

A circulant matrix is a matrix with all of its diagonals constant.

### Circulant Matrices

Example. Consider the $$3\times 3$$ circulant matrices

$C = I + 2P^{2} = \begin{bmatrix} 1 & 0 & 2\\ 2 & 1 & 0\\ 0 & 2 & 1\end{bmatrix}$

and

$D = I +5P + 3P^{2} = \begin{bmatrix} 1 & 5 & 3\\ 3 & 1 & 5\\ 5 & 3 & 1\end{bmatrix}$

Find $$CD$$ and $$DC$$. How do these matrices relate to $[1\ \ 0\ \ 2]^{\top}\circledast [1\ \ 5\ \ 3]^{\top}?$

### Circulant Matrices

Definition. Let $$N\in\N$$, and let $$P$$ be the $$N\times N$$ upward shift matrix. Given a vector $$x=[a_{0}\ \ a_{1}\ \ a_{2}\ \ \cdots\ \ a_{N-1}]^{\top}$$, define the circulant matrix

$C(x) = a_{0}I + a_{1}P + a_{2}P^{2} + \cdots + a_{N-1}P^{N-1}.$

Lemma. Given $$x,y\in\mathbb{C}^{n}$$ we have

$C(x)C(y) = C(x\circledast y)$

Proof.

## Prove it!

### Circulant Matrices

Lemma. Given $$x,y\in\mathbb{C}^{n}$$ we have

$C(x)C(y) = C(x\circledast y)$

Proof. Let $$x = [a_{0}\ \ a_{1}\ \ a_{2}\ \ \cdots\ \ a_{n-1}]^{\top}$$, $$y=[b_{0}\ \ b_{1}\ \ b_{2}\ \ \cdots\ \ b_{n-1}]^{\top}$$, and $$x\ast y = [c_{0}\ \ c_{1}\ \ c_{2}\ \ \cdots\ \ c_{2n-2}]^{\top}$$, then

$$= (a_{0}I + a_{1}P + a_{2}P^{2} + \cdots + a_{n-1}P^{n-1})(b_{0}I + b_{1}P + b_{2}P^{2} + \cdots + b_{n-1}P^{n-1})$$

$$= c_{0}I+c_{1}P+c_{2}P^{2} + \cdots + c_{n-1}P^{n-1}$$

$$\quad + c_{n}I + c_{n+1}P + c_{n+2}P^{2} +\cdots + c_{2n-2}P^{n-2}$$

$$= c_{0}I+c_{1}P+c_{2}P^{2} + \cdots + c_{n-1}P^{n-1}$$

$$\quad + c_{n}P^{n} + c_{n+1}P^{n+1} + c_{n+2}P^{n+2} +\cdots + c_{2n-2}P^{2n-2}$$

$$C(x)C(y)$$

$$= (c_{0}+c_{n})I+(c_{1}+c_{n+1})P+(c_{2}+c_{n+2})P^{2} + \cdots +(c_{n-2}+c_{2n-2})P^{n-2}$$

$$\quad+c_{n-1}P^{n-1}.\ \Box$$

### Eigenvalues/vectors of the upward shift

Lemma. If $$P$$ is the $$N\times N$$ upward shift matrix, and $$f_{k}$$ is the $$k$$th column of the Fourier Transform matrix $$F_{N}$$, then

$Pf_{k} = e^{-2k\pi i/N}f_{k} = \omega^{k}f_{k}.$

Proof.  $f_{k} = \frac{1}{\sqrt{N}}\begin{bmatrix} 1\\ \omega^k\\ \omega^{2k}\\ \omega^{3k}\\ \vdots\\ \omega^{(N-1)k}\end{bmatrix}$

$Pf_{k} = \frac{1}{\sqrt{N}}\begin{bmatrix} \omega^k\\ \omega^{2k}\\ \omega^{3k}\\ \vdots\\ \omega^{(N-1)k}\\ 1\end{bmatrix} = \frac{\omega^{k}}{\sqrt{N}}\begin{bmatrix} \omega^0\\ \omega^{k}\\ \omega^{2k}\\ \vdots\\ \omega^{(N-2)k}\\ \omega^{-k}\end{bmatrix} = \omega^{k}\frac{1}{\sqrt{N}}\begin{bmatrix} 1\\ \omega^{k}\\ \omega^{2k}\\ \vdots\\ \omega^{(N-2)k}\\ \omega^{Nk}\omega^{-k}\end{bmatrix} = \omega^{k}f_{k}$

### Eigenvalues/vectors of the upward shift

By the lemma, the columns of $$F_{N}$$ are an orthonormal basis of eigenvectors of the $$N\times N$$ upward shift matrix $$P$$, thus

$P = F_{N}\begin{bmatrix} \omega^{0} & 0 & 0 & 0 & \cdots & 0\\ 0 & \omega & 0 & 0 & \cdots & 0\\ 0 & 0 & \omega^2 & 0 & \cdots & 0\\ 0 & 0 & 0 & \omega^3 & \cdots & 0\\ \vdots & \vdots & \vdots & \vdots & \ddots & \vdots\\ 0 & 0 & 0 & 0 & \cdots & \omega^{N-1}\end{bmatrix}F_{N}^{\ast}$

By John Jasper

• 315