# The Eigenstory

### Permutation

det(A - \lambda I) = 0

### (where are we?)

(characteristic equation)
(desirable)

# The bigger picture

* more than 1 value can repeat - e.g. in a projection matrix both the eigenvalues 1 and 0 may repeat

# Multiplicity

GM\leq AM

### The good case (no repeating eigenvalues)

GM = AM=1~~\forall \lambda

# Multiplicity

### If an eigenvalue repeats $$p$$ times and the dimension of the nullspace of $$(A-\lambda I)$$ is also $$p$$ then GM = AM

(i.e. when would GM be equal to AM)
(i.e. by finding vectors in the nullspace of
(i.e.~~if~~rank(A - \lambda I) = n-p)
A - \lambda I)

# The bigger picture

* more than 1 value can repeat - e.g. in a projection matrix both the eigenvalues 1 and 0 may repeat

# Real Symmetric matrices

we like real numbers as opposed to imaginary numbers

### *

the best possible basis
S = Q\Lambda Q^{-1} = Q\Lambda Q^{T}
even when there are repeating eigenvalues
(we will prove this soon)
(we will prove this soon)
(all elements are real and the matrix is symmetric)
(we will not prove this completely - not even in HW5 :-))

# Real Symmetric matrices

(all elements are real and the matrix is symmetric)

### $$\overline{\lambda} = a - ib$$

S\mathbf{x} = \lambda \mathbf{x}
\therefore S\overline{\mathbf{x}} = \overline{\lambda} \overline{\mathbf{x}}
(taking conjugate on both sides, S is real)
\therefore \overline{\mathbf{x}}^\top S = \overline{\mathbf{x}}^\top\overline{\lambda}
(taking transpose on both sides, S is symmetric)
multiply both sides by conjugate of x
multiply both sides by x
\overline{\mathbf{x}}^\top S\mathbf{x} = \overline{\mathbf{x}}^\top \lambda \mathbf{x}
\overline{\mathbf{x}}^\top S\mathbf{x} = \overline{\mathbf{x}}^\top \overline{\lambda} \mathbf{x}
\therefore \lambda \overline{\mathbf{x}}^\top \mathbf{x} = \overline{\lambda} \overline{\mathbf{x}}^\top \mathbf{x}
\therefore \lambda = \overline{\lambda}
\because \overline{\mathbf{x}}^\top \mathbf{x} \neq 0
\therefore a+ib = a-ib
\implies b = 0
Hence~proved!
Where did we use the property that the matrix is real symmetric?

# Real Symmetric matrices

(all elements are real and the matrix is symmetric)

### Proof:

Let~~S\mathbf{x} = \lambda_1 \mathbf{x}~~and~~S\mathbf{y} = \lambda_2 \mathbf{y}~~and~~\lambda_1 \neq \lambda_2
\mathbf{x}^\top\lambda_1\mathbf{y}
= (\lambda_1\mathbf{x})^\top\mathbf{y}
= (S\mathbf{x})^\top\mathbf{y}
= \mathbf{x}^\top S^\top\mathbf{y}
= \mathbf{x}^\top S\mathbf{y}
= \mathbf{x}^\top \lambda_2\mathbf{y}
\therefore \lambda_1\mathbf{x}^\top\mathbf{y}=\lambda_2\mathbf{x}^\top\mathbf{y}
\implies \lambda_1=\lambda_2~~or~~\mathbf{x}^\top\mathbf{y}=0
But~~\lambda_1\neq\lambda_2
\therefore~~\mathbf{x}^\top\mathbf{y}=0
(Hence proved)

# Real Symmetric matrices

Proved

### *

Proved for the case when eigenvalues are not repeating
(all elements are real and the matrix is symmetric)
Follows from Theorem 2 when the eigenvalues are not repeating

### $$n$$ orthogonal eigenvectors

(see next few slides)
S = Q\Lambda Q^{-1} = Q\Lambda Q^{T}

# Schur's Theorem

### Proof:

A = QTQ^\top
A^\top = (Q T Q^\top)^\top = Q T^\top Q^\top
But~~A = A^\top
\therefore QTQ^\top = QT^\top Q^\top
\therefore T = T^\top
upper triang.
lower triang.
Hence, T must be diagonal
(we will not prove this)
from Schur's Theorem
symmetric

# Spectral Theorem

### $$A$$ can be diagonalised as $$Q D Q^\top$$

(orthogonal matrix of eigenvectors of A)
(diagonal matrix of eigenvalues of A)

### *

Proved
Proved
therefore q_i's must be eigenvectors and d_i's must be eigenvalues
From~~corollary~~of~~Schur's~~theorem

### Proof (of 3rd part):

A = QTQ^\top
\therefore AQ = QT
T is diagonal
\therefore A \begin{bmatrix} \uparrow&\uparrow&\uparrow \\ q_1&\dots&q_n \\ \downarrow&\downarrow&\downarrow \\ \end{bmatrix} = \begin{bmatrix} \uparrow&\uparrow&\uparrow \\ q_1&\dots&q_n \\ \downarrow&\downarrow&\downarrow \\ \end{bmatrix} \begin{bmatrix} d_1&\dots&0 \\ 0&\dots&0 \\ 0&\dots&d_n \\ \end{bmatrix}
\therefore \begin{bmatrix} \uparrow&\uparrow&\uparrow \\ Aq_1&\dots&Aq_n \\ \downarrow&\downarrow&\downarrow \\ \end{bmatrix} = \begin{bmatrix} \uparrow&\uparrow&\uparrow \\ d_1q_1&\dots&d_nq_n \\ \downarrow&\downarrow&\downarrow \\ \end{bmatrix}
(even if there are repeating eigenvalues)
even if there are repeating eigenvalues

# Spectral Theorem

### with orthogonal eigenbasis

we love diagonalizability
we love real numbers
we love orthogonal basis

### One last property about symmetric matrices

sorry for squeezing it in here

### number of positive pivots = number of positive eigenvalues

thus connecting the two halves of the course

# The Eigenstory

### Permutation

det(A - \lambda I) = 0

### (where are we?)

(characteristic equation)
(desirable)

# Two properties

### The trace of a matrix is the sum of the diagonal elements of the matrix

tr(A) = \sum_{i=1}^n a_{ii}
defined only for square matrices

### If $$\lambda_1, \lambda_2, \dots, \lambda_n$$ are the eigenvalues of a matrix $$A$$ then

tr(A) = \sum_{i=1}^n \lambda_{i}
det(A) = \prod_{i=1}^n \lambda_{i}

# The Eigenstory

### Permutation

det(A - \lambda I) = 0

### (where are we?)

(characteristic equation)
(desirable)