det(A - \lambda I) = 0

(characteristic equation)

(desirable)

* more than 1 value can repeat - e.g. in a projection matrix both the eigenvalues 1 and 0 may repeat

GM\leq AM

GM = AM=1~~\forall \lambda

(i.e. when would GM be equal to AM)

(i.e. by finding vectors in the nullspace of

(i.e.~~if~~rank(A - \lambda I) = n-p)

A - \lambda I)

* more than 1 value can repeat - e.g. in a projection matrix both the eigenvalues 1 and 0 may repeat

we like real numbers as opposed to imaginary numbers

the best possible basis

S = Q\Lambda Q^{-1} = Q\Lambda Q^{T}

even when there are repeating eigenvalues

(we will prove this soon)

(we will prove this soon)

(all elements are real and the matrix is symmetric)

(we will not prove this completely - not even in HW5 :-))

(all elements are real and the matrix is symmetric)

S\mathbf{x} = \lambda \mathbf{x}

\therefore S\overline{\mathbf{x}} = \overline{\lambda} \overline{\mathbf{x}}

(taking conjugate on both sides, S is real)

\therefore \overline{\mathbf{x}}^\top S = \overline{\mathbf{x}}^\top\overline{\lambda}

(taking transpose on both sides, S is symmetric)

multiply both sides by conjugate of x

multiply both sides by x

\overline{\mathbf{x}}^\top S\mathbf{x} = \overline{\mathbf{x}}^\top \lambda \mathbf{x}

\overline{\mathbf{x}}^\top S\mathbf{x} = \overline{\mathbf{x}}^\top \overline{\lambda} \mathbf{x}

\therefore \lambda \overline{\mathbf{x}}^\top \mathbf{x} = \overline{\lambda} \overline{\mathbf{x}}^\top \mathbf{x}

\therefore \lambda = \overline{\lambda}

\because \overline{\mathbf{x}}^\top \mathbf{x} \neq 0

\therefore a+ib = a-ib

\implies b = 0

Hence~proved!

Where did we use the property that the matrix is real symmetric?

(all elements are real and the matrix is symmetric)

Let~~S\mathbf{x} = \lambda_1 \mathbf{x}~~and~~S\mathbf{y} = \lambda_2 \mathbf{y}~~and~~\lambda_1 \neq \lambda_2

\mathbf{x}^\top\lambda_1\mathbf{y}

= (\lambda_1\mathbf{x})^\top\mathbf{y}

= (S\mathbf{x})^\top\mathbf{y}

= \mathbf{x}^\top S^\top\mathbf{y}

= \mathbf{x}^\top S\mathbf{y}

= \mathbf{x}^\top \lambda_2\mathbf{y}

\therefore \lambda_1\mathbf{x}^\top\mathbf{y}=\lambda_2\mathbf{x}^\top\mathbf{y}

\implies \lambda_1=\lambda_2~~or~~\mathbf{x}^\top\mathbf{y}=0

But~~\lambda_1\neq\lambda_2

\therefore~~\mathbf{x}^\top\mathbf{y}=0

(Hence proved)

Proved

Proved for the case when eigenvalues are not repeating

(all elements are real and the matrix is symmetric)

Follows from Theorem 2 when the eigenvalues are not repeating

(see next few slides)

S = Q\Lambda Q^{-1} = Q\Lambda Q^{T}

A = QTQ^\top

A^\top = (Q T Q^\top)^\top = Q T^\top Q^\top

But~~A = A^\top

\therefore QTQ^\top = QT^\top Q^\top

\therefore T = T^\top

upper triang.

lower triang.

Hence, T must be diagonal

(we will not prove this)

from Schur's Theorem

symmetric

(orthogonal matrix of eigenvectors of A)

(diagonal matrix of eigenvalues of A)

Proved

Proved

therefore q_i's must be eigenvectors and d_i's must be eigenvalues

From~~corollary~~of~~Schur's~~theorem

A = QTQ^\top

\therefore AQ = QT

T is diagonal

\therefore A
\begin{bmatrix}
\uparrow&\uparrow&\uparrow \\
q_1&\dots&q_n \\
\downarrow&\downarrow&\downarrow \\
\end{bmatrix}
=
\begin{bmatrix}
\uparrow&\uparrow&\uparrow \\
q_1&\dots&q_n \\
\downarrow&\downarrow&\downarrow \\
\end{bmatrix}
\begin{bmatrix}
d_1&\dots&0 \\
0&\dots&0 \\
0&\dots&d_n \\
\end{bmatrix}

\therefore \begin{bmatrix}
\uparrow&\uparrow&\uparrow \\
Aq_1&\dots&Aq_n \\
\downarrow&\downarrow&\downarrow \\
\end{bmatrix}
=
\begin{bmatrix}
\uparrow&\uparrow&\uparrow \\
d_1q_1&\dots&d_nq_n \\
\downarrow&\downarrow&\downarrow \\
\end{bmatrix}

(even if there are repeating eigenvalues)

even if there are repeating eigenvalues

we love diagonalizability

we love real numbers

we love orthogonal basis

sorry for squeezing it in here

thus connecting the two halves of the course

det(A - \lambda I) = 0

(characteristic equation)

(desirable)

tr(A) = \sum_{i=1}^n a_{ii}

defined only for square matrices

tr(A) = \sum_{i=1}^n \lambda_{i}

det(A) = \prod_{i=1}^n \lambda_{i}

det(A - \lambda I) = 0

(characteristic equation)

(desirable)