CS6015: Linear Algebra and Random Processes
Lecture 17: Change of basis
Learning Objectives
How do you translate a point from one basis to another?
How do you translate a transformation from one basis to another?
The Eigenstory
real
imaginary
distinct
repeating
\(A^\top\)
\(A^{-1}\)
\(AB\)
\(A^\top A\)
(basis)
powers of A
PCA
optimisation
diagonalisation
\(A+B\)
\(U\)
\(R\)
\(A^2\)
\(A + kI\)
How to compute eigenvalues?
What are the possible values?
What are the eigenvalues of some special matrices ?
What is the relation between the eigenvalues of related matrices?
What do eigen values reveal about a matrix?
What are some applications in which eigenvalues play an important role?
Identity
Projection
Reflection
Markov
Rotation
Singular
Orthogonal
Rank one
Symmetric
Permutation
det(A - \lambda I) = 0
trace
determinant
invertibility
rank
nullspace
columnspace
(positive semidefinite matrices)
positive pivots
(independent eigenvectors)
(orthogonal eigenvectors)
... ...
(symmetric)
(where are we?)
(characteristic equation)
(desirable)
HW5
distinct values
independent eigenvectors
\(\implies\)
steady state
(Markov matrices)
... ... but before that a slight detour to understand "change of basis"
Basis \(\implies\) some coordinate system
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
Standard basis:
\mathbf{y}=\begin{bmatrix}
0\\1
\end{bmatrix}
\mathbf{x}=\begin{bmatrix}
1\\0
\end{bmatrix}
Every vector is expressed as a linear combination of these two vectors
\begin{bmatrix}
a\\b
\end{bmatrix}=a\mathbf{x} + b\mathbf{y}
\begin{bmatrix}
1\\1
\end{bmatrix}=1\cdot\mathbf{x} + 1\cdot\mathbf{y}
But we are free to choose any other basis
a, b are the coordinates in this coordinate system
\begin{bmatrix}
1\\1
\end{bmatrix}
Free to choose any basis
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
Alternative basis:
\mathbf{\hat{y}}=\begin{bmatrix}
-1\\1
\end{bmatrix}
\mathbf{\hat{x}}=\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
Every vector is expressed as a linear combination of these two vectors
\begin{bmatrix}
a\\b
\end{bmatrix}=a\mathbf{\hat{x}} + b\mathbf{\hat{y}}
\begin{bmatrix}
1\\1
\end{bmatrix}\implies1\cdot\mathbf{\hat{x}} + 1\cdot\mathbf{\hat{y}}
a, b are the coordinates in this coordinate system
Same coordinates, different points
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
The vectors \(\begin{bmatrix}1\\1\end{bmatrix}\) are not the same in this two basis
\begin{bmatrix}
1\\1
\end{bmatrix}\implies1\cdot\mathbf{\hat{x}} + 1\cdot\mathbf{\hat{y}}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}\implies1\cdot\mathbf{x} + 1\cdot\mathbf{y}
=\begin{bmatrix}
1\\2
\end{bmatrix}
=\begin{bmatrix}
1\\1
\end{bmatrix}
Translating from one basis to another
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\begin{bmatrix}
2\\1
\end{bmatrix}
Vector in new basis
\mathbf{\hat{a}} = 1\begin{bmatrix}
2\\1
\end{bmatrix} + 1\begin{bmatrix}
-1\\1
\end{bmatrix}
\mathbf{\hat{a}}
\mathbf{\hat{a}} = \begin{bmatrix}
2&-1\\1&1
\end{bmatrix}\begin{bmatrix}
1\\1
\end{bmatrix}
new basis
Vector in std. basis
(to the standard basis)
Translating form one basis to another
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\begin{bmatrix}
2\\1
\end{bmatrix}
=\begin{bmatrix}
1\\1
\end{bmatrix}
A=\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}
This is the matrix which took us from our new basis to the standard basis
(the other direction )
\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}\mathbf{\hat{a}}
=\mathbf{a}
Vector in new basis
Vector in std basis
Suppose you are given a vector in the std. basis then how do you represent it in the new basis?
\mathbf{\hat{a}}=\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}^{-1}
\mathbf{a}
Translating form one basis to another
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\begin{bmatrix}
2\\1
\end{bmatrix}
=\begin{bmatrix}
1\\1
\end{bmatrix}
This is the matrix which took us from our standard basis to the new basis
(the other direction )
\mathbf{\hat{a}}=\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}^{-1}
\begin{bmatrix}
1\\1
\end{bmatrix}
\mathbf{\hat{a}}=\frac{1}{3}\begin{bmatrix}
1&1\\-1&2
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
=\begin{bmatrix}
\frac{2}{3}\\~\\\frac{1}{3}
\end{bmatrix}
A=\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
\frac{2}{3}\begin{bmatrix}
2\\1
\end{bmatrix}
+~\frac{1}{3}\begin{bmatrix}
-1\\1
\end{bmatrix}
=\begin{bmatrix}
1\\1
\end{bmatrix}
\frac{2}{3}
\frac{1}{3}
Translating transformations
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
This transformation rotates a vector in the standard basis by 90 degrees
A=\begin{bmatrix}
0&-1\\1&0
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
What would the corresponding matrix be for the new basis?
(i.e., if v is a vector in the new basis, what is the matrix that will rotate it by 90 degrees)
\begin{bmatrix}
-1\\1
\end{bmatrix}
\mathbf{v} = \begin{bmatrix}a\\b\end{bmatrix}
a
b
\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
Translating transformations
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\mathbf{v}
a
b
\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
A vector in the new basis
\underbrace{~~~~~~~~~~~~~~~~~}
the same vector represented in the standard basis
\begin{bmatrix}
0&-1\\1&0
\end{bmatrix}
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
the desired transformation applied in the standard basis
\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}^{-1}
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
The result translated back to the new basis
\mathbf{v} = \begin{bmatrix}a\\b\end{bmatrix}
Translating transformations
\begin{bmatrix}
1\\0
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
\begin{bmatrix}
2&-1\\1&1
\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
a
b
\begin{bmatrix}
2\\1
\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
A vector in the new basis
\underbrace{~~~~~~~~~~~~~~~~~}
the same vector represented in the standard basis
\begin{bmatrix}
0&-1\\1&0
\end{bmatrix}
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
the desired transformation applied in the standard basis
\begin{bmatrix}
1&1\\-1&2
\end{bmatrix}
\underbrace{~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~}
The result translated back to the new basis
\mathbf{v} = \begin{bmatrix}\frac{2}{3}\\\frac{1}{3}\end{bmatrix}
\frac{1}{3}
\begin{bmatrix}\frac{2}{3}\\\frac{1}{3}\end{bmatrix}
\begin{bmatrix}
1\\1
\end{bmatrix}
\begin{bmatrix}\frac{2}{3}\\\frac{1}{3}\end{bmatrix}
\begin{bmatrix}
-1\\1
\end{bmatrix}
\begin{bmatrix}
0\\1
\end{bmatrix}
Translating transformations
B^{-1}AB\mathbf{x}
New Basis
Desired transformation
Vector in new basis
(we will see this form soon)
Puzzle: If \(A\) is the transformation that we are interested in then in what situation would we prefer the basis \(B\) as opposed to the standard basis?
Hint: What if \(B^{-1}AB\) was "simpler" than \(A\) ?
(of course, you need to think what simpler means - that's a part of the puzzle!)
Learning Objectives (achieved)
How do you translate a point from one basis to another?
How do you translate a transformation from one basis to another?
CS6015: Lecture 17
By Mitesh Khapra
CS6015: Lecture 17
Lecture 17: Change of basis
- 1,569
