CS6015: Linear Algebra and Random Processes
Lecture 10: The four fundamental subspaces
Learning Objectives
What are the four fundamental subspaces of a matrix?
How do you find a basis for each of these subspaces?
What are the dimensions of these subspaces?
(for today's lecture)
The 4 fundamental subspaces of \(A_{m\times n}\)
Column space of \(A\): \( \mathcal{C}(A) \)
Span (linear combinations) of the columns of \( A \)
Nullspace of \(A\): \( \mathcal{N}(A) \)
Solutions to \(A\mathbf{x} = \mathbf{0} \)
Row space of \(A = \) Column space of \(A^\top\): \( \mathcal{C}(A^\top) \)
Span (linear combinations) of the columns of \( A^\top \) (or rows of \(A\))
Nullspace of \(A^\top\): \( \mathcal{N}(A^\top) \)
Solutions to \(A^\top\mathbf{x} = \mathbf{0} \)
each vector \(\in \mathbb{R}^m \)
each vector \(\in \mathbb{R}^n \)
each vector \(\in \mathbb{R}^n \)
each vector \(\in \mathbb{R}^m \)
Where are these subspaces?
\mathcal{C}(A)
\mathcal{N}(A^\top)
inside~\mathbb{R}^m
\mathcal{C}(A^\top)
\mathcal{N}(A)
inside~\mathbb{R}^n
What are their dimensions?
\mathcal{C}(A)
\mathcal{N}(A^\top)
inside~\mathbb{R}^m
\mathcal{C}(A^\top)
\mathcal{N}(A)
inside~\mathbb{R}^n
dim=r
dim=n-r
dim=r
dim=m-r
rank nullity theorem
row rank = column rank
What are their basis vectors?
Basis for \( \mathcal{C}(A) \)
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
A
U
R
Pivot columns of \(A\)
Basis for \( \mathcal{N}(A) \)
Special solutions for \(A\mathbf{x} = \mathbf{0} \)
\begin{bmatrix}
1&2\\
-2&-3\\
1&0\\
0&1
\end{bmatrix}
X
Basis for \( \mathcal{C}(A^\top) \): row space of A
Pivot rows of \(A\)
A cleaner basis: Pivot rows of \(R\)
What are their basis vectors?
Basis for \( \mathcal{N}(A^\top) \)
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
A
U
R
The rows of \(E\) corresponding to the 0 rows of \( R \)
\begin{bmatrix}
A & I
\end{bmatrix}
E
How did we get \(R \) ?
= \begin{bmatrix}
R & E
\end{bmatrix}
r2 = r2 - r1
r3 = r3 - 2r1
r3 = r3 - r2
\begin{bmatrix}
1&0&0\\
0&1&0\\
0&0&1
\end{bmatrix}
\begin{bmatrix}
1&0&0\\
-1&1&0\\
-1&-1&1
\end{bmatrix}
r1 = r1 - r2
\begin{bmatrix}
2&-1&0\\
-1&1&0\\
-1&-1&1
\end{bmatrix}
This linear combination of the rows of A produce a 0 vector (hence it is a solution to \(A^\top\mathbf{x} = \mathbf{0} \) )
E
if we apply these operations to \( I \) we will get \(E\)
What are their basis vectors?
Basis for \( \mathcal{N}(A^\top) \)
A
U
R
The rows of \(E\) corresponding to the 0 rows of \( R \)
\begin{bmatrix}
A & I
\end{bmatrix}
E
How did we get \(R \) ?
= \begin{bmatrix}
R & E
\end{bmatrix}
r2 = r2 - r1
r3 = r3 - r1
\begin{bmatrix}
1&0&0&0\\
0&1&0&0\\
0&0&1&0\\
0&0&0&1
\end{bmatrix}
r3 = r3 - 2r2
These linear combinations of the rows of A produce the 0 vector (hence they are solutions to \(A^\top\mathbf{x} = \mathbf{0} \) )
E
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\begin{bmatrix}
1&0&1\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
r4 = r4 - r1
r4 = r4 - 3r2
r1 = r1 - r2
\begin{bmatrix}
1&0&0&0\\
-1&1&0&0\\
1&-2&1&0\\
2&-3&0&1
\end{bmatrix}
\begin{bmatrix}
2&-1&0&0\\
-1&1&0&0\\
1&-2&1&0\\
2&-3&0&1
\end{bmatrix}
(another example)
if we apply these operations to \( I \) we will get \(E\)
Practice Problems
For each of the above matrices, find the dimensions and the basis of the 4 fundamental subspaces
\begin{bmatrix}
1&2&3\\
1&1&2\\
2&1&0
\end{bmatrix}
\begin{bmatrix}
1&2&3&3&2&1\\
1&2&2&1&1&2\\
2&0&0&1&2&2
\end{bmatrix}
\begin{bmatrix}
0&2&2\\
1&2&1\\
2&1&3\\
3&1&1\\
-1&1&-2\\
2&1&0\\
\end{bmatrix}
\begin{bmatrix}
1&2&3&3&2&1\\
2&1&0&1&2&2\\
4&5&6&7&6&4
\end{bmatrix}
\( \mathcal{N}(A^\top) \) is left null space of \(A\)
A^\top \mathbf{y} = 0
Let~\mathbf{y}\in \mathcal{N}(A^\top)
Taking transpose on both sides
\mathbf{y}^\top A = 0^\top
Hence it is in the left null space of A (multiplying on the left by A gives the 0 vector)
Learning Objectives
What are the four fundamental subspaces of a matrix?
How do you find a basis for each of these subspaces?
What are the dimensions of these subspaces?
(achieved)
CS6015: Lecture 10
By Mitesh Khapra
CS6015: Lecture 10
Lecture 10: The four fundamental subspaces
- 2,437
