CS6015: Linear Algebra and Random Processes

Lecture 10: The four fundamental subspaces

Learning Objectives

What are the four fundamental subspaces of a matrix?

How do you find a basis for each of these subspaces?

What are the dimensions of these subspaces?

(for today's lecture)

The 4 fundamental subspaces of \(A_{m\times n}\)

Column space of \(A\): \( \mathcal{C}(A) \)

Span (linear combinations) of the columns of \( A \)

Nullspace of \(A\): \( \mathcal{N}(A) \)

Solutions to \(A\mathbf{x} = \mathbf{0} \)

Row space of \(A = \) Column space of \(A^\top\): \( \mathcal{C}(A^\top) \)

Span (linear combinations) of the columns of \( A^\top \) (or rows of \(A\))

Nullspace of \(A^\top\): \( \mathcal{N}(A^\top) \)

Solutions to \(A^\top\mathbf{x} = \mathbf{0} \)

each vector \(\in \mathbb{R}^m \) 

each vector \(\in \mathbb{R}^n \) 

each vector \(\in \mathbb{R}^n \) 

each vector \(\in \mathbb{R}^m \) 

Where are these subspaces?

\mathcal{C}(A)
\mathcal{N}(A^\top)
inside~\mathbb{R}^m
\mathcal{C}(A^\top)
\mathcal{N}(A)
inside~\mathbb{R}^n

What are their dimensions?

\mathcal{C}(A)
\mathcal{N}(A^\top)
inside~\mathbb{R}^m
\mathcal{C}(A^\top)
\mathcal{N}(A)
inside~\mathbb{R}^n
dim=r
dim=n-r
dim=r
dim=m-r

rank nullity theorem

row rank = column rank

What are their basis vectors?

Basis for \( \mathcal{C}(A) \)

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&0&-1&-2\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&1&1&1\\ 1&2&3&4\\ 2&3&4&5 \end{bmatrix}
A
U
R

Pivot columns of \(A\)

Basis for \( \mathcal{N}(A) \)

Special solutions for \(A\mathbf{x} = \mathbf{0} \)

\begin{bmatrix} 1&2\\ -2&-3\\ 1&0\\ 0&1 \end{bmatrix}
X

Basis for \( \mathcal{C}(A^\top) \): row space of A

Pivot rows of \(A\)

A cleaner basis: Pivot rows of \(R\)

What are their basis vectors?

Basis for \( \mathcal{N}(A^\top) \)

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&0&-1&-2\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&1&1&1\\ 1&2&3&4\\ 2&3&4&5 \end{bmatrix}
A
U
R

The rows of \(E\) corresponding to the 0 rows of \( R \)

\begin{bmatrix} A & I \end{bmatrix}
E

How did we get \(R \) ?

= \begin{bmatrix} R & E \end{bmatrix}
r2 = r2 - r1
r3 = r3 - 2r1
r3 = r3 - r2
\begin{bmatrix} 1&0&0\\ 0&1&0\\ 0&0&1 \end{bmatrix}
\begin{bmatrix} 1&0&0\\ -1&1&0\\ -1&-1&1 \end{bmatrix}
r1 = r1 - r2
\begin{bmatrix} 2&-1&0\\ -1&1&0\\ -1&-1&1 \end{bmatrix}

This linear combination of the rows of A produce a 0 vector (hence it is a solution to \(A^\top\mathbf{x} = \mathbf{0} \) )

E

if we apply these operations to \( I \) we will get \(E\)

What are their basis vectors?

Basis for \( \mathcal{N}(A^\top) \)

A
U
R

The rows of \(E\) corresponding to the 0 rows of \( R \)

\begin{bmatrix} A & I \end{bmatrix}
E

How did we get \(R \) ?

= \begin{bmatrix} R & E \end{bmatrix}
r2 = r2 - r1
r3 = r3 - r1
\begin{bmatrix} 1&0&0&0\\ 0&1&0&0\\ 0&0&1&0\\ 0&0&0&1 \end{bmatrix}
r3 = r3 - 2r2

These linear combinations of the rows of A produce the 0 vector (hence they are solutions to \(A^\top\mathbf{x} = \mathbf{0} \) )

E
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\begin{bmatrix} 1&0&1\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
r4 = r4 - r1
r4 = r4 - 3r2
r1 = r1 - r2
\begin{bmatrix} 1&0&0&0\\ -1&1&0&0\\ 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}
\begin{bmatrix} 2&-1&0&0\\ -1&1&0&0\\ 1&-2&1&0\\ 2&-3&0&1 \end{bmatrix}

(another example)

if we apply these operations to \( I \) we will get \(E\)

Practice Problems

For each of the above matrices, find the dimensions and the basis of the 4 fundamental subspaces

\begin{bmatrix} 1&2&3\\ 1&1&2\\ 2&1&0 \end{bmatrix}
\begin{bmatrix} 1&2&3&3&2&1\\ 1&2&2&1&1&2\\ 2&0&0&1&2&2 \end{bmatrix}
\begin{bmatrix} 0&2&2\\ 1&2&1\\ 2&1&3\\ 3&1&1\\ -1&1&-2\\ 2&1&0\\ \end{bmatrix}
\begin{bmatrix} 1&2&3&3&2&1\\ 2&1&0&1&2&2\\ 4&5&6&7&6&4 \end{bmatrix}

\( \mathcal{N}(A^\top) \) is left null space of \(A\)

A^\top \mathbf{y} = 0
Let~\mathbf{y}\in \mathcal{N}(A^\top)

Taking transpose on both sides

\mathbf{y}^\top A = 0^\top

Hence it is in the left null space of A (multiplying on the left by A gives the 0 vector)

Learning Objectives

What are the four fundamental subspaces of a matrix?

How do you find a basis for each of these subspaces?

What are the dimensions of these subspaces?

(achieved)