# Day 5:

Finding bases for subspaces

### Bases

Definition 4. Let $$V$$ be a subspace. A basis for $$V$$ is a collection of vectors $$\{v_{1},v_{2},\ldots,v_{r}\}\subset V$$ with the following properties:

1. If $$v\in V$$, then there are scalars $$a_{1},\ldots,a_{r}$$ such that $a_{1}v_{1} + a_{2}v_{2} + \cdots + a_{r}v_{r} = v$
2. If $b_{1}v_{1}+b_{2}v_{2}+\cdots+b_{r}v_{r} = 0$ for some scalars $$b_{1},\ldots,b_{r}$$, then $$b_{1}=b_{2}=\cdots=b_{r}=0$$.

If a sequence of vectors satisfy Property 1, then we say that $$\{v_{1},v_{2},\ldots,v_{r}\}$$ spans $$V$$.

If a sequence of vectors satisfy Property 2, then we say that $$\{v_{1},v_{2},\ldots,v_{r}\}$$ is linearly independent.

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

Note that the vectors

$\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}$

span $$C(A)$$.

However, they are not independent, since

$1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}+(-1)\cdot \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}+17\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}+0\cdot\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix}0\\ 0\\ 0\end{bmatrix}$

However, they are not independent, since

$\begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 17\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

Theorem. Let $$x_{1},\ldots,x_{m}$$ be a collection of vectors in $$\R^{n}$$.

If $$x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}$$, then

$\text{span}\{x_{1},\ldots,x_{m-1}\} = \text{span}\{x_{1},\ldots,x_{m}\}.$

Proof. The assumption that $$x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}$$ means that there are scalars $$a_{1},\ldots,a_{m-1}$$ so that

$x_{m} = a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}.$

It is clear that

$\text{span}\{x_{1},\ldots,x_{m-1}\} \subseteq \text{span}\{x_{1},\ldots,x_{m}\}.$

If $$y\in\text{span}\{x_{1},\ldots,x_{m}\}$$, then there are scalars $$b_{1},\ldots,b_{m}$$ so that

$= b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}\big(a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}\big)$

$= (b_{1}+a_{1}b_{m})x_{1} + (b_{2}+a_{2}b_{m})x_{2} + \cdots + (b_{m-1}+a_{m-1}b_{m})x_{m-1}$

$y = b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}x_{m}$

This shows that $$y\in\text{span}\{x_{1},\ldots,x_{m-1}\}$$, and thus $\text{span}\{x_{1},\ldots,x_{m-1}\} \supseteq \text{span}\{x_{1},\ldots,x_{m}\}.$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

Note that the vectors

$\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}$

span $$C(A)$$.

Next, note that

$1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}$

$\text{and}\quad 1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+(-1)\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}$

Spans $$C(A)$$, but is it independent?

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}$

We want to find the solutions to $\begin{bmatrix} 1 & 1\\ 2 & 1\\ 3 & 2\end{bmatrix} \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

or equivalently, the system of equations $\left\{\begin{array}{ccccc} x_{1} & + & x_{2} & = & 0\\ 2x_{2} & + & x_{2} & = & 0\\ 3x_{1} & + & 2x_{2} & = & 0\end{array}\right.$

The only solution is $$x_{1} = x_{2} =0$$. We have found a basis for $$C(A)$$!

Theorem. Let $$A$$ be a matrix and let $$\text{rref}(A)$$ be the reduced row echelon form of the matrix. For a vector $$x$$ we have $$Ax=\mathbf{0}$$ if and only if $$\text{rref}(A)x=\mathbf{0}$$.

Proof. Let $$B$$ be the matrix so that

$BA = \text{rref}(A),$

and let $$C$$ be the matrix so that

$C\text{rref}(A) = A.$

Assume $$x$$ is a vector such that $$Ax=\mathbf{0}$$. Then we have

$\text{rref}(A)x = BAx = B(\mathbf{0}) = \mathbf{0}.$

On the other hand, if $$\text{rref}(A)x = \mathbf{0}$$, then we have

$Ax = C\text{rref}(A)x = C(\text{rref}(A)x) = C\mathbf{0} = \mathbf{0}.\ \Box$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

$\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

$\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + 1\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

$\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

$\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}$

$\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}$

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

Conclusion: The columns in the original matrix corresponding to the pivot columns in $$\operatorname{rref}(A)$$ form a spanning set for $$C(A)$$.

Hence, $$\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}$$ is a spanning set for $$C(A)$$.

Example. Consider the matrix

$A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}$

Find a basis for the column space $$C(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

Is $$\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}$$ independent?

If they were dependent, then there would be a nonzero vector $x=\begin{bmatrix} a & b & 0 & 0 & 0\end{bmatrix}^{\top}$ such that $$Ax=0$$. However, then it would also be the case that $$\operatorname{rref}(A)x = 0$$, but distinct pivot columns are always independent.

YES!

Theorem. Let $$A$$ be an $$m\times n$$ matrix with columns $$a_{1},a_{2},\ldots,a_{n}$$, and let $$\text{rref}(A)$$ be the reduced row echelon form of $$A$$. If the pivots of $$\text{rref}(A)$$ are in columns $$n_{1},n_{2},\ldots,n_{r}$$, then $$a_{n_{1}},a_{n_{2}},\ldots,a_{n_{r}}$$ is a basis for $$C(A)$$.

To summarize what we just learned, we have the following theorem:

Exercise. Find a basis for $$C(A)$$ where

$A = \begin{bmatrix} 1 & -1 & 1 & 0 & 2\\ 1 & -1 & 1 & 0 & 2\\ -1 & 1 & 0 & 1 & -1\end{bmatrix}$

$\text{rref}(A) = \begin{bmatrix} 1 & -1 & 0 & -1 & 1\\ 0 & 0 & 1 & 1 & 1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}$

So, the vectors $\left\{\begin{bmatrix}1\\ 1\\ -1\end{bmatrix}, \begin{bmatrix}1\\ 1\\ 0\end{bmatrix}\right\}$

form a basis for $$C(A)$$

By John Jasper

• 454