Day 5:

Finding bases for subspaces

Bases

Definition 4. Let \(V\) be a subspace. A basis for \(V\) is a collection of vectors \(\{v_{1},v_{2},\ldots,v_{r}\}\subset V\) with the following properties:

  1. If \(v\in V\), then there are scalars \(a_{1},\ldots,a_{r}\) such that \[a_{1}v_{1} + a_{2}v_{2} + \cdots + a_{r}v_{r} = v\]
  2. If \[b_{1}v_{1}+b_{2}v_{2}+\cdots+b_{r}v_{r} = 0\] for some scalars \(b_{1},\ldots,b_{r}\), then \(b_{1}=b_{2}=\cdots=b_{r}=0\).

If a sequence of vectors satisfy Property 1, then we say that \(\{v_{1},v_{2},\ldots,v_{r}\}\) spans \(V\).

If a sequence of vectors satisfy Property 2, then we say that \(\{v_{1},v_{2},\ldots,v_{r}\}\) is linearly independent.

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

Note that the vectors

\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]

span \(C(A)\). 

However, they are not independent, since

\[1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}+(-1)\cdot \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}+17\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}+0\cdot\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix}0\\ 0\\ 0\end{bmatrix}\]

However, they are not independent, since

\[\begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 17\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

Theorem. Let \(x_{1},\ldots,x_{m}\) be a collection of vectors in \(\R^{n}\).

If \(x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}\), then

\[\text{span}\{x_{1},\ldots,x_{m-1}\} = \text{span}\{x_{1},\ldots,x_{m}\}.\]

Proof. The assumption that \(x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}\) means that there are scalars \(a_{1},\ldots,a_{m-1}\) so that

\[x_{m} = a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}.\]

It is clear that

\[\text{span}\{x_{1},\ldots,x_{m-1}\} \subseteq \text{span}\{x_{1},\ldots,x_{m}\}.\]

If \(y\in\text{span}\{x_{1},\ldots,x_{m}\}\), then there are scalars \(b_{1},\ldots,b_{m}\) so that


\[ = b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}\big(a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}\big)\]

\[ = (b_{1}+a_{1}b_{m})x_{1} + (b_{2}+a_{2}b_{m})x_{2} + \cdots + (b_{m-1}+a_{m-1}b_{m})x_{m-1}\]

\[y = b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}x_{m}\]

This shows that \(y\in\text{span}\{x_{1},\ldots,x_{m-1}\}\), and thus \[\text{span}\{x_{1},\ldots,x_{m-1}\} \supseteq \text{span}\{x_{1},\ldots,x_{m}\}.\]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

Note that the vectors

\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]

span \(C(A)\). 

Next, note that

\[1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}\]

\[\text{and}\quad 1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+(-1)\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]

Spans \(C(A)\), but is it independent?

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}\]

We want to find the solutions to \[\begin{bmatrix} 1 & 1\\ 2 & 1\\ 3 & 2\end{bmatrix} \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} =  \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

or equivalently, the system of equations \[\left\{\begin{array}{ccccc} x_{1} & + & x_{2} & = & 0\\ 2x_{2} & + & x_{2} & = & 0\\ 3x_{1} & + & 2x_{2} & = & 0\end{array}\right.\]

The only solution is \(x_{1} = x_{2} =0\). We have found a basis for \(C(A)\)!

Theorem. Let \(A\) be a matrix and let \(\text{rref}(A)\) be the reduced row echelon form of the matrix. For a vector \(x\) we have \(Ax=\mathbf{0}\) if and only if \(\text{rref}(A)x=\mathbf{0}\).

Proof. Let \(B\) be the matrix so that 

\[BA = \text{rref}(A),\]

and let \(C\) be the matrix so that

\[C\text{rref}(A) = A.\]

Assume \(x\) is a vector such that \(Ax=\mathbf{0}\). Then we have

\[\text{rref}(A)x = BAx = B(\mathbf{0}) = \mathbf{0}.\]

On the other hand, if \(\text{rref}(A)x = \mathbf{0}\), then we have

\[Ax = C\text{rref}(A)x = C(\text{rref}(A)x) = C\mathbf{0} = \mathbf{0}.\ \Box\]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix} \]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + 1\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix} \]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} \]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]

\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} \]

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

Conclusion: The columns in the original matrix corresponding to the pivot columns in \(\operatorname{rref}(A)\) form a spanning set for \(C(A)\).

Hence, \(\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}\) is a spanning set for \(C(A)\).

Example. Consider the matrix

\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]

Find a basis for the column space \(C(A)\).

\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

Is \(\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}\) independent?

If they were dependent, then there would be a nonzero vector \[x=\begin{bmatrix} a & b & 0 & 0 & 0\end{bmatrix}^{\top}\] such that \(Ax=0\). However, then it would also be the case that \(\operatorname{rref}(A)x = 0\), but distinct pivot columns are always independent.

YES!

Theorem. Let \(A\) be an \(m\times n\) matrix with columns \(a_{1},a_{2},\ldots,a_{n}\), and let \(\text{rref}(A)\) be the reduced row echelon form of \(A\). If the pivots of \(\text{rref}(A)\) are in columns \(n_{1},n_{2},\ldots,n_{r}\), then \(a_{n_{1}},a_{n_{2}},\ldots,a_{n_{r}}\) is a basis for \(C(A)\).

To summarize what we just learned, we have the following theorem:

Exercise. Find a basis for \(C(A)\) where

\[A = \begin{bmatrix} 1 & -1 & 1 & 0 & 2\\ 1 & -1 & 1 & 0 & 2\\ -1 & 1 & 0 & 1 & -1\end{bmatrix}\]

\[\text{rref}(A) = \begin{bmatrix} 1 & -1 & 0 & -1 & 1\\ 0 & 0 & 1 & 1 & 1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]

So, the vectors \[\left\{\begin{bmatrix}1\\ 1\\ -1\end{bmatrix}, \begin{bmatrix}1\\ 1\\ 0\end{bmatrix}\right\}\]

form a basis for \(C(A)\)

End Day 6

Linear Algebra Day 5

By John Jasper

Linear Algebra Day 5

  • 454