Day 5:
Finding bases for subspaces
Bases
Definition 4. Let \(V\) be a subspace. A basis for \(V\) is a collection of vectors \(\{v_{1},v_{2},\ldots,v_{r}\}\subset V\) with the following properties:
- If \(v\in V\), then there are scalars \(a_{1},\ldots,a_{r}\) such that \[a_{1}v_{1} + a_{2}v_{2} + \cdots + a_{r}v_{r} = v\]
- If \[b_{1}v_{1}+b_{2}v_{2}+\cdots+b_{r}v_{r} = 0\] for some scalars \(b_{1},\ldots,b_{r}\), then \(b_{1}=b_{2}=\cdots=b_{r}=0\).
If a sequence of vectors satisfy Property 1, then we say that \(\{v_{1},v_{2},\ldots,v_{r}\}\) spans \(V\).
If a sequence of vectors satisfy Property 2, then we say that \(\{v_{1},v_{2},\ldots,v_{r}\}\) is linearly independent.
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
Note that the vectors
\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]
span \(C(A)\).
However, they are not independent, since
\[1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}+(-1)\cdot \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}+17\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}+0\cdot\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix}0\\ 0\\ 0\end{bmatrix}\]
However, they are not independent, since
\[\begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 17\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
Theorem. Let \(x_{1},\ldots,x_{m}\) be a collection of vectors in \(\R^{n}\).
If \(x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}\), then
\[\text{span}\{x_{1},\ldots,x_{m-1}\} = \text{span}\{x_{1},\ldots,x_{m}\}.\]
Proof. The assumption that \(x_{m}\in\text{span}\{x_{1},x_{2},\ldots,x_{m-1}\}\) means that there are scalars \(a_{1},\ldots,a_{m-1}\) so that
\[x_{m} = a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}.\]
It is clear that
\[\text{span}\{x_{1},\ldots,x_{m-1}\} \subseteq \text{span}\{x_{1},\ldots,x_{m}\}.\]
If \(y\in\text{span}\{x_{1},\ldots,x_{m}\}\), then there are scalars \(b_{1},\ldots,b_{m}\) so that
\[ = b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}\big(a_{1}x_{1} + a_{2}x_{2} + \cdots + a_{m-1}x_{m-1}\big)\]
\[ = (b_{1}+a_{1}b_{m})x_{1} + (b_{2}+a_{2}b_{m})x_{2} + \cdots + (b_{m-1}+a_{m-1}b_{m})x_{m-1}\]
\[y = b_{1}x_{1} + b_{2}x_{2} + \cdots + b_{m-1}x_{m-1} + b_{m}x_{m}\]
This shows that \(y\in\text{span}\{x_{1},\ldots,x_{m-1}\}\), and thus \[\text{span}\{x_{1},\ldots,x_{m-1}\} \supseteq \text{span}\{x_{1},\ldots,x_{m}\}.\]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
Note that the vectors
\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}, \begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}, \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]
span \(C(A)\).
Next, note that
\[1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+1\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix}\]
\[\text{and}\quad 1\cdot\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}+(-1)\cdot\begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}=\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix}\]
Spans \(C(A)\), but is it independent?
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\begin{bmatrix} 1\\ 2\\ 3\end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix}\]
We want to find the solutions to \[\begin{bmatrix} 1 & 1\\ 2 & 1\\ 3 & 2\end{bmatrix} \begin{bmatrix} x_{1}\\ x_{2}\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
or equivalently, the system of equations \[\left\{\begin{array}{ccccc} x_{1} & + & x_{2} & = & 0\\ 2x_{2} & + & x_{2} & = & 0\\ 3x_{1} & + & 2x_{2} & = & 0\end{array}\right.\]
The only solution is \(x_{1} = x_{2} =0\). We have found a basis for \(C(A)\)!
Theorem. Let \(A\) be a matrix and let \(\text{rref}(A)\) be the reduced row echelon form of the matrix. For a vector \(x\) we have \(Ax=\mathbf{0}\) if and only if \(\text{rref}(A)x=\mathbf{0}\).
Proof. Let \(B\) be the matrix so that
\[BA = \text{rref}(A),\]
and let \(C\) be the matrix so that
\[C\text{rref}(A) = A.\]
Assume \(x\) is a vector such that \(Ax=\mathbf{0}\). Then we have
\[\text{rref}(A)x = BAx = B(\mathbf{0}) = \mathbf{0}.\]
On the other hand, if \(\text{rref}(A)x = \mathbf{0}\), then we have
\[Ax = C\text{rref}(A)x = C(\text{rref}(A)x) = C\mathbf{0} = \mathbf{0}.\ \Box\]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ 1\\ -1\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\quad 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 1\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 2\\ 3\\ 5\end{bmatrix} \]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + 1\cdot\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\quad 0\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + 0\cdot \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 0\\ 0\end{bmatrix} \]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} \]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
\[\begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\ \Leftrightarrow \ \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\begin{bmatrix} 1\\ -1\\ 0\\ 0\\ -1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} + (-1)\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 0\end{bmatrix}\]
\[\Leftrightarrow\ 1\cdot \begin{bmatrix} 1\\ 2\\ 3\end{bmatrix} + (-1) \begin{bmatrix} 1\\ 1\\ 2\end{bmatrix} =\begin{bmatrix} 0\\ 1\\ 1\end{bmatrix} \]
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
Conclusion: The columns in the original matrix corresponding to the pivot columns in \(\operatorname{rref}(A)\) form a spanning set for \(C(A)\).
Hence, \(\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}\) is a spanning set for \(C(A)\).
Example. Consider the matrix
\[A = \begin{bmatrix} 1 & 1 & 2 & 0 & 0\\ 2 & 1 & 3 & 0 & 1\\ 3 & 2 & 5 & 0 & 1\end{bmatrix}\]
Find a basis for the column space \(C(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 0 & 1 & 0 & 1\\ 0 & 1 & 1 & 0 & -1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
Is \(\left\{\begin{bmatrix} 1\\ 2\\ 3 \end{bmatrix}, \begin{bmatrix} 1\\ 1\\ 2 \end{bmatrix}\right\}\) independent?
If they were dependent, then there would be a nonzero vector \[x=\begin{bmatrix} a & b & 0 & 0 & 0\end{bmatrix}^{\top}\] such that \(Ax=0\). However, then it would also be the case that \(\operatorname{rref}(A)x = 0\), but distinct pivot columns are always independent.
YES!
Theorem. Let \(A\) be an \(m\times n\) matrix with columns \(a_{1},a_{2},\ldots,a_{n}\), and let \(\text{rref}(A)\) be the reduced row echelon form of \(A\). If the pivots of \(\text{rref}(A)\) are in columns \(n_{1},n_{2},\ldots,n_{r}\), then \(a_{n_{1}},a_{n_{2}},\ldots,a_{n_{r}}\) is a basis for \(C(A)\).
To summarize what we just learned, we have the following theorem:
Exercise. Find a basis for \(C(A)\) where
\[A = \begin{bmatrix} 1 & -1 & 1 & 0 & 2\\ 1 & -1 & 1 & 0 & 2\\ -1 & 1 & 0 & 1 & -1\end{bmatrix}\]
\[\text{rref}(A) = \begin{bmatrix} 1 & -1 & 0 & -1 & 1\\ 0 & 0 & 1 & 1 & 1\\ 0 & 0 & 0 & 0 & 0\end{bmatrix}\]
So, the vectors \[\left\{\begin{bmatrix}1\\ 1\\ -1\end{bmatrix}, \begin{bmatrix}1\\ 1\\ 0\end{bmatrix}\right\}\]
form a basis for \(C(A)\)
End Day 6
Linear Algebra Day 5
By John Jasper
Linear Algebra Day 5
- 530
