Day 8:
Dimension and rank
Dimension of subspaces
Definition. Let \(U\) be a subspace of \(\R^{n}\). The dimension of \(U\), denoted \(\text{dim}\, U\), is the number of vectors in a basis for \(U\).
Example. Let \(A = \begin{bmatrix} 1 & 1 & 0 & -1\\ 1 & 1 & 0 & -2\\ 0 & 0 & 0 & 1\end{bmatrix}\).
Find the dimension of \(C(A)\) and \(N(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 1 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0\end{bmatrix}\]
\(\left\{\begin{bmatrix} 1\\ 1\\ 0\end{bmatrix},\begin{bmatrix} -1\\ -2\\ 1\end{bmatrix}\right\}\) is a basis for \(C(A)\), so \(\text{dim}\,C(A) = 2\), and
\(\left\{\begin{bmatrix} -1\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 1\\ 0\end{bmatrix}\right\}\) is a basis for \(N(A)\) \(\text{dim}\,N(A) = 2\).
Theorem. Let \(V\subset\mathbb{R}^{n}\) be a subspace. If \[S = \{v_{1},v_{2},\ldots,v_{k}\}\quad \text{and} \quad R = \{w_{1},w_{2},\ldots,w_{\ell}\}\] are both bases for \(V\), then \(|S| = |R|\).
Corollary 1. If \(V\subset \R^{n}\) is a subspace, and \(S\subset V\) is a spanning set for \(V\), then \(|S|\geq \operatorname{dim}V.\)
Proof. If there is an \(x\in S\) such that \(x\) is in the span of \(S\setminus\{x\}\), then remove \(x\) from \(S\). If no such \(x\) exists, then \(S\) is linearly independent. Continue removing such \(x\)'s until there are no such vectors left in \(S\). The remaining set is a basis. Hence, for a spanning set \(S\) there is a basis \(B\subset S\). Thus,
\[\operatorname{dim}V = |B|\leq |S|.\ \Box\]
Corollary 2. If \(V\subset \R^{n}\) is a subspace, and \(S\subset V\) is linearly independent, then \(|S|\leq \operatorname{dim}V.\)
Proof. We will show that there is a basis \(B\supset S\) for \(V\). If \(S\) does not span \(V\), then there is a vector \(v\in V\) such that \(v\notin\operatorname{span}S\). Add this vector to the set \(S\). Repeat until there are \(\operatorname{dim}V\) elements in the set. This must be a basis for \(V\). \(\Box\)
Example 1. For subspaces \(V\subset\mathbb{R}^{n}\) such that there exists a spanning set \(\{x,y,z\}\) for \(V\), it _____________ holds that \(\dim V = 3\).
Fill in the blank with always, sometimes, or never:
Example 2. For subspaces \(V\subset\mathbb{R}^{n}\) such that there exists a linearly independent set \(\{x,y,z\}\subset V\), it _____________ holds that \(\dim V > 3\).
sometimes
sometimes
Example 3. For subspaces \(V\subset\mathbb{R}^{3}\) such that there exists an independent set \(\{x,y,z\}\subset V\) it ______________ holds that \(\dim V = 3\).
always
Transpose
Definition. Let \(A\) be an \(m\times n\) matrix, and let \(a_{ij}\) denote the entry of \(A\) in row \(i\) and column \(j\). The transpose of \(A\), denoted \(A^{\top}\) is the \(m\times n\) matrix with \(a_{ji}\) in row \(i\) column \(j\).
- If \(A = \left[\begin{array}{rr} 1 & -2\\ 0 & 3\end{array}\right]\), then \(A^{\top} = \left[\begin{array}{rr} 1 & 0\\ -2 & 3\end{array}\right]\)
- If \(A = \left[\begin{array}{rr} 1 & 2 & 3\\ 0 & -2 & -1\end{array}\right]\), then \(A^{\top} = \left[\begin{array}{rr} 1 & 0\\ 2 & -2\\ 3 & -1\end{array}\right]\)
- If \(A = \left[\begin{array}{rr} 1 & 2 & 3\end{array}\right]\), then \(A^{\top} = \left[\begin{array}{rr} 1\\ 2\\ 3 \end{array}\right]\)
Examples.
Also, note that \((A^{\top})^{\top} = A\).
Theorem. (The fundamental theorem of transposes) If \(A\in\mathbb{R}^{m\times n}\) and \(B\in\mathbb{R}^{n\times p}\), then \[(AB)^{\top} = B^{\top}A^{\top}.\]
Suppose
\[A=\left[\begin{matrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & & a_{2n}\\ \vdots & & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B=\left[\begin{matrix} b_{11} & b_{12} & \cdots & b_{1p}\\ b_{21} & b_{22} & & b_{2p}\\ \vdots & & \ddots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{np}\end{matrix}\right]\]
The entry in row \(i\), column \(j\) in the matrix \(AB\) is
\[\sum_{k=1}^{n}a_{jk}b_{ki}\]
The entry in row \(i\), column \(j\) in the matrix \((AB)^{\top}\) is
\[\left[\begin{matrix} a_{i1} & a_{i2} & \cdots & a_{in}\end{matrix}\right] \left[\begin{matrix} b_{1j}\\ b_{2j}\\ \vdots\\ b_{nj}\end{matrix}\right] = \sum_{k=1}^{n}a_{ik}b_{kj}\]
Then,
\[A^{\top}=\left[\begin{matrix} a_{11} & a_{21} & \cdots & a_{m1}\\ a_{12} & a_{22} & & a_{m2}\\ \vdots & & \ddots & \vdots \\ a_{1n} & a_{2n} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B^{\top}=\left[\begin{matrix} b_{11} & b_{21} & \cdots & b_{n1}\\ b_{12} & b_{22} & & b_{n2}\\ \vdots & & \ddots & \vdots \\ b_{1p} & b_{2p} & \cdots & b_{np}\end{matrix}\right]\]
The entry in row \(i\), column \(j\) in the matrix \(B^{\top}A^{\top}\) is
\[\left[\begin{matrix} b_{1i} & b_{2i} & \cdots & b_{ni}\end{matrix}\right] \left[\begin{matrix} a_{j1}\\ a_{j2}\\ \vdots\\ a_{jn}\end{matrix}\right] = \sum_{k=1}^{n}a_{jk}b_{ki}.\ \Box\]
Four Fundamental Subspaces
Definition. Given a matrix \(A\) define the four subspaces:
- \(C(A) = \{Ax : x \text{ is any vector.}\}\) is the column space of \(A\)
- \(C(A^{\top})\) is the row space of \(A\).
- \(N(A) = \{x : Ax=0\}\) is the nullspace of \(A\).
- \(N(A^{\top})\) is the left nullspace of \(A\).
Proof. Let \(B\) be the matrix such that \(BA=\operatorname{rref}(A)\), and let \(C\) be the matrix such that \(C\operatorname{rref}(A) = A\).
Now, suppose \(v\in C(A^{\top})\), then \(v = A^{\top}x\) for some vector \(x\). Set \(y=C^{\top}x\), then
\[\operatorname{rref}(A)^{\top}y = \operatorname{rref}(A)^{\top}C^{\top}x = (C\operatorname{rref}(A))^{\top}x = A^{\top}x = v\]
and hence \(v\in C(\operatorname{rref}(A)^{\top})\). The other direction is similar. \(\Box\)
Theorem. If \(A\) is any matrix, then \(C(A^{\top}) = C(\operatorname{rref}(A)^{\top})\).
Example. Let
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}\]
Find a basis for each of the four fundamental subspaces of \(A\).
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0\\ 0 & 0 & 0\end{bmatrix} \]
\[C(A)=\text{span}\left\{\begin{bmatrix} 2\\ -1\\ 1\\ 0\end{bmatrix},\begin{bmatrix}-6\\3\\3\\ 0\end{bmatrix}\right\}\]
\[N(A) = \left\{\begin{bmatrix}0\\ 0\\ \alpha\end{bmatrix} : \alpha\in\R\right\}=\text{span}\left\{\begin{bmatrix} 0\\ 0\\ 1\end{bmatrix}\right\}\]
Example. Let
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}\]
Find a basis for each of the four fundamental subspaces of \(A\).
\[A^{\top}=\begin{bmatrix} 2 & -1 & 1 & 0\\ -6 & 3 & 3 & 0\\ 0 & 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix}1 & -1/2 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0\end{bmatrix} \]
\[C(A^{\top})=\text{span}\left\{\begin{bmatrix} 2\\ -6\\ 0\end{bmatrix},\begin{bmatrix}1\\ 3\\ 0\end{bmatrix}\right\}\]
\[N(A^{\top}) = \left\{\begin{bmatrix}a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{bmatrix} : a_{1}-\frac{1}{2}a_{2}=0,\ a_{3}=0\right\}=\left\{\begin{bmatrix}\frac{1}{2}a_{2}\\ a_{2}\\ 0\\ a_{4}\end{bmatrix} : a_{2},a_{4}\in\R\right\}\]
\[=\text{span}\left\{\begin{bmatrix} 1/2\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 0\\ 1\end{bmatrix}\right\}\]
Rank
Matrices don't have dimension!
Instead, we define the following quantity:
Definition. The rank of a matrix \(A\) is the dimension of the column space of \(A\). We denote this number by \(\text{rank}(A)\)
Theorem. The following quantities are equal:
- The rank of \(A\)
- The number of pivots in \(\text{rref}(A)\)
- The number of pivots in \(\text{rref}(A^{\top})\)
- The rank of \(A^{\top}\)
Proof. The only thing we need to show is that \(\text{rref}(A)\) and \(\text{rref}(A^{\top})\) have the same number of pivots.
Row Rank vs. Column Rank
Example. Set \(A = \begin{bmatrix} 1 & 2 & 0 & -1\\ 1 & 3 & 0 & 1\\ 0 & 0 & 1 & 0\end{bmatrix}\), then \(\text{rref}(A) = \begin{bmatrix} 1 & 0 & 0 & -5\\ 0 & 1 & 0 & 2\\ 0 & 0 & 1 & 0\end{bmatrix}.\)
Note that the rows of \(\text{rref}(A)\) are linear combinations of the rows of \(A\).
The columns of \((\text{rref}(A))^{\top} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ -5 & 2 & 0\end{bmatrix}\) are in \(C(A^{\top})\).
\(\Rightarrow \text{rank}(A) = 3\)
There are \(\text{rank}(A) = 3\) independent columns in \((\text{rref}(A))^\top\).
\(\Rightarrow\ \text{rank}(A) = \text{dim} C(A^{\top})\geq 3\)
Row Rank vs. Column Rank
Consider a matrix \(A\).
- By definition \(\text{rank}(A) = \text{dim}\,C(A)\)
- We have already seen that \(\text{dim}\,C(A)\) is equal to the number of pivots in \(\text{rref}(A)\).
- The columns of \((\text{rref}(A))^{\top}\) are in the row space of \(A\). There are \(\text{rank}(A)\) independent columns in \((\text{rref}(A))^{\top}\).
- The dimension of \(C(A^{\top})\) is at least \(\text{rank}(A)\).
- \(\text{rank}(A)\leq \text{rank}(A^{\top}).\)
Applying the same reasoning to \(A^{\top}\) we obtain
\[\text{rank}(A^{\top})\leq \text{rank}(A)\]
Therefore,
\[\text{rank}(A^{\top}) = \text{rank}(A).\]
Theorem. The following quantities are equal:
- The rank of \(A\)
- The number of pivots in \(\text{rref}(A)\)
- The number of pivots in \(\text{rref}(A^{\top})\)
- The rank of \(A^{\top}\)
Corollary. The subspaces \(C(A)\) and \(C(A^{\top})\) have the same dimension.
Caution: \(C(A)\) and \(C(A^{\top})\) are almost never the same subspace. Indeed, if \(A\) is \(m\times n\), then \(C(A)\) is a subspace of \(\R^{m}\) and \(C(A^{\top})\) is a subspace of \(\R^{n}\).
End Day 8
Linear Algebra Day 8
By John Jasper
Linear Algebra Day 8
- 503
