Dimension and rank
Definition. Let \(U\) be a subspace of \(\R^{n}\). The dimension of \(U\), denoted \(\text{dim}\, U\), is the number of vectors in a basis for \(U\).
Example. Let \(A = \begin{bmatrix} 1 & 1 & 0 & -1\\ 1 & 1 & 0 & -2\\ 0 & 0 & 0 & 1\end{bmatrix}\).
Find the dimension of \(C(A)\) and \(N(A)\).
\[\text{rref}(A) = \begin{bmatrix} 1 & 1 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0\end{bmatrix}\]
\(\left\{\begin{bmatrix} 1\\ 1\\ 0\end{bmatrix},\begin{bmatrix} -1\\ -2\\ 1\end{bmatrix}\right\}\) is a basis for \(C(A)\), so \(\text{dim}\,C(A) = 2\), and
\(\left\{\begin{bmatrix} -1\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 1\\ 0\end{bmatrix}\right\}\) is a basis for \(N(A)\) \(\text{dim}\,N(A) = 2\).
Theorem. Let \(V\subset\mathbb{R}^{n}\) be a subspace. If \[S = \{v_{1},v_{2},\ldots,v_{k}\}\quad \text{and} \quad R = \{w_{1},w_{2},\ldots,w_{\ell}\}\] are both bases for \(V\), then \(|S| = |R|\).
Corollary 1. If \(V\subset \R^{n}\) is a subspace, and \(S\subset V\) is a spanning set for \(V\), then \(|S|\geq \operatorname{dim}V.\)
Proof. If there is an \(x\in S\) such that \(x\) is in the span of \(S\setminus\{x\}\), then remove \(x\) from \(S\). If no such \(x\) exists, then \(S\) is linearly independent. Continue removing such \(x\)'s until there are no such vectors left in \(S\). The remaining set is a basis. Hence, for a spanning set \(S\) there is a basis \(B\subset S\). Thus,
\[\operatorname{dim}V = |B|\leq |S|.\ \Box\]
Corollary 2. If \(V\subset \R^{n}\) is a subspace, and \(S\subset V\) is linearly independent, then \(|S|\leq \operatorname{dim}V.\)
Proof. We will show that there is a basis \(B\supset S\) for \(V\). If \(S\) does not span \(V\), then there is a vector \(v\in V\) such that \(v\notin\operatorname{span}S\). Add this vector to the set \(S\). Repeat until there are \(\operatorname{dim}V\) elements in the set. This must be a basis for \(V\). \(\Box\)
Example 1. For subspaces \(V\subset\mathbb{R}^{n}\) such that there exists a spanning set \(\{x,y,z\}\) for \(V\), it _____________ holds that \(\dim V = 3\).
Fill in the blank with always, sometimes, or never:
Example 2. For subspaces \(V\subset\mathbb{R}^{n}\) such that there exists a linearly independent set \(\{x,y,z\}\subset V\), it _____________ holds that \(\dim V > 3\).
sometimes
sometimes
Example 3. For subspaces \(V\subset\mathbb{R}^{3}\) such that there exists an independent set \(\{x,y,z\}\subset V\) it ______________ holds that \(\dim V = 3\).
always
Definition. Let \(A\) be an \(m\times n\) matrix, and let \(a_{ij}\) denote the entry of \(A\) in row \(i\) and column \(j\). The transpose of \(A\), denoted \(A^{\top}\) is the \(m\times n\) matrix with \(a_{ji}\) in row \(i\) column \(j\).
Examples.
Also, note that \((A^{\top})^{\top} = A\).
Theorem. (The fundamental theorem of transposes) If \(A\in\mathbb{R}^{m\times n}\) and \(B\in\mathbb{R}^{n\times p}\), then \[(AB)^{\top} = B^{\top}A^{\top}.\]
Suppose
\[A=\left[\begin{matrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & & a_{2n}\\ \vdots & & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B=\left[\begin{matrix} b_{11} & b_{12} & \cdots & b_{1p}\\ b_{21} & b_{22} & & b_{2p}\\ \vdots & & \ddots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{np}\end{matrix}\right]\]
The entry in row \(i\), column \(j\) in the matrix \(AB\) is
\[\sum_{k=1}^{n}a_{jk}b_{ki}\]
The entry in row \(i\), column \(j\) in the matrix \((AB)^{\top}\) is
\[\left[\begin{matrix} a_{i1} & a_{i2} & \cdots & a_{in}\end{matrix}\right] \left[\begin{matrix} b_{1j}\\ b_{2j}\\ \vdots\\ b_{nj}\end{matrix}\right] = \sum_{k=1}^{n}a_{ik}b_{kj}\]
Then,
\[A^{\top}=\left[\begin{matrix} a_{11} & a_{21} & \cdots & a_{m1}\\ a_{12} & a_{22} & & a_{m2}\\ \vdots & & \ddots & \vdots \\ a_{1n} & a_{2n} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B^{\top}=\left[\begin{matrix} b_{11} & b_{21} & \cdots & b_{n1}\\ b_{12} & b_{22} & & b_{n2}\\ \vdots & & \ddots & \vdots \\ b_{1p} & b_{2p} & \cdots & b_{np}\end{matrix}\right]\]
The entry in row \(i\), column \(j\) in the matrix \(B^{\top}A^{\top}\) is
\[\left[\begin{matrix} b_{1i} & b_{2i} & \cdots & b_{ni}\end{matrix}\right] \left[\begin{matrix} a_{j1}\\ a_{j2}\\ \vdots\\ a_{jn}\end{matrix}\right] = \sum_{k=1}^{n}a_{jk}b_{ki}.\ \Box\]
Definition. Given a matrix \(A\) define the four subspaces:
Proof. Let \(B\) be the matrix such that \(BA=\operatorname{rref}(A)\), and let \(C\) be the matrix such that \(C\operatorname{rref}(A) = A\).
Now, suppose \(v\in C(A^{\top})\), then \(v = A^{\top}x\) for some vector \(x\). Set \(y=C^{\top}x\), then
\[\operatorname{rref}(A)^{\top}y = \operatorname{rref}(A)^{\top}C^{\top}x = (C\operatorname{rref}(A))^{\top}x = A^{\top}x = v\]
and hence \(v\in C(\operatorname{rref}(A)^{\top})\). The other direction is similar. \(\Box\)
Theorem. If \(A\) is any matrix, then \(C(A^{\top}) = C(\operatorname{rref}(A)^{\top})\).
Example. Let
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}\]
Find a basis for each of the four fundamental subspaces of \(A\).
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0\\ 0 & 0 & 0\end{bmatrix} \]
\[C(A)=\text{span}\left\{\begin{bmatrix} 2\\ -1\\ 1\\ 0\end{bmatrix},\begin{bmatrix}-6\\3\\3\\ 0\end{bmatrix}\right\}\]
\[N(A) = \left\{\begin{bmatrix}0\\ 0\\ \alpha\end{bmatrix} : \alpha\in\R\right\}=\text{span}\left\{\begin{bmatrix} 0\\ 0\\ 1\end{bmatrix}\right\}\]
Example. Let
\[A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}\]
Find a basis for each of the four fundamental subspaces of \(A\).
\[A^{\top}=\begin{bmatrix} 2 & -1 & 1 & 0\\ -6 & 3 & 3 & 0\\ 0 & 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix}1 & -1/2 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0\end{bmatrix} \]
\[C(A^{\top})=\text{span}\left\{\begin{bmatrix} 2\\ -6\\ 0\end{bmatrix},\begin{bmatrix}1\\ 3\\ 0\end{bmatrix}\right\}\]
\[N(A^{\top}) = \left\{\begin{bmatrix}a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{bmatrix} : a_{1}-\frac{1}{2}a_{2}=0,\ a_{3}=0\right\}=\left\{\begin{bmatrix}\frac{1}{2}a_{2}\\ a_{2}\\ 0\\ a_{4}\end{bmatrix} : a_{2},a_{4}\in\R\right\}\]
\[=\text{span}\left\{\begin{bmatrix} 1/2\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 0\\ 1\end{bmatrix}\right\}\]
Matrices don't have dimension!
Instead, we define the following quantity:
Definition. The rank of a matrix \(A\) is the dimension of the column space of \(A\). We denote this number by \(\text{rank}(A)\)
Theorem. The following quantities are equal:
Proof. The only thing we need to show is that \(\text{rref}(A)\) and \(\text{rref}(A^{\top})\) have the same number of pivots.
Example. Set \(A = \begin{bmatrix} 1 & 2 & 0 & -1\\ 1 & 3 & 0 & 1\\ 0 & 0 & 1 & 0\end{bmatrix}\), then \(\text{rref}(A) = \begin{bmatrix} 1 & 0 & 0 & -5\\ 0 & 1 & 0 & 2\\ 0 & 0 & 1 & 0\end{bmatrix}.\)
Note that the rows of \(\text{rref}(A)\) are linear combinations of the rows of \(A\).
The columns of \((\text{rref}(A))^{\top} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ -5 & 2 & 0\end{bmatrix}\) are in \(C(A^{\top})\).
\(\Rightarrow \text{rank}(A) = 3\)
There are \(\text{rank}(A) = 3\) independent columns in \((\text{rref}(A))^\top\).
\(\Rightarrow\ \text{rank}(A) = \text{dim} C(A^{\top})\geq 3\)
Consider a matrix \(A\).
Applying the same reasoning to \(A^{\top}\) we obtain
\[\text{rank}(A^{\top})\leq \text{rank}(A)\]
Therefore,
\[\text{rank}(A^{\top}) = \text{rank}(A).\]
Theorem. The following quantities are equal:
Corollary. The subspaces \(C(A)\) and \(C(A^{\top})\) have the same dimension.
Caution: \(C(A)\) and \(C(A^{\top})\) are almost never the same subspace. Indeed, if \(A\) is \(m\times n\), then \(C(A)\) is a subspace of \(\R^{m}\) and \(C(A^{\top})\) is a subspace of \(\R^{n}\).