# Day 8:

Dimension and rank

### Dimension of subspaces

Definition. Let $$U$$ be a subspace of $$\R^{n}$$. The dimension of $$U$$, denoted $$\text{dim}\, U$$, is the number of vectors in a basis for $$U$$.

Example. Let $$A = \begin{bmatrix} 1 & 1 & 0 & -1\\ 1 & 1 & 0 & -2\\ 0 & 0 & 0 & 1\end{bmatrix}$$.

Find the dimension of $$C(A)$$ and $$N(A)$$.

$\text{rref}(A) = \begin{bmatrix} 1 & 1 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0 & 0 & 0 & 0\end{bmatrix}$

$$\left\{\begin{bmatrix} 1\\ 1\\ 0\end{bmatrix},\begin{bmatrix} -1\\ -2\\ 1\end{bmatrix}\right\}$$ is a basis for $$C(A)$$, so $$\text{dim}\,C(A) = 2$$, and

$$\left\{\begin{bmatrix} -1\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 1\\ 0\end{bmatrix}\right\}$$ is a basis for $$N(A)$$ $$\text{dim}\,N(A) = 2$$.

Theorem. Let $$V\subset\mathbb{R}^{n}$$ be a subspace. If $S = \{v_{1},v_{2},\ldots,v_{k}\}\quad \text{and} \quad R = \{w_{1},w_{2},\ldots,w_{\ell}\}$ are both bases for $$V$$, then $$|S| = |R|$$.

Corollary 1. If $$V\subset \R^{n}$$ is a subspace, and $$S\subset V$$ is a spanning set for $$V$$, then $$|S|\geq \operatorname{dim}V.$$

Proof. If there is an $$x\in S$$ such that $$x$$ is in the span of $$S\setminus\{x\}$$, then remove $$x$$ from $$S$$. If no such $$x$$ exists, then $$S$$ is linearly independent. Continue removing such $$x$$'s until there are no such vectors left in $$S$$. The remaining set is a basis. Hence, for a spanning set $$S$$ there is a basis $$B\subset S$$. Thus,

$\operatorname{dim}V = |B|\leq |S|.\ \Box$

Corollary 2. If $$V\subset \R^{n}$$ is a subspace, and $$S\subset V$$ is linearly independent, then $$|S|\leq \operatorname{dim}V.$$

Proof. We will show that there is a basis $$B\supset S$$ for $$V$$. If $$S$$ does not span $$V$$, then there is a vector $$v\in V$$ such that $$v\notin\operatorname{span}S$$. Add this vector to the set $$S$$. Repeat until there are $$\operatorname{dim}V$$ elements in the set. This must be a basis for $$V$$. $$\Box$$

Example 1. For subspaces $$V\subset\mathbb{R}^{n}$$ such that there exists a spanning set $$\{x,y,z\}$$ for $$V$$, it _____________ holds that $$\dim V = 3$$.

Fill in the blank with always, sometimes, or never:

Example 2. For subspaces $$V\subset\mathbb{R}^{n}$$ such that there exists a linearly independent set $$\{x,y,z\}\subset V$$, it _____________ holds that $$\dim V > 3$$.

sometimes

sometimes

Example 3. For subspaces $$V\subset\mathbb{R}^{3}$$ such that there exists an independent set $$\{x,y,z\}\subset V$$ it ______________ holds that $$\dim V = 3$$.

always

### Transpose

Definition. Let $$A$$ be an $$m\times n$$ matrix, and let $$a_{ij}$$ denote the entry of $$A$$ in row $$i$$ and column $$j$$. The transpose of $$A$$, denoted $$A^{\top}$$ is the $$m\times n$$ matrix with $$a_{ji}$$ in row $$i$$ column $$j$$.

• If $$A = \left[\begin{array}{rr} 1 & -2\\ 0 & 3\end{array}\right]$$, then $$A^{\top} = \left[\begin{array}{rr} 1 & 0\\ -2 & 3\end{array}\right]$$
• If $$A = \left[\begin{array}{rr} 1 & 2 & 3\\ 0 & -2 & -1\end{array}\right]$$, then $$A^{\top} = \left[\begin{array}{rr} 1 & 0\\ 2 & -2\\ 3 & -1\end{array}\right]$$
• If $$A = \left[\begin{array}{rr} 1 & 2 & 3\end{array}\right]$$, then $$A^{\top} = \left[\begin{array}{rr} 1\\ 2\\ 3 \end{array}\right]$$

Examples.

Also, note that $$(A^{\top})^{\top} = A$$.

Theorem. (The fundamental theorem of transposes) If $$A\in\mathbb{R}^{m\times n}$$ and $$B\in\mathbb{R}^{n\times p}$$, then $(AB)^{\top} = B^{\top}A^{\top}.$

Suppose

$A=\left[\begin{matrix} a_{11} & a_{12} & \cdots & a_{1n}\\ a_{21} & a_{22} & & a_{2n}\\ \vdots & & \ddots & \vdots \\ a_{m1} & a_{m2} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B=\left[\begin{matrix} b_{11} & b_{12} & \cdots & b_{1p}\\ b_{21} & b_{22} & & b_{2p}\\ \vdots & & \ddots & \vdots \\ b_{n1} & b_{n2} & \cdots & b_{np}\end{matrix}\right]$

The entry in row $$i$$, column $$j$$ in the matrix $$AB$$ is

$\sum_{k=1}^{n}a_{jk}b_{ki}$

The entry in row $$i$$, column $$j$$ in the matrix $$(AB)^{\top}$$ is

$\left[\begin{matrix} a_{i1} & a_{i2} & \cdots & a_{in}\end{matrix}\right] \left[\begin{matrix} b_{1j}\\ b_{2j}\\ \vdots\\ b_{nj}\end{matrix}\right] = \sum_{k=1}^{n}a_{ik}b_{kj}$

Then,

$A^{\top}=\left[\begin{matrix} a_{11} & a_{21} & \cdots & a_{m1}\\ a_{12} & a_{22} & & a_{m2}\\ \vdots & & \ddots & \vdots \\ a_{1n} & a_{2n} & \cdots & a_{mn}\end{matrix}\right]\quad\text{and}\quad B^{\top}=\left[\begin{matrix} b_{11} & b_{21} & \cdots & b_{n1}\\ b_{12} & b_{22} & & b_{n2}\\ \vdots & & \ddots & \vdots \\ b_{1p} & b_{2p} & \cdots & b_{np}\end{matrix}\right]$

The entry in row $$i$$, column $$j$$ in the matrix $$B^{\top}A^{\top}$$ is

$\left[\begin{matrix} b_{1i} & b_{2i} & \cdots & b_{ni}\end{matrix}\right] \left[\begin{matrix} a_{j1}\\ a_{j2}\\ \vdots\\ a_{jn}\end{matrix}\right] = \sum_{k=1}^{n}a_{jk}b_{ki}.\ \Box$

### Four Fundamental Subspaces

Definition. Given a matrix $$A$$ define the four subspaces:

• $$C(A) = \{Ax : x \text{ is any vector.}\}$$ is the column space of $$A$$
• $$C(A^{\top})$$ is the row space of $$A$$.
• $$N(A) = \{x : Ax=0\}$$ is the nullspace of $$A$$.
• $$N(A^{\top})$$ is the left nullspace of $$A$$.

Proof. Let $$B$$ be the matrix such that $$BA=\operatorname{rref}(A)$$, and let $$C$$ be the matrix such that $$C\operatorname{rref}(A) = A$$.

Now, suppose $$v\in C(A^{\top})$$, then $$v = A^{\top}x$$ for some vector $$x$$. Set $$y=C^{\top}x$$, then

$\operatorname{rref}(A)^{\top}y = \operatorname{rref}(A)^{\top}C^{\top}x = (C\operatorname{rref}(A))^{\top}x = A^{\top}x = v$

and hence $$v\in C(\operatorname{rref}(A)^{\top})$$. The other direction is similar. $$\Box$$

Theorem. If $$A$$ is any matrix, then $$C(A^{\top}) = C(\operatorname{rref}(A)^{\top})$$.

Example. Let

$A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}$

Find a basis for each of the four fundamental subspaces of $$A$$.

$A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 0\\ 0 & 0 & 0\end{bmatrix}$

$C(A)=\text{span}\left\{\begin{bmatrix} 2\\ -1\\ 1\\ 0\end{bmatrix},\begin{bmatrix}-6\\3\\3\\ 0\end{bmatrix}\right\}$

$N(A) = \left\{\begin{bmatrix}0\\ 0\\ \alpha\end{bmatrix} : \alpha\in\R\right\}=\text{span}\left\{\begin{bmatrix} 0\\ 0\\ 1\end{bmatrix}\right\}$

Example. Let

$A=\begin{bmatrix} 2 & -6 & 0\\ -1 & 3 & 0\\ 1 & 3 & 0\\ 0 & 0 & 0\end{bmatrix}$

Find a basis for each of the four fundamental subspaces of $$A$$.

$A^{\top}=\begin{bmatrix} 2 & -1 & 1 & 0\\ -6 & 3 & 3 & 0\\ 0 & 0 & 0 & 0\end{bmatrix} \sim \begin{bmatrix}1 & -1/2 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 0\end{bmatrix}$

$C(A^{\top})=\text{span}\left\{\begin{bmatrix} 2\\ -6\\ 0\end{bmatrix},\begin{bmatrix}1\\ 3\\ 0\end{bmatrix}\right\}$

$N(A^{\top}) = \left\{\begin{bmatrix}a_{1}\\ a_{2}\\ a_{3}\\ a_{4}\end{bmatrix} : a_{1}-\frac{1}{2}a_{2}=0,\ a_{3}=0\right\}=\left\{\begin{bmatrix}\frac{1}{2}a_{2}\\ a_{2}\\ 0\\ a_{4}\end{bmatrix} : a_{2},a_{4}\in\R\right\}$

$=\text{span}\left\{\begin{bmatrix} 1/2\\ 1\\ 0\\ 0\end{bmatrix},\begin{bmatrix} 0\\ 0\\ 0\\ 1\end{bmatrix}\right\}$

### Rank

Matrices don't have dimension!

Instead, we define the following quantity:

Definition. The rank of a matrix $$A$$ is the dimension of the column space of $$A$$. We denote this number by $$\text{rank}(A)$$

Theorem. The following quantities are equal:

1. The rank of $$A$$
2. The number of pivots in $$\text{rref}(A)$$
3. The number of pivots in $$\text{rref}(A^{\top})$$
4. The rank of $$A^{\top}$$

Proof. The only thing we need to show is that $$\text{rref}(A)$$ and $$\text{rref}(A^{\top})$$ have the same number of pivots.

### Row Rank vs. Column Rank

Example. Set $$A = \begin{bmatrix} 1 & 2 & 0 & -1\\ 1 & 3 & 0 & 1\\ 0 & 0 & 1 & 0\end{bmatrix}$$, then $$\text{rref}(A) = \begin{bmatrix} 1 & 0 & 0 & -5\\ 0 & 1 & 0 & 2\\ 0 & 0 & 1 & 0\end{bmatrix}.$$

Note that the rows of $$\text{rref}(A)$$ are linear combinations of the rows of $$A$$.

The columns of $$(\text{rref}(A))^{\top} = \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ -5 & 2 & 0\end{bmatrix}$$ are in $$C(A^{\top})$$.

$$\Rightarrow \text{rank}(A) = 3$$

There are $$\text{rank}(A) = 3$$ independent columns in $$(\text{rref}(A))^\top$$.

$$\Rightarrow\ \text{rank}(A) = \text{dim} C(A^{\top})\geq 3$$

### Row Rank vs. Column Rank

Consider a matrix $$A$$.

• By definition $$\text{rank}(A) = \text{dim}\,C(A)$$
• We have already seen that $$\text{dim}\,C(A)$$ is equal to the number of pivots in $$\text{rref}(A)$$.
• The columns of $$(\text{rref}(A))^{\top}$$ are in the row space of $$A$$. There are $$\text{rank}(A)$$ independent columns in $$(\text{rref}(A))^{\top}$$.
• The dimension of $$C(A^{\top})$$ is at least $$\text{rank}(A)$$.
• $$\text{rank}(A)\leq \text{rank}(A^{\top}).$$

Applying the same reasoning to $$A^{\top}$$ we obtain

$\text{rank}(A^{\top})\leq \text{rank}(A)$

Therefore,

$\text{rank}(A^{\top}) = \text{rank}(A).$

Theorem. The following quantities are equal:

1. The rank of $$A$$
2. The number of pivots in $$\text{rref}(A)$$
3. The number of pivots in $$\text{rref}(A^{\top})$$
4. The rank of $$A^{\top}$$

Corollary. The subspaces $$C(A)$$ and $$C(A^{\top})$$ have the same dimension.

Caution: $$C(A)$$ and $$C(A^{\top})$$ are almost never the same subspace. Indeed, if $$A$$ is $$m\times n$$, then $$C(A)$$ is a subspace of $$\R^{m}$$ and $$C(A^{\top})$$ is a subspace of $$\R^{n}$$.

By John Jasper

• 525