Day 11:

Inverse of a matrix

Abstract vector spaces

Computing \(B\) such that \(BA=\text{rref}(A)\).

Theorem. Let \(A\) be an \(m\times n\) matrix. Let \(I\) be an \(m\times m\) identity matrix. If

\[\text{rref}([A\ |\ I]) = [D\ |\ B]\]

then \(BA=\text{rref}(A)\).

Proof. Note that \[\text{rref}([A\ |\ I]) = [\text{rref}(A)\ |\ B].\] Assume \(C\) is the matrix such that

\[C\cdot [A\ \vert\ I] = \text{rref}([A\ \vert\ I])= [\text{rref}(A)\ \vert\ B].\]

Finally, note that \[C\cdot [A\ \vert\ I] = [CA\ \vert\ CI] = [CA\ \vert\ C],\] and hence \(C=B\) and \(BA=\text{rref}(A)\). \(\Box\)

Computing \(B\) such that \(BA=\text{rref}(A)\).

Example.

\[\text{rref}\left(\left[\begin{array}{rrr|rr} 2 & 3 & 1 & 1 & 0\\ 2 & 3 & -2 & 0 & 1 \end{array}\right]\right) = \left[\begin{array}{rrr|rr} 1 & 3/2 & 0 & 1/3 & 1/6\\ 0 & 0 & 1 & 1/3 & -1/3 \end{array}\right]\]

\[\left[\begin{array}{rrr} 1/3 & 1/6\\ 1/3 & -1/3 \end{array}\right]\left[\begin{array}{rrr} 2 & 3 & 1\\ 2 & 3 & -2 \end{array}\right] = \left[\begin{array}{rrr} 1 & 3/2 & 0\\ 0 & 0 & 1\end{array}\right]\]

The Inverse of a matrix

Definition. Given a square matrix \(A\), a square matrix \(B\) such that \(AB=BA=I\) is called the inverse of \(A\). The inverse of \(A\) is denoted \(A^{-1}\). If a matrix \(A\) has an inverse, then we say that \(A\) is invertible.

Examples.

• If \(A=\begin{bmatrix} 1 & 1\\ 0 & 1\end{bmatrix}\), then \(A^{-1} = \begin{bmatrix} 1 & -1\\ 0 & 1\end{bmatrix}\)

• If \(A = \begin{bmatrix} 1 & 0 & -2\\ 3  & 1 & 0\\ 0 & 0 & 1\end{bmatrix}\), then \(A^{-1} = \begin{bmatrix} 1 & 0 & 2\\ -3  & 1 & -6\\ 0 & 0 & 1\end{bmatrix}\)

Theorem. A matrix \(A\) is invertible if and only if \(A\) is square and \(\text{rref}(A)=I\).

Proof. Suppose \(\text{rref}(A)=I\). There exists a matrix \(B\) so that \(BA=\text{rref}(A)\). From this we deduce that \(BA=I\). For each elementary matrix \(E\) there is an elementary matrix \(F\) such that \(FE=I\). Since \(B\) is a product of elementary matrices \[B= E_{k}\cdot E_{k-1}\cdots E_{1}\]

We take \(F_{i}\) such that \(F_{i}E_{i}=I\) for each \(i\), and set  \[C = F_{1}\cdot F_{2}\cdots F_{k}\]

and we see that \(CB=I\). Finally, we have \[AB = CBAB =C I B = CB = I.\]

Now, suppose \(A\) is invertible. By definition \(A\) is square. If \(x\) is a vector such that \(Ax=0\), then \(A^{-1}Ax=A^{-1}0\). Thus, \(x=0\) is the only solution to \(Ax=0\). Hence, \(\{0\}=N(A) = N(\operatorname{rref}(A))\). This implies that each row of \(\operatorname{rref}(A)\) must have a pivot. Since it is square, every column of \(\operatorname{rref}(A)\) contains a pivot. This implies \(\operatorname{rref}(A)=I\). \(\Box\)

Example. Let \[A = \begin{bmatrix} 2 & 3 & 4\\ 3 & 4 & 0\\ 5 & 7 & 4\end{bmatrix}\]

Is \(A\) invertible? If it is, find \(A^{-1}\).

Example.  Let \[A = \begin{bmatrix} 0 & -1 & -2\\ 1 & 1 & 1\\ -1 & -1 & 0\end{bmatrix}\]

Is \(A\) invertible? If it is, find \(A^{-1}\).

Vector Spaces

Definition. A vector space (over \(\mathbb{R}\)) is a set \(V\) together with two operations:

1. vector addition: a map from \(V\times V\to V\). Given two inputs \(x,y\in V\), the output of this map is denoted \(x+y\)
2. scalar multiplication: a map from \(\mathbb{R}\times V\to V\). Given inputs \(a\in\mathbb{R}\) and \(x\in V\), the output of this map is denoted \(ax\)

and an element \(0_{V}\in V\) called the zero vector such that for each \(x,y,z\in V\) and \(a,b\in\mathbb{R}\), the following hold:

(a) \(x+y=y+x\)

(b) \((x+y)+z = x + (y+z)\)

(c) \(0x=0_{V}\)

(d) \(1x=x\)

(e) \((ab)x = a(bx)\)

(f) \(a(x+y)=ax+ay\)

(g) \((a+b)x = ax+bx\)

Examples of Vector Spaces

• \(\R^{n}\) is a vector space. Vector addition and scalar multiplication are defined as follows: \[\begin{bmatrix} x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{bmatrix} + \begin{bmatrix} y_{1}\\ y_{2}\\ \vdots\\ y_{n}\end{bmatrix} = \begin{bmatrix} x_{1}+y_{1}\\ x_{2}+y_{2}\\ \vdots\\ x_{n}+y_{n}\end{bmatrix}\quad\text{and}\quad a\begin{bmatrix} x_{1}\\ x_{2}\\ \vdots\\ x_{n}\end{bmatrix} = \begin{bmatrix} ax_{1}\\ ax_{2}\\ \vdots\\ ax_{n}\end{bmatrix}\] and \[0_{\mathbb{R}^{n}} := [0\ \ 0\ \ \cdots\ \ 0]^{\top}.\]
• If \(W\subset\mathbb{R}^{n}\) is a subspace, then \(W\) is a vector space with the same vector addition, scalar multiplication, and zero vector as \(\mathbb{R}^{n}\). 

Examples of Vector Spaces

• For any set \(S\), consider the set of functions \[\mathbb{R}^{S}:= \{f:S\to\mathbb{R}\}.\] Then \(\mathbb{R}^{S}\) is a vector space with vector addition and scalar multiplication given by \[(f+g)(s) := f(s)+g(s), \quad (af)(s):=a\cdot f(s),\quad s\in S\] and with the zero vector given by \(0_{\mathbb{R}^{S}}(s)=0\) for all \(s\in S\).
• A particular example of the above vector space: Consider the set \(\mathbb{R}^{\mathbb{N}}\). This is the set of all functions with domain \(\mathbb{N}\) and codomain \(\mathbb{R}.\) For example, if we set \(f(n) := \frac{1}{n}\) and \(g(n) := 2^{n}\) for all \(n\in\mathbb{N}\), then \(f\) and \(g\) are vectors in the vector space \(\mathbb{R}^{\mathbb{N}}\). 

Examples of Vector Spaces

• For \(n\in\N\cup\{0\}\), set \[\mathbb{P}_{n} = \left\{a_{0}+a_{1}x+a_{2}x^2+\cdots+a_{n}x^{n} : a_{0},a_{1},a_{2},\ldots,a_{n}\in\mathbb{R}\right\},\] that is, \(\mathbb{P}_{n}\) is the set of polynomials of degree \(\leq n\). Define vector addition and scalar multiplication to be the usual sum of polynomials and multiplication of a polynomial by a scalar, respectively. Let \(0_{\mathbb{P}_{n}}\) be the zero polynomial. With all of this \(\mathbb{P}_{n}\) is a vector space.

Familiar definitions in a new setting

Definition. Let \(V\) be a vector space.

• A nonempty subset \(W\subset V\) is a subspace if for every \(x,y\in W\) and \(a\in\mathbb{R}\) it follows that \(x+y\in W\) and \(ax\in W\). 
• A linear combination of \(x_{1},x_{2},\ldots,x_{k}\in V\), is any vector of the form \[a_{1}x_{1}+a_{2}x_{2} + \cdots + a_{k}x_{k}\] where \(a_{1},a_{2},\ldots,a_{k}\in\mathbb{R}.\)
• The span of \(x_{1},x_{2},\ldots,x_{k}\in V\) is the set of all linear combinations. This set is denoted \(\operatorname{span}\{x_{i}\}_{i=1}^{k}.\) If \(\operatorname{span}\{x_{i}\}_{i=1}^{k}=V\), then we say that \(\{x_{i}\}_{i=1}^{k}\) spans \(V\).
• The vectors \(\{x_{i}\}_{i=1}^{k}\) are linearly dependent if there are scalars \(a_{1},a_{2},\ldots,a_{k}\in\mathbb{R}\), at least one of which is not zero, such that \[a_{1}x_{1}+a_{2}x_{2}+\cdots + a_{k}x_{k} = 0_{V}.\] Otherwise, \(\{x_{i}\}_{i=1}^{k}\) is linearly independent. 

Familiar definitions in a new setting

Definition continued. Let \(V\) be a vector space.

• A linearly independent spanning set for \(V\) is known as a basis for \(V\).
• The dimension of \(V\), denoted \(\operatorname{dim}V\), is the size of any basis for \(V.\) If \(V\) has a finite basis, then we say that \(V\) is finite-dimensional.

Linear maps

Definition. Suppose \(V\) and \(W\) are vector spaces. A function \(L:V\to W\) is called linear if for every \(x,y\in V\) and \(a\in\mathbb{R}\) the following two properties hold:

1. \(L(x+y) = L(x)+L(y)\)
2. \(L(ax)=aL(x)\).

Linear maps

Example 2. Define the function \(D:\mathbb{P}_{2}\to\mathbb{P}_{1}\) by \(Df(x) = f'(x)\) for \(f(x)\in\mathbb{P}_{n}\). For example,

\[D(2x^2+x-1) = 4x+1.\]

Using some properties of the derivative from calculus, if \(f(x),g(x)\in\mathbb{P}_{2}\) and \(c\in\mathbb{R}\), then

\[D(f(x) + g(x)) = f'(x) + g'(x) = D(f(x)) + D(g(x))\]

and

\[D(cf(x)) = cf'(x) = cD(f(x)).\] 

Therefore, \(D\) is linear.

Image and Kernel

Definition. Suppose \(V\) and \(W\) are vector spaces, and \(L:V\to W\) is linear. The image of \(L\) is the set

\[\operatorname{im}(L) := \{Lv : v\in V\}.\]

The kernel of \(L\) is the set

\[\operatorname{ker}(L):= \{v : L(v) = 0\}.\]

If \(A\in\mathbb{R}^{m\times n}\), and we set \(L(x) = Ax\) for \(x\in\mathbb{R}^{n}\), then \(L:\mathbb{R}^{n}\to\mathbb{R}^{m}\) is linear, moreover, 

\[\operatorname{im}(L) = C(A)\quad\text{and}\quad \operatorname{ker}(L) = N(A).\]

End Day 11