Day 15:

Examples of solving \(Ax=b\)

Theorem. Let \(A\) be an \(m\times n\) matrix, and let \(b\in\R^{m}\).

If \(\mathbf{x}_{0}\in\R^{n}\) is a solution to \(Ax=b\), then

\[\{x\in\R^{n} : Ax=b\} = \{\mathbf{x}_{0} + z : z\in N(A)\}\]

Moreover, if \(\{v_{1},v_{2},\ldots,v_{k}\}\) is a basis for \(N(A)\), then

\[\{x\in\R^{n} : Ax=b\} = \{\mathbf{x}_{0} +a_{1}v_{1}+a_{2}v_{2}+\cdots+a_{k}v_{k} : a_{1},a_{2},\ldots,a_{k}\in\R\}\]

Proof. Let \(x\) be any vector such that \(Ax=b\). Note that \[A(x-\mathbf{x}_{0}) = Ax-A\mathbf{x}_{0} = b-b = \mathbf{0}.\]

This shows that \(x-\mathbf{x}_{0}\in N(A)\). Set \(z = x-\mathbf{x}_{0}\). Since \[x=\mathbf{x}_{0} + (x-\mathbf{x}_{0}),\] we see that \(x\in \{\mathbf{x}_{0} + z : z\in N(A)\}\).

Next, assume \(y\in \{\mathbf{x}_{0} : z\in N(A)\}\), that is, \(y=\mathbf{x}_{0}+z\) for some \(z\in N(A)\). Then we see that

\[Ay = A(\mathbf{x}_{0}+z) = A\mathbf{x}_{0} + Az = b+\mathbf{0} = b.\]

Therefore, \(y\) is a solution to \(Ax=b\). \(\Box\)

This is a very nice description of the set of solutions to \(Ax=b\)

We call this the vector parametric form of the set of soltuions.

Solving \(Ax=b\)

Finding a solution to \(Ax=b\).

Assume \(A\) is an \(m\times n\) matrix and \(b\in\R^{m}\).

  1. Find the matrix \(B\) so that \(BA = \text{rref}(A)\).
  2. Find a solution \(\mathbf{x}_{0}\) to \(\text{rref}(A)x = Bb\).
  3. The vector \(\mathbf{x}_{0}\) is also a solution to \(Ax=b\).

Why?

Because \(B\) is invertible! \[\text{rref}(A)\mathbf{x}_{0} = Bb\quad \Rightarrow\quad BA\mathbf{x}_{0} = Bb\]

\[\quad \Rightarrow\quad B^{-1}BA\mathbf{x}_{0} = B^{-1}Bb\quad\Rightarrow\quad A\mathbf{x}_{0} = b \]

Finding a "particular" solution to \(Ax=b\)

Example. Consider the matrix equation \(Ax=b\):

\[\begin{bmatrix} 1 & 3 & 0 & -2 & -1\\ 2 & 6 & 2 & 0 & -2\\ 0 & 0 & 1 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix}=\begin{bmatrix} 1\\ 2\\ 1\end{bmatrix}\]

First, we find the matrix \(B\) so that \(BA=\text{rref}(A)\), and we find \(B^{-1}\).

\[B = \frac{1}{2}\begin{bmatrix} -2 & 2 & -4\\ 2 & -1 & 4\\ -2 & 1 & -2\end{bmatrix} \quad\text{and}\quad B^{-1}=\begin{bmatrix} 1 & 0 & -2\\ 2 & 2 & 0\\ 0 & 1 & 1\end{bmatrix}.\]

Multiplying by \(B\) on both sides of \(Ax=b\) we obtain

\[\begin{bmatrix} 1 & 3 & 0 & 0 & -1\\ 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix} = \begin{bmatrix} -1\\ 2\\ -1\end{bmatrix}\]

Finding a "particular" solution to \(Ax=b\)

Example. Consider the matrix equation \(Ax=b\):

\[\begin{bmatrix} 1 & 3 & 0 & -2 & -1\\ 2 & 6 & 2 & 0 & -2\\ 0 & 0 & 1 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix}=\begin{bmatrix} 1\\ 2\\ 1\end{bmatrix}\]

Note that we can compute \(Bb\) without finding \(B\). Indeed, we can perform the same row operations on \(b\) that we preformed on \(A\) to obtain \(\operatorname{rref}(A)\).

Multiplying by \(B\) on both sides of \(Ax=b\) we obtain

\[\begin{bmatrix} 1 & 3 & 0 & 0 & -1\\ 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix} = \begin{bmatrix} -1\\ 2\\ -1\end{bmatrix}\]

Finding a "particular" solution to \(Ax=b\)

Example continued.

\[\begin{bmatrix} 1 & 3 & 0 & 0 & -1\\ 0 & 0 & 1 & 0 & 0\\ 0 & 0 & 0 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix} = \begin{bmatrix} -1\\ 2\\ -1\end{bmatrix}\]

\[\mathbf{x}_{0} = \begin{bmatrix}\ &\ &\ &\ &\ &\ &\ \end{bmatrix}^{\top}\]

\[0\ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ \ 0\]

Constructing a solution \(\mathbf{x}_{0}\) to \(\text{rref}(A)x=Bb\).

  1. Put a zero in each row of \(\mathbf{x}_{0}\) corresponding to a non-pivot column of \(\text{rref}(A)\).
  2. For each row of \(\mathbf{x}_{0}\) corresponding to a pivot column of \(\text{rref}(A)\), if the pivot is in row \(i\), then the entry in \(\mathbf{x}_{0}\) is the entry in row \(i\) of \(Bb.\)

\(-1\)

\(2\)

\(-1\)

Warning: This algorithm might not produce a solution!

An equation without a solution

\[\begin{bmatrix} 1 & 0 & 1 & 0\\ 0 & 1 & 0& -2\\ 1 & 1 & 1 & -2\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\end{bmatrix} = \begin{bmatrix} 0\\ 3\\ 0\end{bmatrix}\] 

Multiply on the left by the matrix \(B\) so that \(BA=\text{rref}(A)\).

\[\begin{bmatrix} 1 & 0 & 1 & 0\\ 0 & 1 & 0 & -2\\ 0 & 0 & 0 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 1\end{bmatrix}\quad \] 

\[\Downarrow\]

\[\begin{bmatrix} x_{1}+x_{3}\\ x_{2}-x_{4}\\ 0\end{bmatrix} = \begin{bmatrix} 0\\ 0\\ 1\end{bmatrix}\] 

We cannot find numbers \(x_{1},x_{2},x_{3},x_{4}\) that satisfy this equation. Hence, the original equation has no solution.

What would be the "particular" solution we would write by the algorithm?

  • \(Ax=b\) has a solution if and only if \(b\in C(A)\).
  • \(Ax=b\) has exactly one solution if and only if \(b\in C(A)\) and \(\operatorname{nullity}(A)=0\)
  • \(Ax=b\) has infinitely many solutions if and only if \(b\in C(A)\) and \(\operatorname{nullity}(A)>0\).

Some takeaways

Quiz:

  1. For \(A\in\mathbb{R}^{m\times n}\) and \(b\in\mathbb{R}^{m}\) such that \(n\geq m\) and \(\operatorname{nullity}(A)=0\) it ___________ holds that \(Ax=b\) has a solution.
  2. For \(A\in\mathbb{R}^{n\times n}\) and \(b\in\mathbb{R}^{n}\) such that \(Ax=b\) has infinitely many solutions, it __________ holds that there is some \(c\in\mathbb{R}^{n}\) such that \(Ax=c\) has no solutions.
  3. For \(A\in\mathbb{R}^{m\times n}\) and \(b\in\mathbb{R}^{m}\) such that \(n\geq m\) and \(\operatorname{nullity}(A)=1\) it ___________ holds that \(Ax=b\) has infinitely many solutions.

always

always

sometimes

Solving \(Ax=b\)

Example. Consider the matrix equation \(Ax=b\):

\[\begin{bmatrix} 2 & 4 & 0 & -2\\ 2 & 4 & 1 & 3\\ 0 & 0 & 1 & 1\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\end{bmatrix}=\begin{bmatrix} 0\\ 0\\ -4\end{bmatrix}\]

\[\text{rref}(A) = \begin{bmatrix} 1 & 2 & 0 & 0\\ 0 & 0 & 1 & 0\\ 0 & 0 & 0 & 1\end{bmatrix}\]

 

We already found (last time) that \(\mathbf{x}_{0} = \begin{bmatrix} 1\\ 0\\ -5\\ 1\end{bmatrix}\) is a solution.

Hence, the set of solutions to \(Ax=b\) is \(\left\{\begin{bmatrix} 1\\ 0\\ -5\\ 1\end{bmatrix} + a\begin{bmatrix} -2\\ 1\\ 0\\ 0\end{bmatrix}: a\in\R\right\}\)

\(\Rightarrow\ \left\{\begin{bmatrix} -2\\ 1\\ 0\\ 0\end{bmatrix}\right\}\) is a basis for \(N(A)\)

Finding all solutions to \(Ax=b\)

A basis for \(N(A)\) is \(\left\{\begin{bmatrix} -3\\ 1\\ 0\\ 0\\ 0\end{bmatrix}, \begin{bmatrix} 1\\ 0\\ 0\\ 0\\ 1\end{bmatrix}\right\}\)

A solution is \(\mathbf{x}_{0} = \begin{bmatrix} -1\\ 0\\ 2\\ -1\\ 0 \end{bmatrix}\)

Hence, the set of solutions to \(Ax=b\) is

\[\left\{\begin{bmatrix} -1\\ 0\\ 2\\ -1\\ 0\end{bmatrix} + a_{1}\begin{bmatrix} -3\\ 1\\ 0\\ 0\\ 0\end{bmatrix} + a_{2}\begin{bmatrix} 1\\ 0\\ 0\\ 0\\ 1\end{bmatrix} : a_{1},a_{2}\in\R\right\}\]

Example. Consider the matrix equation \(Ax=b\):

\[\begin{bmatrix} 1 & 3 & 0 & -2 & -1\\ 2 & 6 & 2 & 0 & -2\\ 0 & 0 & 1 & 1 & 0\end{bmatrix}\begin{bmatrix} x_{1}\\ x_{2}\\ x_{3}\\ x_{4}\\ x_{5}\end{bmatrix}=\begin{bmatrix} 1\\ 2\\ 1\end{bmatrix}\]

Exercise. Which of the following vectors is in \(C(B)\) where

\[B = \left[\begin{array}{rrr} 1 & 0 & 2\\ -2 & 1 & -2\\ 3 & -1 & 4 \end{array}\right]?\]

\[\text{(a) }\left[\begin{array}{rrr} 0\\ 1\\ -1\end{array}\right]\ \text{(b) }\left[\begin{array}{rrr} 1\\ -1\\ 2\end{array}\right]\ \text{(c) } \left[\begin{array}{rrr} 0\\ -1\\ -1\end{array}\right]\ \text{(d) }\left[\begin{array}{rrr} 2\\ -1\\ 3 \end{array}\right]\ \text{(e) }\left[\begin{array}{rrr} 0\\ 0\\ 0\end{array}\right]\ \]

\[\left[\begin{array}{rrr|c} 1 & 0 & 2 & b_{1}\\ -2 & 1 & -2 & b_{2}\\ 3 & -1 & 4 & b_{3} \end{array}\right] \sim \left[\begin{array}{rrr|c} 1 & 0 & 2 & b_{2}+b_{2}\\ 0 & 1 & 2 & 3b_{2}+2b_{3}\\ 0 & 0 & 0 & b_{1} - b_{2} - b_{3} \end{array}\right]\]

Hence, \(Bx=\begin{bmatrix} b_{1}\\ b_{2}\\ b_{3}\end{bmatrix}\) has a solution if and only if \(b_{1}-b_{2}-b_{3} = 0\).

Linear Algebra Day 15

By John Jasper

Linear Algebra Day 15

  • 414