CS6015: Linear Algebra and Random Processes
Lecture 8: Solving Ax=0
Admin
Evaluation pattern
70% assignments
30% quizzes (2 quizzes)
Quiz 1
Date: 19th Oct
Syllabus: Everything covered till 16th Oct
Format: Online, need to keep video ON
We will be shifting to Cisco Webex (more instructions will follow)
Learning Objectives
What are pivot variables?
How to describe the null space of A? (or all solutions to Ax=0)
What are free variables?
(for today's lecture)
Recap: GE for solving Ax=0
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
(r2 = r2 - r1)
(r3 = r3 - r1)
(r4 = r4 - r1)
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
(r3 = r3 - 2r2)
(r4 = r4 - 3r2)
Obvious: GE will discover dependent rows (by producing zero rows)
Net effect: r3 = r3 - r1 - 2(r2 - r1) = r3 + r1 - 2r2 =0
(we get a 0 since r3 was a lin. comb. of r1,r2)
Not so obvious: GE will discover dependent columns
number of independent rows = number of non-zero pivots after GE
Theorem
(We will prove this formally later)
Hence, the not so obvious conclusion: GE will discover dependent columns
number of independent rows = number of non-zero pivots after GE = number of independent colums
Pivot variables and Free variables
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
Why do we call them "free" variables?
You are free to set their value to whatever you want!
The pivot variables can then be adjusted to satisfy \(A\mathbf{x} = \mathbf{0} \)
Pivot variables and Free variables
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0
Set \( x_3 = 1 \)
(you are free to choose any value)
x_1 + (-1) + 2(1) = 0
x_2 + (1) = 0
\therefore x_2 = -1
\therefore x_1 = -1
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
is one solution
Pivot variables and Free variables
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0
Set \( x_3 = 2 \)
(you are free to choose any value)
x_1 + (-2) + 2(2) = 0
x_2 + (2) = 0
\therefore x_2 = -2
\therefore x_1 = -2
\mathbf{x} = \begin{bmatrix}
-2\\
-2\\
2\end{bmatrix}
is another solution
Pivot variables and Free variables
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0
Set \( x_3 = c \)
(you are free to choose any value)
x_1 + (-c) + 2(c) = 0
x_2 + (c) = 0
\therefore x_2 = -c
\therefore x_1 = -c
\mathbf{x} = c\begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
an entire line of solutions
What if we have more free variables?
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
x_1 + x_2 + x_3 + x_4 = 0
x_2 + 2x_3 + 3 x_4 = 0
Set \( x_3 = 1, x_4 = 0 \)
(you are free to choose any value)
x_1 + (-2) + (1) + (0) = 0
\therefore x_2 = -2
\therefore x_1 = 1
\mathbf{x} = ~\begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
an entire line of solutions
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
x_2 + 2(1) + 3(0) = 0
c
What if we have more free variables?
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
x_1 + x_2 + x_3 + x_4 = 0
x_2 + 2x_3 + 3 x_4 = 0
(you are free to choose any value)
x_1 + (-3) + (0) + (1) = 0
\therefore x_2 = -3
\therefore x_1 = 2
\mathbf{x} = ~~\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
a different entire line of solutions
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
Set \( x_3 = 0, x_4 = 1 \)
x_2 + 2(0) + 3(1) = 0
d
What if we have more free variables?
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
a different entire line of solutions
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
+
one entire line of solutions
all linear combinations will be solutions
Why?
because we know that the nullspace is a subspace
c
d
Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
(staircase form)
Reduced Row Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free columns
get zero above pivots
divide each row by the corresponding pivots
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\begin{bmatrix}
1&0&1\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
U
U
R
R
\begin{bmatrix}
1&2\\
-2&-3\\
1&0\\
0&1
\end{bmatrix}
=\begin{bmatrix}
0&0\\
0&0\\
0&0\\
0&0
\end{bmatrix}
X
\mathbf{0}
Reduced Row Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
get zero above pivots
divide each row by the corresponding pivots
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\begin{bmatrix}
1&0&1\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
U
U
R
\begin{bmatrix}
-1\\
-1\\
1
\end{bmatrix}
=\begin{bmatrix}
0\\
0\\
0\\
0
\end{bmatrix}
X
\begin{bmatrix}
I&F
\end{bmatrix}
\begin{bmatrix}
-F\\
I
\end{bmatrix}
= \begin{bmatrix}
-IF + IF
\end{bmatrix}
=[0]
Dimension of Nullspace
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\underbrace{~}
rank = r = number of pivot columns
dimension of nullspace = n - r
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
n = total number of columns
Pivot columns
Free column
Interleaved Pivot and Free Columns
\begin{bmatrix}
1&3&5&7\\
1&3&2&2\\
2&6&7&9
\end{bmatrix}
A
X = \begin{bmatrix}
-3&\frac{4}{3}\\
1&0\\
0&-\frac{5}{3}\\
0&1
\end{bmatrix}
Free columns
\begin{bmatrix}
1&3&5&7\\
0&0&-3&-5\\
0&0&0&0
\end{bmatrix}
Pivot columns
U
R
\begin{bmatrix}
1&3&0&-\frac{4}{3}\\
0&0&1&\frac{5}{3}\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\begin{bmatrix}
1&2\\
-2&-3\\
1&0\\
0&1
\end{bmatrix}
Recap: Previous example
\begin{bmatrix}
I_1&F_1&I_2&F_2
\end{bmatrix}
\begin{bmatrix}
-F_1\\
I_1\\
-F_2\\
I_2
\end{bmatrix}
= \begin{bmatrix}
\mathbf{0}
\end{bmatrix}
Set \( x_2 = 0, x_4 = 1 \)
Set \( x_2 = 1, x_4 = 0 \)
x_1 + 3 x_2 + 5 x_3 + 7 x_4 = 0
- 3 x_3 - 5 x_4 = 0