CS6015: Linear Algebra and Random Processes
Lecture 8: Solving Ax=0
Admin
Evaluation pattern
70% assignments
30% quizzes (2 quizzes)
Quiz 1
Date: 19th Oct
Syllabus: Everything covered till 16th Oct
Format: Online, need to keep video ON
We will be shifting to Cisco Webex (more instructions will follow)
Learning Objectives
What are pivot variables?
How to describe the null space of A? (or all solutions to Ax=0)
What are free variables?
(for today's lecture)
Recap: GE for solving Ax=0
111112342345
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
100011232123
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
(r2 = r2 - r1)
(r3 = r3 - r1)
(r4 = r4 - r1)
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
(r3 = r3 - 2r2)
(r4 = r4 - 3r2)
Obvious: GE will discover dependent rows (by producing zero rows)
Net effect: r3 = r3 - r1 - 2(r2 - r1) = r3 + r1 - 2r2 =0
(we get a 0 since r3 was a lin. comb. of r1,r2)
Not so obvious: GE will discover dependent columns
number of independent rows = number of non-zero pivots after GE
Theorem
(We will prove this formally later)
Hence, the not so obvious conclusion: GE will discover dependent columns
number of independent rows = number of non-zero pivots after GE = number of independent colums
Pivot variables and Free variables
111112342345
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
100011232123
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
x1x2x3
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
Why do we call them "free" variables?
You are free to set their value to whatever you want!
The pivot variables can then be adjusted to satisfy Ax=0
Pivot variables and Free variables
111112342345
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
100011232123
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
x1x2x3
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x1+x2+2x3=0
x_1 + x_2 + 2x_3 = 0
x2+x3=0
x_2 + x_3 = 0
Set x3=1
(you are free to choose any value)
x1+(−1)+2(1)=0
x_1 + (-1) + 2(1) = 0
x2+(1)=0
x_2 + (1) = 0
∴x2=−1
\therefore x_2 = -1
∴x1=−1
\therefore x_1 = -1
x=−1−11
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
is one solution
Pivot variables and Free variables
111112342345
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
100011232123
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
x1x2x3
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x1+x2+2x3=0
x_1 + x_2 + 2x_3 = 0
x2+x3=0
x_2 + x_3 = 0
Set x3=2
(you are free to choose any value)
x1+(−2)+2(2)=0
x_1 + (-2) + 2(2) = 0
x2+(2)=0
x_2 + (2) = 0
∴x2=−2
\therefore x_2 = -2
∴x1=−2
\therefore x_1 = -2
x=−2−22
\mathbf{x} = \begin{bmatrix}
-2\\
-2\\
2\end{bmatrix}
is another solution
Pivot variables and Free variables
111112342345
\begin{bmatrix}
1&1&2\\
1&2&3\\
1&3&4\\
1&4&5\\
\end{bmatrix}
100011232123
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&2&2\\
0&3&3\\
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
x1x2x3
\begin{bmatrix}
x_1\\
x_2\\
x_3\end{bmatrix}
Pivot variables
Free variable
x1+x2+2x3=0
x_1 + x_2 + 2x_3 = 0
x2+x3=0
x_2 + x_3 = 0
Set x3=c
(you are free to choose any value)
x1+(−c)+2(c)=0
x_1 + (-c) + 2(c) = 0
x2+(c)=0
x_2 + (c) = 0
∴x2=−c
\therefore x_2 = -c
∴x1=−c
\therefore x_1 = -c
x=c−1−11
\mathbf{x} = c\begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
an entire line of solutions
What if we have more free variables?
112123134145
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
x1x2x3x4
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
x1+x2+x3+x4=0
x_1 + x_2 + x_3 + x_4 = 0
x2+2x3+3x4=0
x_2 + 2x_3 + 3 x_4 = 0
Set x3=1,x4=0
(you are free to choose any value)
x1+(−2)+(1)+(0)=0
x_1 + (-2) + (1) + (0) = 0
∴x2=−2
\therefore x_2 = -2
∴x1=1
\therefore x_1 = 1
x= 1−210
\mathbf{x} = ~\begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
an entire line of solutions
100111122133
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
x2+2(1)+3(0)=0
x_2 + 2(1) + 3(0) = 0
c
c
What if we have more free variables?
112123134145
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
x1x2x3x4
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
x1+x2+x3+x4=0
x_1 + x_2 + x_3 + x_4 = 0
x2+2x3+3x4=0
x_2 + 2x_3 + 3 x_4 = 0
(you are free to choose any value)
x1+(−3)+(0)+(1)=0
x_1 + (-3) + (0) + (1) = 0
∴x2=−3
\therefore x_2 = -3
∴x1=2
\therefore x_1 = 2
x= 2−301
\mathbf{x} = ~~\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
a different entire line of solutions
100111122133
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
Set x3=0,x4=1
x2+2(0)+3(1)=0
x_2 + 2(0) + 3(1) = 0
d
d
What if we have more free variables?
112123134145
\begin{bmatrix}
1&1&1&1\\
1&2&3&4\\
2&3&4&5
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
x1x2x3x4
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
Pivot variables
Free variables
2−301
\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
a different entire line of solutions
100111122133
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&1&2&3
\end{bmatrix}
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
1−210
\begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
+
+
one entire line of solutions
all linear combinations will be solutions
Why?
because we know that the nullspace is a subspace
c
c
d
d
Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
(staircase form)
Reduced Row Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free columns
get zero above pivots
divide each row by the corresponding pivots
x=−1−11
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
x=1−210
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,2−301
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
100001001100
\begin{bmatrix}
1&0&1\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
100010−120−230
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
U
U
U
U
R
R
R
R
1−2102−301
\begin{bmatrix}
1&2\\
-2&-3\\
1&0\\
0&1
\end{bmatrix}
=00000000
=\begin{bmatrix}
0&0\\
0&0\\
0&0\\
0&0
\end{bmatrix}
X
X
0
\mathbf{0}
Reduced Row Echelon form
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
Pivot columns
\underbrace{~~~~~~~~~~}
\underbrace{~}
Free column
get zero above pivots
divide each row by the corresponding pivots
x=−1−11
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
x=1−210
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,2−301
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
100001001100
\begin{bmatrix}
1&0&1\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
U
U
U
U
R
R
−1−11
\begin{bmatrix}
-1\\
-1\\
1
\end{bmatrix}
=0000
=\begin{bmatrix}
0\\
0\\
0\\
0
\end{bmatrix}
X
X
[IF]
\begin{bmatrix}
I&F
\end{bmatrix}
[−FI]
\begin{bmatrix}
-F\\
I
\end{bmatrix}
=[−IF+IF]
= \begin{bmatrix}
-IF + IF
\end{bmatrix}
=[0]
=[0]
Dimension of Nullspace
Pivot columns
\underbrace{~~~~~~~~~~}
Free columns
100110120130
\begin{bmatrix}
1&1&1&1\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
\underbrace{~~~~~~~~~~}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
\underbrace{~~~~~~~~~~}
\underbrace{~}
rank = r = number of pivot columns
dimension of nullspace = n - r
x=−1−11
\mathbf{x} = \begin{bmatrix}
-1\\
-1\\
1\end{bmatrix}
x=1−210
\mathbf{x} = \begin{bmatrix}
1\\
-2\\
1\\
0
\end{bmatrix}
,2−301
,\begin{bmatrix}
2\\
-3\\
0\\
1
\end{bmatrix}
100011002100
\begin{bmatrix}
1&1&2\\
0&1&1\\
0&0&0\\
0&0&0\\
\end{bmatrix}
n = total number of columns
Pivot columns
Free column
Interleaved Pivot and Free Columns
112336527729
\begin{bmatrix}
1&3&5&7\\
1&3&2&2\\
2&6&7&9
\end{bmatrix}
A
A
X=−3100340−351
X = \begin{bmatrix}
-3&\frac{4}{3}\\
1&0\\
0&-\frac{5}{3}\\
0&1
\end{bmatrix}
Free columns
1003005−307−50
\begin{bmatrix}
1&3&5&7\\
0&0&-3&-5\\
0&0&0&0
\end{bmatrix}
Pivot columns
U
U
R
R
100300010−34350
\begin{bmatrix}
1&3&0&-\frac{4}{3}\\
0&0&1&\frac{5}{3}\\
0&0&0&0
\end{bmatrix}
100010−120−230
\begin{bmatrix}
1&0&-1&-2\\
0&1&2&3\\
0&0&0&0
\end{bmatrix}
1−2102−301
\begin{bmatrix}
1&2\\
-2&-3\\
1&0\\
0&1
\end{bmatrix}
Recap: Previous example
[I1F1I2F2]
\begin{bmatrix}
I_1&F_1&I_2&F_2
\end{bmatrix}
−F1I1−F2I2
\begin{bmatrix}
-F_1\\
I_1\\
-F_2\\
I_2
\end{bmatrix}
=[0]
= \begin{bmatrix}
\mathbf{0}
\end{bmatrix}
Set x2=0,x4=1
Set x2=1,x4=0
x1+3x2+5x3+7x4=0
x_1 + 3 x_2 + 5 x_3 + 7 x_4 = 0
−3x3−5x4=0
- 3 x_3 - 5 x_4 = 0
Learning Objectives
(achieved)
What are pivot variables?
How to describe the null space of A? (or all solutions to Ax=0)
What are free variables?
CS6015: Linear Algebra and Random Processes Lecture 8: Solving Ax=0
CS6015: Lecture 8
By Mitesh Khapra
CS6015: Lecture 8
Lecture 8: Solving Ax=0
- 2,718
