CS6015: Linear Algebra and Random Processes

Lecture 8: Solving Ax=0

Admin

Evaluation pattern

70% assignments

30% quizzes (2 quizzes)

Quiz 1

Date: 19th Oct

Syllabus: Everything covered till 16th Oct

Format: Online, need to keep video ON

We will be shifting to Cisco Webex (more instructions will follow)

Learning Objectives

What are pivot variables?

How to describe the null space of A? (or all solutions to Ax=0)

What are free variables?

(for today's lecture)

Recap: GE for solving Ax=0

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
(r2 = r2 - r1) 
(r3 = r3 - r1) 
(r4 = r4 - r1) 
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
(r3 = r3 - 2r2) 
(r4 = r4 - 3r2) 

Obvious: GE will discover dependent rows (by producing zero rows)

Net effect: r3 = r3 - r1 - 2(r2 - r1) = r3 + r1 - 2r2 =0 
(we get a 0 since r3 was a lin. comb. of r1,r2)

Not so obvious: GE will discover dependent columns 

number of independent rows = number of non-zero pivots after GE

Theorem

(We will prove this formally later)

Hence, the not so obvious conclusion: GE will discover dependent columns 

number of independent rows = number of non-zero pivots after GE = number of independent colums

Pivot variables and Free variables

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

\begin{bmatrix} x_1\\ x_2\\ x_3\end{bmatrix}

Pivot variables

Free variable

Why do we call them "free" variables?

You are free to set their value to whatever you want!

The pivot variables can then be adjusted to satisfy \(A\mathbf{x} = \mathbf{0} \)  

Pivot variables and Free variables

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

\begin{bmatrix} x_1\\ x_2\\ x_3\end{bmatrix}

Pivot variables

Free variable

x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0

Set \( x_3 = 1 \)

(you are free to choose any value)
x_1 + (-1) + 2(1) = 0
x_2 + (1) = 0
\therefore x_2 = -1
\therefore x_1 = -1
\mathbf{x} = \begin{bmatrix} -1\\ -1\\ 1\end{bmatrix}
is one solution

Pivot variables and Free variables

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

\begin{bmatrix} x_1\\ x_2\\ x_3\end{bmatrix}

Pivot variables

Free variable

x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0

Set \( x_3 = 2 \)

(you are free to choose any value)
x_1 + (-2) + 2(2) = 0
x_2 + (2) = 0
\therefore x_2 = -2
\therefore x_1 = -2
\mathbf{x} = \begin{bmatrix} -2\\ -2\\ 2\end{bmatrix}
is another solution

Pivot variables and Free variables

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

\begin{bmatrix} x_1\\ x_2\\ x_3\end{bmatrix}

Pivot variables

Free variable

x_1 + x_2 + 2x_3 = 0
x_2 + x_3 = 0

Set \( x_3 = c \)

(you are free to choose any value)
x_1 + (-c) + 2(c) = 0
x_2 + (c) = 0
\therefore x_2 = -c
\therefore x_1 = -c
\mathbf{x} = c\begin{bmatrix} -1\\ -1\\ 1\end{bmatrix}
an entire line of solutions

What if we have more free variables?

\begin{bmatrix} 1&1&1&1\\ 1&2&3&4\\ 2&3&4&5 \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}

Pivot variables

Free variables

x_1 + x_2 + x_3 + x_4 = 0
x_2 + 2x_3 + 3 x_4 = 0

Set \( x_3 = 1, x_4 = 0 \)

(you are free to choose any value)
x_1 + (-2) + (1) + (0) = 0
\therefore x_2 = -2
\therefore x_1 = 1
\mathbf{x} = ~\begin{bmatrix} 1\\ -2\\ 1\\ 0 \end{bmatrix}
an entire line of solutions
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&1&2&3 \end{bmatrix}
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
x_2 + 2(1) + 3(0) = 0
c

What if we have more free variables?

\begin{bmatrix} 1&1&1&1\\ 1&2&3&4\\ 2&3&4&5 \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}

Pivot variables

Free variables

x_1 + x_2 + x_3 + x_4 = 0
x_2 + 2x_3 + 3 x_4 = 0
(you are free to choose any value)
x_1 + (-3) + (0) + (1) = 0
\therefore x_2 = -3
\therefore x_1 = 2
\mathbf{x} = ~~\begin{bmatrix} 2\\ -3\\ 0\\ 1 \end{bmatrix}
a different entire line of solutions
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&1&2&3 \end{bmatrix}
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}

Set \( x_3 = 0, x_4 = 1 \)

x_2 + 2(0) + 3(1) = 0
d

What if we have more free variables?

\begin{bmatrix} 1&1&1&1\\ 1&2&3&4\\ 2&3&4&5 \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}

Pivot variables

Free variables

\begin{bmatrix} 2\\ -3\\ 0\\ 1 \end{bmatrix}
a different entire line of solutions
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&1&2&3 \end{bmatrix}
\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix} 1\\ -2\\ 1\\ 0 \end{bmatrix}
+
one entire line of solutions
all linear combinations will be solutions 

Why? 

because we know that the nullspace is a subspace

c
d

Echelon form

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

(staircase form)

Reduced Row Echelon form

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free columns

get zero above pivots

divide each row by the corresponding pivots

\mathbf{x} = \begin{bmatrix} -1\\ -1\\ 1\end{bmatrix}
\mathbf{x} = \begin{bmatrix} 1\\ -2\\ 1\\ 0 \end{bmatrix}
,\begin{bmatrix} 2\\ -3\\ 0\\ 1 \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\begin{bmatrix} 1&0&1\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\begin{bmatrix} 1&0&-1&-2\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
U
U
R
R
\begin{bmatrix} 1&2\\ -2&-3\\ 1&0\\ 0&1 \end{bmatrix}
=\begin{bmatrix} 0&0\\ 0&0\\ 0&0\\ 0&0 \end{bmatrix}
X
\mathbf{0}

Reduced Row Echelon form

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

Pivot columns

\underbrace{~~~~~~~~~~}
\underbrace{~}

Free column

get zero above pivots

divide each row by the corresponding pivots

\mathbf{x} = \begin{bmatrix} -1\\ -1\\ 1\end{bmatrix}
\mathbf{x} = \begin{bmatrix} 1\\ -2\\ 1\\ 0 \end{bmatrix}
,\begin{bmatrix} 2\\ -3\\ 0\\ 1 \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\begin{bmatrix} 1&0&1\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
U
U
R
\begin{bmatrix} -1\\ -1\\ 1 \end{bmatrix}
=\begin{bmatrix} 0\\ 0\\ 0\\ 0 \end{bmatrix}
X
\begin{bmatrix} I&F \end{bmatrix}
\begin{bmatrix} -F\\ I \end{bmatrix}
= \begin{bmatrix} -IF + IF \end{bmatrix}
=[0]

Dimension of Nullspace

Pivot columns

\underbrace{~~~~~~~~~~}

Free columns

\begin{bmatrix} 1&1&1&1\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\underbrace{~~~~~~~~~~}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\underbrace{~~~~~~~~~~}
\underbrace{~}

rank = r = number of pivot columns

dimension of nullspace =  n - r

\mathbf{x} = \begin{bmatrix} -1\\ -1\\ 1\end{bmatrix}
\mathbf{x} = \begin{bmatrix} 1\\ -2\\ 1\\ 0 \end{bmatrix}
,\begin{bmatrix} 2\\ -3\\ 0\\ 1 \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}

n = total number of columns

Pivot columns

Free column

Interleaved Pivot and Free Columns

\begin{bmatrix} 1&3&5&7\\ 1&3&2&2\\ 2&6&7&9 \end{bmatrix}
A
X = \begin{bmatrix} -3&\frac{4}{3}\\ 1&0\\ 0&-\frac{5}{3}\\ 0&1 \end{bmatrix}

Free columns

\begin{bmatrix} 1&3&5&7\\ 0&0&-3&-5\\ 0&0&0&0 \end{bmatrix}

Pivot columns

U
R
\begin{bmatrix} 1&3&0&-\frac{4}{3}\\ 0&0&1&\frac{5}{3}\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&0&-1&-2\\ 0&1&2&3\\ 0&0&0&0 \end{bmatrix}
\begin{bmatrix} 1&2\\ -2&-3\\ 1&0\\ 0&1 \end{bmatrix}

Recap: Previous example

\begin{bmatrix} I_1&F_1&I_2&F_2 \end{bmatrix}
\begin{bmatrix} -F_1\\ I_1\\ -F_2\\ I_2 \end{bmatrix}
= \begin{bmatrix} \mathbf{0} \end{bmatrix}

Set \( x_2 = 0, x_4 = 1 \)

Set \( x_2 = 1, x_4 = 0 \)

x_1 + 3 x_2 + 5 x_3 + 7 x_4 = 0
- 3 x_3 - 5 x_4 = 0

Learning Objectives

(achieved)

What are pivot variables?

How to describe the null space of A? (or all solutions to Ax=0)

What are free variables?

Made with Slides.com