# The bigger picture

\begin{bmatrix} ~~~&~~~&~~~\\ ~~~&~~~&~~~\\ ~~~&~~~&~~~\\ \end{bmatrix}
m < n
m=n
\begin{bmatrix} ~~~&~~~&~~~&~~~&~~~\\ ~~~&~~~&~~~&~~~&~~~\\ ~~~&~~~&~~~&~~~&~~~\\ \end{bmatrix}
\begin{bmatrix} ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~& \end{bmatrix}
m > n
rank =
A
A
A

### Q1. For which b's does a solution not exist?

0,1,\infty~solutions

### Q3. What's the connection between these two Qs?

A\mathbf{x}=\mathbf{b}

A^{-1}

# Column space of a matrix

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{a_1}
\mathbf{a_2}
\mathbf{a_3}
\mathbf{x}
=\begin{bmatrix} b_1\\ b_2\\ b_3\\ b_4 \end{bmatrix}
\mathbf{b}

### column space: all linear combinations of the columns of A

(these are the b's which cannot be expressed as a linear combination of the columns of A)

# Column space of a matrix

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{a_1}
\mathbf{a_2}
\mathbf{a_3}
\mathbf{x}
=\begin{bmatrix} b_1\\ b_2\\ b_3\\ b_4 \end{bmatrix}
\mathbf{b}

# Column space of a matrix

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{a_1}
\mathbf{a_2}
\mathbf{a_3}
\mathbf{x}
=\begin{bmatrix} b_1\\ b_2\\ b_3\\ b_4 \end{bmatrix}
\mathbf{b}

### column space: all linear combinations of the columns of A

\mathbf{b}=\mathbf{a_1}
\mathbf{b}=\mathbf{a_2}
\mathbf{b}=\mathbf{a_3}
\mathbf{x} = [1~0~0]^\top
\mathbf{x} = [0~1~0]^\top
\mathbf{x} = [0~0~1]^\top
\mathbf{b}=2\mathbf{a_1}-\mathbf{a_2}+5\mathbf{a_3}
\mathbf{x} = [2~-1~5]^\top

# Column space of a matrix

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{a_1}
\mathbf{a_2}
\mathbf{a_3}
\mathbf{x}
=\begin{bmatrix} 1\\ 2\\ 3\\ 5 \end{bmatrix}
\mathbf{b}

# What does Gauss Elimination tell us?

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 1\\ 2\\ 3\\ 4 \end{bmatrix}
0=0
0=1
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
(row2 = row2 -row1)
(row3 = row3 -row1)
(row4 = row4 -row1)
\mathbf{x}
=\begin{bmatrix} 1\\ 1\\ 2\\ 3 \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 1\\ 1\\ 0\\ 0 \end{bmatrix}
(row3 = row3 - 2row2)
(row4 = row4 - 3row2)
=\begin{bmatrix} 1\\ 2\\ 3\\ 5 \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 1\\ 1\\ 0\\ 1 \end{bmatrix}

# The bigger picture

\begin{bmatrix} ~~~&~~~&~~~\\ ~~~&~~~&~~~\\ ~~~&~~~&~~~\\ \end{bmatrix}
m < n
m=n
\begin{bmatrix} ~~~&~~~&~~~&~~~&~~~\\ ~~~&~~~&~~~&~~~&~~~\\ ~~~&~~~&~~~&~~~&~~~\\ \end{bmatrix}
\begin{bmatrix} ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~&\\ ~~~&~~~& \end{bmatrix}
m > n
rank =
A
A
A

### Q1. For which b's does a solution not exist?

0,1,\infty~solutions

### Q3. What's the connection between these two Qs?

A\mathbf{x}=\mathbf{b}

A^{-1}

# Ax = 0

## Linear equations

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 0\\ 0\\ 0\\ 0 \end{bmatrix}

### A: $$\mathbf{x} = \mathbf{0}$$ is always a solution for $$A\mathbf{x} = 0$$ for any $$A$$

(a 0-combination of the columns of A will always produce a 0 vector)

### Q: When would a non-0 comb. of the columns of $$A$$ produce a 0 vector?

0-combination: a linear combination where each vector is scaled by 0 (my informal term)


### A: When the columns of $$A$$ are linearly dependent!

(this simply follows from the definition of independence)

# Ax = 0

## Linear equations

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 0\\ 0\\ 0\\ 0 \end{bmatrix}

### A: $$\mathbf{x} = [c~~c~-c]^\top$$

(a line)


# Ax = 0

## Linear equations

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} 0\\ 0\\ 0\\ 0 \end{bmatrix}

# What does Gauss Elimination tell us?

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&2\\ 0&3&3\\ \end{bmatrix}
(r2 = r2 - r1)
(r3 = r3 - r1)
(r4 = r4 - r1)
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&0\\ 0&0&0\\ \end{bmatrix}
(r3 = r3 - 2r2)
(r4 = r4 - 3r2)

### Why?

We are not ready to fully understand that yet but we will get there soon
\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&5\\ 1&4&5\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&2&3\\ 0&3&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2\\ 0&1&1\\ 0&0&1\\ 0&0&0\\ \end{bmatrix}

# The nullspace of A: $$\mathcal{N}(A)$$

### To prove that if $$\mathbf{u} \in N(A)$$ and $$\mathbf{v} \in N(A)$$ then $$c\mathbf{u} + d\mathbf{v} \in N(A)$$

\because \mathbf{u},\mathbf{v} \in \mathcal{N}(A), ~~A\mathbf{u}=A\mathbf{v}=\mathbf{0}
A(c\mathbf{u} + d\mathbf{v}) = A (c\mathbf{u}) + A (d\mathbf{v})
= c(A\mathbf{u}) + d(A\mathbf{v})
= c(\mathbf{0}) + d(\mathbf{0})
= \mathbf{0}
Hence~proved, c\mathbf{u} + d\mathbf{v} \in \mathcal{N}(A)

# Summary: The two subspaces

A\mathbf{x} = \mathbf{b}
m\times n
n\times 1
m\times 1

### the cols. of A are indep. when $$\mathcal{N}(A) = \mathbf{0}$$

\begin{bmatrix} 1&1&2\\ 1&2&3\\ 1&3&4\\ 1&4&5\\ \end{bmatrix}
\mathbf{x}
=\begin{bmatrix} b_1\\ b_2\\ b_3\\ b_4 \end{bmatrix}

# Why do we care about these subspaces

### What does the null space tell us?

(a solution to Ax = b exists)

### How many solutions exist for $$A\mathbf{x} = \mathbf{b}$$?

(a solution to Ax = b exists)
(a line or a 2d plane or a 3d plane or ...)

# Why do we care about these subspaces

(a solution to Ax = b exists)

### How many solutions exist for $$\mathbf{b}$$?

(a line or a 2d plane or a 3d plane or ...)

### Let $$\mathbf{x} \in \mathbb{R}^n$$ be a solution for  $$A\mathbf{x} = \mathbf{b}$$

(we know that a solution exists)
A(\mathbf{x} + c\mathbf{u})
= A\mathbf{x} + cA\mathbf{u}
= \mathbf{b} + 0
=\mathbf{b}
c \in \mathbb{R}
\mathbf{x} + c\mathbf{u}

### is also a solution $$\forall c \in \mathbb{R}$$

(a whole line, infinite solutions)

# Why do we care about these subspaces

Is b in the column space of A?
Is nullspace of A non-zero
No
Yes
No
Yes
1~solution
\infty~solutions
0~solutions

0=k

### GE will discover this

\#pivots = \#ind. cols.

# Feedback

### The involvement of TAs is not effective

Any concrete suggestions on what can be done?

### More practice problems, answers to tutorials

Please refer to specific sections in the textbook
Also refer to http://web.mit.edu/18.06/www/old.shtml

### More quizzes/short tests

Did you mean ungraded quizzes/tests? (logistics)


already being done (please check my webpage)


### Doubts: don't entertain, take only at the end, can be handled by TAs

sorry, can't really do that


### Increase the pace

trying my best, will try to makeup with extra classes


### More applications

this is a theory course with applications in other courses


### Latex is painful!

sorry, I don't agree

\begin{bmatrix} 1&1&2&2\\ 1&2&3&2\\ 1&3&5&1\\ 1&4&5&3\\ \end{bmatrix}
\begin{bmatrix} 1&1&2&2\\ 0&1&1&0\\ 0&2&3&-1\\ 0&3&3&1\\ \end{bmatrix}
\begin{bmatrix} 1&1&2&2\\ 0&1&1&0\\ 0&0&1&-1\\ 0&0&0&1\\ \end{bmatrix}
\begin{bmatrix} 1&1&2&2\\ 0&1&1&0\\ 0&0&1&-1\\ 0&0&0&1\\ \end{bmatrix}
\begin{bmatrix} 1&1&2&2\\ 0&1&1&0\\ 0&0&1&-1\\ 0&0&0&1\\ \end{bmatrix}
\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}
x_1 + x_2 + 2 x_3 + 2 x_4 = 0
x_2 + x_3 = 0
x_3 + (-1)x_4 = 0
x_4 = 0