CS6015: Linear Algebra and Random Processes
Lecture 2: Fun with matrix multiplication, System of linear equations
Learning Objectives
How do you multiply two matrices? (4 ways)
What is the transpose of a matrix?
What is the inverse of a matrix?
What are some of the laws of matrix multiplication?
What is a symmetric matrix?
What are some interesting matrices which lead to special products?
How do you represent a system of linear equations using matrices?
(for today's lecture)
Recap
(with a puzzle)
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Find~one~such~\mathbf{x}
A\mathbf{x} =\begin{bmatrix}
4\\
3\\
3\\
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
0\\
1\\
2\\
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
5\\
6\\
5\\
\end{bmatrix}
\mathbf{x} =\begin{bmatrix}
0\\
1\\
1\\
0\\
\end{bmatrix}
\mathbf{x} =\begin{bmatrix}
0\\
0\\
0\\
1\\
\end{bmatrix}
\mathbf{x} =\begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
2\\
1\\
2\\
\end{bmatrix}
\mathbf{x} =\begin{bmatrix}
0\\
1\\
0\\
0\\
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
0\\
-1\\
1\\
\end{bmatrix}
\mathbf{x} =\begin{bmatrix}
0\\
1\\
-1\\
0\\
\end{bmatrix}
Multiplying two matrices
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
Method 1: Multiply the rows of A by columns of B (1 row, 1 column at a time)
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
-5
0
-1
4
5
-4
\begin{bmatrix}
2\\
1\\
2\\
1
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
1\\
-2\\
-1\\
2\\
\end{bmatrix}
\begin{bmatrix}
0\\
2\\
1\\
2
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
1\\
-2\\
-1\\
2\\
\end{bmatrix}
\begin{bmatrix}
2\\
-1\\
2\\
-2\\
\end{bmatrix}
\begin{bmatrix}
1\\
2\\
2\\
0\\
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
2\\
1\\
2\\
1
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
2\\
-1\\
2\\
-2\\
\end{bmatrix}
\begin{bmatrix}
0\\
2\\
1\\
2
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
2\\
-1\\
2\\
-2\\
\end{bmatrix}
\begin{bmatrix}
1\\
-2\\
-1\\
2\\
\end{bmatrix}
\begin{bmatrix}
1\\
2\\
2\\
0\\
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
Multiplying two matrices
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
Method 1: Multiply the rows of A by columns of B (1 row, 1 column at a time)
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
-5
0
-1
4
5
-4
c_{12}
= A_{11}B_{12} + A_{12}B_{22} + A_{13}B_{32} + A_{14}B_{42}
=\sum_{k=1}^n A_{1k}B_{k2}
c_{ij}
= A_{i1}B_{1j} + A_{i2}B_{2j} + A_{i3}B_{3j} + A_{i4}B_{4j}
c_{32}
=\sum_{k=1}^n A_{3k}B_{k2}
=\sum_{k=1}^n A_{ik}B_{kj}
\begin{bmatrix}
2\\
-1\\
2\\
-2\\
\end{bmatrix}
\begin{bmatrix}
1\\
2\\
2\\
0\\
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
0\\
2\\
1\\
2
\end{bmatrix}
\begin{matrix}
*\\
*\\
*\\
\end{matrix}
\begin{matrix}
~\\
+\\
+\\
\end{matrix}
\begin{bmatrix}
2\\
-1\\
2\\
-2\\
\end{bmatrix}
= A_{31}B_{12} + A_{32}B_{22} + A_{33}B_{32} + A_{34}B_{42}
Multiplying two matrices
Method2: Take linear combinations of columns of A (1 column of B at a time)
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
\begin{bmatrix}
1\\
2\\
0\\
\end{bmatrix}
+
1
\begin{bmatrix}
2\\
1\\
2\\
\end{bmatrix}
+
-2
\begin{bmatrix}
2\\
2\\
1\\
\end{bmatrix}
+
-1
\begin{bmatrix}
0\\
1\\
2\\
\end{bmatrix}
2
\begin{bmatrix}
1\\
2\\
0\\
\end{bmatrix}
+
2
\begin{bmatrix}
2\\
1\\
2\\
\end{bmatrix}
+
-1
\begin{bmatrix}
2\\
2\\
1\\
\end{bmatrix}
+
2
\begin{bmatrix}
0\\
1\\
2\\
\end{bmatrix}
-2
Multiplying two matrices
Method3: Take linear combinations of rows of B (1 row of A at a time)
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
\begin{bmatrix}
~~~1 & ~~~2\\
\end{bmatrix}
+
1
+
2
+
0
\begin{bmatrix}
-2 & -1\\
\end{bmatrix}
\begin{bmatrix}
-1 & ~~~2\\
\end{bmatrix}
\begin{bmatrix}
~~~2 & -2\\
\end{bmatrix}
2
\begin{bmatrix}
~~~1 & ~~~2\\
\end{bmatrix}
+
2
+
1
+
1
\begin{bmatrix}
-2 & -1\\
\end{bmatrix}
\begin{bmatrix}
-1 & ~~~2\\
\end{bmatrix}
\begin{bmatrix}
~~~2 & -2\\
\end{bmatrix}
2
\begin{bmatrix}
~~~1 & ~~~2\\
\end{bmatrix}
+
0
+
2
+
2
\begin{bmatrix}
-2 & -1\\
\end{bmatrix}
\begin{bmatrix}
-1 & ~~~2\\
\end{bmatrix}
\begin{bmatrix}
~~~2 & -2\\
\end{bmatrix}
1
Multiplying two matrices
Method4: One column of A and one row of B at a time
-5
0
-1
4
5
-4
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
\begin{bmatrix}
1 & 2 \\ 2 & 4 \\ 0 & 0
\end{bmatrix}
\begin{bmatrix}
-4 & -2 \\ -2 & -1 \\ -4 & -2
\end{bmatrix}
\begin{bmatrix}
-2 & 4 \\ -2 & 4 \\ -1 & 2
\end{bmatrix}
\begin{bmatrix}
0& 0 \\ 2 & -2 \\ 4 & -4
\end{bmatrix}
+
+
+
\begin{bmatrix}
1 \\ 2 \\ 0
\end{bmatrix}
\begin{bmatrix}
~~~1 & ~~~2\\
\end{bmatrix}
\mathbb{R}^{3 \times 1}
\mathbb{R}^{1 \times 2}
\begin{bmatrix}
2 \\ 1 \\ 2
\end{bmatrix}
\begin{bmatrix}
-2 & -1\\
\end{bmatrix}
\begin{bmatrix}
2 \\ 2 \\ 1
\end{bmatrix}
\begin{bmatrix}
-1 & ~~~2\\
\end{bmatrix}
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
\begin{bmatrix}
0 \\ 1 \\ 2
\end{bmatrix}
\begin{bmatrix}
2 & -2\\
\end{bmatrix}
Multiplying two matrices
Method4: One column of A and one row of B at a time
= \begin{bmatrix}
~~~~ & ~~~~\\
~~~~ & ~~~~\\
~~~~ & ~~~~\\
\end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
c_{ij}
=\sum_{k=1}^n A_{ik}B_{kj}
\begin{bmatrix}
1 & 2 \\ 2 & 4 \\ 0 & 1
\end{bmatrix}
\begin{bmatrix}
-4 & -2 \\ -2 & -1 \\ -4 & -2
\end{bmatrix}
\begin{bmatrix}
-2 & 4 \\ -2 & 4 \\ -1 & 2
\end{bmatrix}
\begin{bmatrix}
0& 0 \\ 2 & -2 \\ 4 & -4
\end{bmatrix}
+
+
+
c_{11}
= A_{11}B_{11} + A_{12}B_{21} + A_{13}B_{31} + A_{14}B_{41}
=\sum_{k=1}^n A_{1k}B_{k1}
A_{11}B_{11}
A_{12}B_{21}
A_{13}B_{31}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
A_{14}B_{41}
\begin{bmatrix}
1 \\ 2 \\ 0
\end{bmatrix}
\begin{bmatrix}
~~~1 & ~~~2\\
\end{bmatrix}
Multiplying two matrices
rows multiplied by columns
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
(produces one cell at a time)
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
linear combination of A's columns
(produces one column at a time)
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
linear combination of B's rows
(produces one row at a time)
A =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix}
1 & 2 \\
-2 & -1 \\
-1 & 2 \\
2 & -2 \\
\end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
columns multiplied by rows
(produces one matrix at a time)
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
I =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0& 0 & 1\\
\end{bmatrix}
IB =
\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix} = B
identity matrix
(diagonal entries 1 off-diagonal entries 0)
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
P =\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0& 0 & 1\\
\end{bmatrix}
PB =
\begin{bmatrix}
2 & 1 & 2 & 1\\
1 & 2 & 2 & 0\\
0& 2 & 1 &2\\
\end{bmatrix}
permutation matrix
(identity matrix with rows exchanged)
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
0
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
1
*
\begin{bmatrix}
0&2 & 1 & 2
\end{bmatrix}
+
0
*
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
1
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
0
*
\begin{bmatrix}
0&2 & 1 & 2
\end{bmatrix}
+
0
*
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
P =\begin{bmatrix}
0 & 1 & 0 & 0\\
1 & 0 & 0 & 0\\
0 & 0 & 0 & 1\\
0& 0 & 1 & 0
\end{bmatrix}
BP =
permutation matrix
How many n x n permutation matrices are possible?
(Hint: what's the matrix called?)
n!
\begin{bmatrix}
0 & 1 & 0 & 0
\end{bmatrix}
1
*
+
\begin{bmatrix}
1 & 0 & 0 & 0
\end{bmatrix}
2
*
\begin{bmatrix}
0&0 & 0 & 1
\end{bmatrix}
+
2
*
\begin{bmatrix}
0&0 & 1 & 0
\end{bmatrix}
+
0
*
\begin{bmatrix}
2 & 1 & 0 & 2\\
1 & 2 & 1 & 2\\
2& 0 & 2 &1\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
P_2 =\begin{bmatrix}
0 & 1 & 0 & 0\\
1 & 0 & 0 & 0\\
0 & 0 & 0 & 1\\
0& 0 & 1 & 0
\end{bmatrix}
permutation matrix
P_1 =\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0& 0 & 1\\
\end{bmatrix}
Pre-multiplying a matrix with a permutation matrix exchanges its rows
Post-multiplying a matrix with a permutation matrix exchanges its cols.
permutation matrix
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
E =\begin{bmatrix}
1 & 0 & 0\\
-1 & 1 & 0\\
0& 0 & 1\\
\end{bmatrix}
EB =
\begin{bmatrix}
1 & 2 & 2 & 0\\
1 & -1 & 0 & 1\\
0 & 2 & 1 & 2\\
\end{bmatrix}
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
1
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
0
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
0
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
1
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
0
*
-1
*
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
EB =
\begin{bmatrix}
1 & 2 & 2 & 0\\
1 & -1 & 0 & 1\\
2 & 3 & 3 & 3\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
E =\begin{bmatrix}
1 & 0 & 0\\
-1 & 1 & 0\\
0& 1 & 1\\
\end{bmatrix}
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
1
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
0
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
0
*
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
0
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
1
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
1
*
Fun with matrix multiplication
(some puzzles)
EB =
\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
-4 & 0 & -3 & 0\\
\end{bmatrix}
(subtracting 2 times row 2 from row 3)
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
E =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0& -2 & 1\\
\end{bmatrix}
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
1
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
0
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
0
*
\begin{bmatrix}
1 & 2 & 2 & 0
\end{bmatrix}
0
*
\begin{bmatrix}
0&2 & 1& 2
\end{bmatrix}
+
1
*
+
\begin{bmatrix}
2 & 1 & 2 & 1
\end{bmatrix}
-2
*
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
C =\begin{bmatrix}
1 & 2 & 0\\
2 & 1 & 2\\
2 & 2 & 1\\
0 & 1 & 2
\end{bmatrix}
CB =
\begin{bmatrix}
~~~ & ~~~ & ~~~ & ~~~\\
~~~ & ~~~ & ~~~ & ~~~\\
~~~ & ~~~ & ~~~ & ~~~\\
~~~ & ~~~ & ~~~ & ~~~
\end{bmatrix}
2
5
4
5
6
8
9
4
4
9
8
5
5
4
6
2
square symmetric matrix
(why did we get this? was there something special about C and B?)
C_{ij} = B_{ji}
C = B^\top
B = C^\top
transposes
Ok, but why was the product a square symmetric matrix?
S = S^T
square symmetric matrix
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
C =\begin{bmatrix}
1 & 2 & 0\\
2 & 1 & 2\\
2 & 2 & 1\\
0 & 1 & 2
\end{bmatrix}
(AB)^\top = B^\top A^\top
Result:
Proof:
(Homework 1 :-) )
Hint:
Is the th entry of , the same as the th entry of
(AB)^\top
(i,j)
(i,j)
B^\top A^\top
C = B^\top
(CB)^\top = B^\top C^\top = CB
Hence, if C is the transpose of B,CB will always be a square symmetric matrix
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
C =\begin{bmatrix}
1 & 2 & 0\\
2 & 1 & 2\\
2 & 2 & 1\\
0 & 1 & 2
\end{bmatrix}
C = B^\top
B^\top = C
(CB)^\top = B^\top C^\top = CB
Hence, if C is the transpose of B,CB will always be a square symmetric matrix
(we will now use this result)
(AB)^\top = B^\top A^\top
Result:
is always a square symmetric matrix
A^\top A
Fun with matrix multiplication
(some puzzles)
B =\begin{bmatrix}
1 & 2 & 2 & 0\\
2 & 1 & 2 & 1\\
0& 2 & 1 &2\\
\end{bmatrix}
E_1 =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & -2 & 1\\
\end{bmatrix}
(subtracting 2 times row 2 from row 3)
What~is~E_2?
E_2E_1B = B
E_2 =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 2 & 1\\
\end{bmatrix}
(add 2 times row 2 to row 3 - the inverse of what E1 did)
E_2E_1 =
\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 2 & 1\\
\end{bmatrix}
\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & -2 & 1\\
\end{bmatrix}
= \begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 1\\
\end{bmatrix} = I
Fun with matrix multiplication
(some puzzles)
A =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & -2 & 1\\
\end{bmatrix}
B =\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 2 & 1\\
\end{bmatrix}
AB = BA = I
Given a square matrix A, the matrix B is called the inverse of A iff
AB = BA = I
B = A^{-1}
(switch to geogebra for the geometric view)
denoted as
Fun with matrix multiplication
(some puzzles)
Notes: the 3d example from the previous slide a 2d example 1 1 0 1 1 -1 0 1
Fun with matrix multiplication
(some puzzles)
P =\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{bmatrix}
P^{-1} = ?
P^{-1}\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{bmatrix}
=\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 1\\
\end{bmatrix}
(switch rows 1 and 2)
\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{bmatrix}
=\begin{bmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 1\\
\end{bmatrix}
\begin{bmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{bmatrix}
(switch rows 1 and 2)
(switch rows 1 and 2 again)
P^{-1} = P^\top
If P is a permutation matrix then
(switch to geogebra for the geometric view in 3D)
(Notes: switched co-ods)
Fun with matrix multiplication
(some puzzles)
B =\begin{bmatrix}
1 & 0 & 0 & 0\\
1 & 1 & 0 & 0\\
2 & 3 & 1 & 0\\
4 & 5 & 6 & 1\\
\end{bmatrix}
What would the entries above the diagonal be?
AB =
\begin{bmatrix}
~ & 0 & 0 & 0\\
~ & ~ & 0 & 0\\
~ & ~ & ~ & 0\\
~ & ~ & ~ & ~\\
\end{bmatrix}
(lower triangular matrix)
(lower triangular matrix)
A =\begin{bmatrix}
1 & 0 & 0 & 0\\
1 & 1 & 0 & 0\\
2 & 3 & 1 & 0\\
4 & 5 & 6 & 1\\
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
A =\begin{bmatrix}
0 & -1\\
1 & 0
\end{bmatrix}
\mathbf{x} = \begin{bmatrix}
3\\
2
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
0 & -1\\
1 & 0
\end{bmatrix}
\begin{bmatrix}
3\\
2
\end{bmatrix}
= \begin{bmatrix}
-2\\
3
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
\frac{\sqrt{3}}{2} & -\frac{1}{2}\\
~\\
\frac{1}{2} & \frac{\sqrt{3}}{2}
\end{bmatrix}
= \begin{bmatrix}
\frac{\sqrt{3} - 1}{2}\\
\\
\frac{\sqrt{3} + 1}{2}
\end{bmatrix}
\begin{bmatrix}
1\\
1
\end{bmatrix}
Fun with matrix multiplication
(some puzzles)
A =\begin{bmatrix}
cos\theta & -sin\theta\\
sin\theta & cos\theta
\end{bmatrix}
A\mathbf{x} =\begin{bmatrix}
\frac{\sqrt{3}}{2} & -\frac{1}{2}\\
~\\
\frac{1}{2} & \frac{\sqrt{3}}{2}
\end{bmatrix}
= \begin{bmatrix}
\frac{\sqrt{3} - 1}{2}\\
\\
\frac{\sqrt{3} + 1}{2}
\end{bmatrix}
\begin{bmatrix}
1\\
1
\end{bmatrix}
(rotation matrix)
Fun with matrix multiplication
(some puzzles)
A_{4\times4} \begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
=\begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
Find~(one~such)~A?
\begin{bmatrix}
0.25&0.25&0.25&0.25\\
0.2&0.3&0.2&0.3\\
0.1&0.2&0.3&0.4\\
0.2&0.1&0.2&0.5\\
\end{bmatrix}
\begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
=\begin{bmatrix}
1\\
1\\
1\\
1\\
\end{bmatrix}
Trivia: That is a Markov matrix. One of its eigen values is 1.
The corresponding eigen vector is a vector of all 1's
Ax = \lambda x~~~~(\lambda = 1)
Enough Fun: Some Serious Qs now
Properties of matrix multiplication
Associative
Distributive
(Not)Commutative
(AB)C = A(BC)
A(B+C) = AB + AC
AB \neq BA
Proof: homework 1
Proof: homework 1
Can you think of a special case when the two sides will be equal?
Why is this a linear transformation?
A\mathbf{x} = \mathbf{b}
matrix
vector
vector
A(c\mathbf{x} + d\mathbf{y})
(distributive property)
= A(c\mathbf{x}) + A(d\mathbf{y})
= cA\mathbf{x} + dA\mathbf{y}
\underbrace{A}_{\mathbb{R}^{m \times n}}\underbrace{\mathbf{x}}_{\mathbb{R}^n} = \underbrace{\mathbf{y}}_{\mathbb{R}^m}
A: \mathbb{R}^n \rightarrow \mathbb{R}^m
T: \mathbb{R}^n \rightarrow \mathbb{R}^m
T(c\mathbf{x} + d\mathbf{y}) = cT(\mathbf{x}) + dT (\mathbf{y})
\equiv
System of linear equations
(and their connection to matrix multiplication)
x_1 + x_2 = 3 \\
x_1 - x_2 = 1
\begin{bmatrix}
x_1\\
x_1
\end{bmatrix}
\begin{bmatrix}
x_2\\
-x_2
\end{bmatrix}
+
=\begin{bmatrix}
3\\
1
\end{bmatrix}
x_1\begin{bmatrix}
1\\
1
\end{bmatrix}
+
x_2\begin{bmatrix}
1\\
-1
\end{bmatrix}
=\begin{bmatrix}
3\\
1
\end{bmatrix}
\begin{bmatrix}
x_1\\
x_2
\end{bmatrix}
\begin{bmatrix}
1&1\\
1&-1
\end{bmatrix}
=\begin{bmatrix}
3\\
1
\end{bmatrix}
A
\mathbf{x}
\mathbf{b}
A\mathbf{x} = \mathbf{b}
System of linear equations
x_1 + x_2 + x_3 - 2x_4= 1
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
x_4
\end{bmatrix}
\begin{bmatrix}
1&1&1&-2\\
2&0&2&-1\\
3&2&-1&0\\
2&-1&1&-1\\
\end{bmatrix}
=\begin{bmatrix}
1\\
2\\
3\\
2\\
\end{bmatrix}
A
\mathbf{x}
\mathbf{b}
2x_1 + 2x_3 - x_4 = 2
3x_1 + 2x_2 - x_3 = 3
2x_1 -x_2 + x_3 - x_4 = 2
Can you guess x?
Hint: Ax is the linear combination of the columns of A