CS6015: Linear Algebra and Random Processes

Lecture 2: Fun with matrix multiplication, System of linear equations

Learning Objectives

How do you multiply two matrices? (4 ways)

What is the transpose of a matrix?

What is the inverse of a matrix?

What are some of the laws of matrix multiplication?

What is a symmetric matrix?

What are some interesting matrices which lead to special products?

How do you represent a system of linear equations using matrices?

(for today's lecture)

Recap

(with a puzzle)

A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
Find~one~such~\mathbf{x}
A\mathbf{x} =\begin{bmatrix} 4\\ 3\\ 3\\ \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} 0\\ 1\\ 2\\ \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} 5\\ 6\\ 5\\ \end{bmatrix}
\mathbf{x} =\begin{bmatrix} 0\\ 1\\ 1\\ 0\\ \end{bmatrix}
\mathbf{x} =\begin{bmatrix} 0\\ 0\\ 0\\ 1\\ \end{bmatrix}
\mathbf{x} =\begin{bmatrix} 1\\ 1\\ 1\\ 1\\ \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} 2\\ 1\\ 2\\ \end{bmatrix}
\mathbf{x} =\begin{bmatrix} 0\\ 1\\ 0\\ 0\\ \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} 0\\ -1\\ 1\\ \end{bmatrix}
\mathbf{x} =\begin{bmatrix} 0\\ 1\\ -1\\ 0\\ \end{bmatrix}

Multiplying two matrices

A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}

Method 1: Multiply the rows of A by columns of B (1 row, 1 column at a time)

= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
-5
0
-1
4
5
-4
\begin{bmatrix} 2\\ 1\\ 2\\ 1 \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 1\\ -2\\ -1\\ 2\\ \end{bmatrix}
\begin{bmatrix} 0\\ 2\\ 1\\ 2 \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 1\\ -2\\ -1\\ 2\\ \end{bmatrix}
\begin{bmatrix} 2\\ -1\\ 2\\ -2\\ \end{bmatrix}
\begin{bmatrix} 1\\ 2\\ 2\\ 0\\ \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 2\\ 1\\ 2\\ 1 \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 2\\ -1\\ 2\\ -2\\ \end{bmatrix}
\begin{bmatrix} 0\\ 2\\ 1\\ 2 \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 2\\ -1\\ 2\\ -2\\ \end{bmatrix}
\begin{bmatrix} 1\\ -2\\ -1\\ 2\\ \end{bmatrix}
\begin{bmatrix} 1\\ 2\\ 2\\ 0\\ \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}

Multiplying two matrices

A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}

Method 1: Multiply the rows of A by columns of B (1 row, 1 column at a time)

= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
-5
0
-1
4
5
-4
c_{12}
= A_{11}B_{12} + A_{12}B_{22} + A_{13}B_{32} + A_{14}B_{42}
=\sum_{k=1}^n A_{1k}B_{k2}
c_{ij}
= A_{i1}B_{1j} + A_{i2}B_{2j} + A_{i3}B_{3j} + A_{i4}B_{4j}
c_{32}
=\sum_{k=1}^n A_{3k}B_{k2}
=\sum_{k=1}^n A_{ik}B_{kj}
\begin{bmatrix} 2\\ -1\\ 2\\ -2\\ \end{bmatrix}
\begin{bmatrix} 1\\ 2\\ 2\\ 0\\ \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 0\\ 2\\ 1\\ 2 \end{bmatrix}
\begin{matrix} *\\ *\\ *\\ \end{matrix}
\begin{matrix} ~\\ +\\ +\\ \end{matrix}
\begin{bmatrix} 2\\ -1\\ 2\\ -2\\ \end{bmatrix}
= A_{31}B_{12} + A_{32}B_{22} + A_{33}B_{32} + A_{34}B_{42}

Multiplying two matrices

Method2: Take linear combinations of columns of A (1 column of B at a time)

= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
\begin{bmatrix} 1\\ 2\\ 0\\ \end{bmatrix}
+
1
\begin{bmatrix} 2\\ 1\\ 2\\ \end{bmatrix}
+
-2
\begin{bmatrix} 2\\ 2\\ 1\\ \end{bmatrix}
+
-1
\begin{bmatrix} 0\\ 1\\ 2\\ \end{bmatrix}
2
\begin{bmatrix} 1\\ 2\\ 0\\ \end{bmatrix}
+
2
\begin{bmatrix} 2\\ 1\\ 2\\ \end{bmatrix}
+
-1
\begin{bmatrix} 2\\ 2\\ 1\\ \end{bmatrix}
+
2
\begin{bmatrix} 0\\ 1\\ 2\\ \end{bmatrix}
-2

Multiplying two matrices

Method3: Take linear combinations of rows of B (1 row of A at a time)

= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
\begin{bmatrix} ~~~1 & ~~~2\\ \end{bmatrix}
+
1
+
2
+
0
\begin{bmatrix} -2 & -1\\ \end{bmatrix}
\begin{bmatrix} -1 & ~~~2\\ \end{bmatrix}
\begin{bmatrix} ~~~2 & -2\\ \end{bmatrix}
2
\begin{bmatrix} ~~~1 & ~~~2\\ \end{bmatrix}
+
2
+
1
+
1
\begin{bmatrix} -2 & -1\\ \end{bmatrix}
\begin{bmatrix} -1 & ~~~2\\ \end{bmatrix}
\begin{bmatrix} ~~~2 & -2\\ \end{bmatrix}
2
\begin{bmatrix} ~~~1 & ~~~2\\ \end{bmatrix}
+
0
+
2
+
2
\begin{bmatrix} -2 & -1\\ \end{bmatrix}
\begin{bmatrix} -1 & ~~~2\\ \end{bmatrix}
\begin{bmatrix} ~~~2 & -2\\ \end{bmatrix}
1

Multiplying two matrices

Method4: One column of A and one row of B at a time

-5
0
-1
4
5
-4
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
\begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 0 & 0 \end{bmatrix}
\begin{bmatrix} -4 & -2 \\ -2 & -1 \\ -4 & -2 \end{bmatrix}
\begin{bmatrix} -2 & 4 \\ -2 & 4 \\ -1 & 2 \end{bmatrix}
\begin{bmatrix} 0& 0 \\ 2 & -2 \\ 4 & -4 \end{bmatrix}
+
+
+
\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}
\begin{bmatrix} ~~~1 & ~~~2\\ \end{bmatrix}
\mathbb{R}^{3 \times 1}
\mathbb{R}^{1 \times 2}
\begin{bmatrix} 2 \\ 1 \\ 2 \end{bmatrix}
\begin{bmatrix} -2 & -1\\ \end{bmatrix}
\begin{bmatrix} 2 \\ 2 \\ 1 \end{bmatrix}
\begin{bmatrix} -1 & ~~~2\\ \end{bmatrix}
= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
\begin{bmatrix} 0 \\ 1 \\ 2 \end{bmatrix}
\begin{bmatrix} 2 & -2\\ \end{bmatrix}

Multiplying two matrices

Method4: One column of A and one row of B at a time

= \begin{bmatrix} ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ ~~~~ & ~~~~\\ \end{bmatrix}
-5
0
-1
4
5
-4
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
c_{ij}
=\sum_{k=1}^n A_{ik}B_{kj}
\begin{bmatrix} 1 & 2 \\ 2 & 4 \\ 0 & 1 \end{bmatrix}
\begin{bmatrix} -4 & -2 \\ -2 & -1 \\ -4 & -2 \end{bmatrix}
\begin{bmatrix} -2 & 4 \\ -2 & 4 \\ -1 & 2 \end{bmatrix}
\begin{bmatrix} 0& 0 \\ 2 & -2 \\ 4 & -4 \end{bmatrix}
+
+
+
c_{11}
= A_{11}B_{11} + A_{12}B_{21} + A_{13}B_{31} + A_{14}B_{41}
=\sum_{k=1}^n A_{1k}B_{k1}
A_{11}B_{11}
A_{12}B_{21}
A_{13}B_{31}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top
A_{14}B_{41}
\begin{bmatrix} 1 \\ 2 \\ 0 \end{bmatrix}
\begin{bmatrix} ~~~1 & ~~~2\\ \end{bmatrix}

Multiplying two matrices

rows multiplied by columns

A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix} 
(produces one cell at a time)
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}

linear combination of A's columns

(produces one column at a time)
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top

linear combination of B's rows

(produces one row at a time)
A =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
\mathbf{a}_1
\mathbf{a}_2
\mathbf{a}_3
\mathbf{a}_4
B =\begin{bmatrix} 1 & 2 \\ -2 & -1 \\ -1 & 2 \\ 2 & -2 \\ \end{bmatrix}
\mathbf{b}_1^\top
\mathbf{b}_2^\top
\mathbf{b}_3^\top
\mathbf{b}_4^\top

columns multiplied by rows

(produces one matrix at a time)
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

I =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0& 0 & 1\\ \end{bmatrix}
IB =
\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix} = B

identity matrix

(diagonal entries 1
off-diagonal entries 0)
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

P =\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0& 0 & 1\\ \end{bmatrix}
PB =
\begin{bmatrix} 2 & 1 & 2 & 1\\ 1 & 2 & 2 & 0\\ 0& 2 & 1 &2\\ \end{bmatrix}

permutation matrix

(identity matrix with rows exchanged)
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
0
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
1
*
\begin{bmatrix} 0&2 & 1 & 2 \end{bmatrix}
+
0
*
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
1
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
0
*
\begin{bmatrix} 0&2 & 1 & 2 \end{bmatrix}
+
0
*
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

P =\begin{bmatrix} 0 & 1 & 0 & 0\\ 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0& 0 & 1 & 0 \end{bmatrix}
BP =

permutation matrix

How many n x n permutation matrices are possible?

(Hint: what's the matrix called?)
n!
\begin{bmatrix} 0 & 1 & 0 & 0 \end{bmatrix}
1
*
+
\begin{bmatrix} 1 & 0 & 0 & 0 \end{bmatrix}
2
*
\begin{bmatrix} 0&0 & 0 & 1 \end{bmatrix}
+
2
*
\begin{bmatrix} 0&0 & 1 & 0 \end{bmatrix}
+
0
*
\begin{bmatrix} 2 & 1 & 0 & 2\\ 1 & 2 & 1 & 2\\ 2& 0 & 2 &1\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

P_2 =\begin{bmatrix} 0 & 1 & 0 & 0\\ 1 & 0 & 0 & 0\\ 0 & 0 & 0 & 1\\ 0& 0 & 1 & 0 \end{bmatrix}

permutation matrix

P_1 =\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0& 0 & 1\\ \end{bmatrix}

Pre-multiplying a matrix with a permutation matrix exchanges its rows

Post-multiplying a matrix with a permutation matrix exchanges its cols.

permutation matrix

B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

E =\begin{bmatrix} 1 & 0 & 0\\ -1 & 1 & 0\\ 0& 0 & 1\\ \end{bmatrix}
EB =
\begin{bmatrix} 1 & 2 & 2 & 0\\ 1 & -1 & 0 & 1\\ 0 & 2 & 1 & 2\\ \end{bmatrix}
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
1
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
0
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
0
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
1
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
0
*
-1
*
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

EB =
\begin{bmatrix} 1 & 2 & 2 & 0\\ 1 & -1 & 0 & 1\\ 2 & 3 & 3 & 3\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
E =\begin{bmatrix} 1 & 0 & 0\\ -1 & 1 & 0\\ 0& 1 & 1\\ \end{bmatrix}
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
1
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
0
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
0
*
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
0
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
1
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
1
*

Fun with matrix multiplication

(some puzzles)

EB =
\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ -4 & 0 & -3 & 0\\ \end{bmatrix} 
(subtracting 2 times row 2 from row 3)
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
E =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0& -2 & 1\\ \end{bmatrix}
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
1
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
0
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
0
*
\begin{bmatrix} 1 & 2 & 2 & 0 \end{bmatrix}
0
*
\begin{bmatrix} 0&2 & 1& 2 \end{bmatrix}
+
1
*
+
\begin{bmatrix} 2 & 1 & 2 & 1 \end{bmatrix}
-2
*
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

C =\begin{bmatrix} 1 & 2 & 0\\ 2 & 1 & 2\\ 2 & 2 & 1\\ 0 & 1 & 2 \end{bmatrix}
CB =
\begin{bmatrix} ~~~ & ~~~ & ~~~ & ~~~\\ ~~~ & ~~~ & ~~~ & ~~~\\ ~~~ & ~~~ & ~~~ & ~~~\\ ~~~ & ~~~ & ~~~ & ~~~ \end{bmatrix}
2
5
4
5
6
8
9
4
4
9
8
5
5
4
6
2

square symmetric matrix

(why did we get this? was there something special about C and B?)
C_{ij} = B_{ji}
C = B^\top
B = C^\top

transposes

Ok, but why was the product a square symmetric matrix?
S = S^T

square symmetric matrix

B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

C =\begin{bmatrix} 1 & 2 & 0\\ 2 & 1 & 2\\ 2 & 2 & 1\\ 0 & 1 & 2 \end{bmatrix}
(AB)^\top = B^\top A^\top

Result:

Proof:

(Homework 1 :-) )

Hint:

Is the          th entry of            , the same as the           th entry of 

(AB)^\top
(i,j)
(i,j)
B^\top A^\top
C = B^\top
(CB)^\top = B^\top C^\top = CB
Hence, if C is the transpose of B,CB will always be a square symmetric matrix
B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

C =\begin{bmatrix} 1 & 2 & 0\\ 2 & 1 & 2\\ 2 & 2 & 1\\ 0 & 1 & 2 \end{bmatrix}
C = B^\top
B^\top = C
(CB)^\top = B^\top C^\top = CB
Hence, if C is the transpose of B,CB will always be a square symmetric matrix 
(we will now use this result)
(AB)^\top = B^\top A^\top

Result:

is always a square symmetric matrix

A^\top A

Fun with matrix multiplication

(some puzzles)

B =\begin{bmatrix} 1 & 2 & 2 & 0\\ 2 & 1 & 2 & 1\\ 0& 2 & 1 &2\\ \end{bmatrix}
E_1 =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & -2 & 1\\ \end{bmatrix} 
(subtracting 2 times row 2 from row 3)
What~is~E_2?
E_2E_1B = B
E_2 =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 2 & 1\\ \end{bmatrix} 
(add 2 times row 2 to row 3 - the inverse of what E1 did)
E_2E_1 =
\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 2 & 1\\ \end{bmatrix}
\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & -2 & 1\\ \end{bmatrix}
= \begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} = I

Fun with matrix multiplication

(some puzzles)

A =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & -2 & 1\\ \end{bmatrix}
B =\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 2 & 1\\ \end{bmatrix}
AB = BA = I

Given a  square matrix A, the matrix B is called the inverse of A iff

AB = BA = I
B = A^{-1}

(switch to geogebra for the geometric view)

denoted as

Fun with matrix multiplication

(some puzzles)

Notes:

the 3d example from the previous slide

a 2d example

1 1 0 1

1 -1 0 1

 

Fun with matrix multiplication

(some puzzles)

P =\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{bmatrix}
P^{-1} = ?
P^{-1}\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{bmatrix}
=\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix} 
(switch rows 1 and 2)
\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{bmatrix}
=\begin{bmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{bmatrix}
\begin{bmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{bmatrix} 
(switch rows 1 and 2)
(switch rows 1 and 2 again)
P^{-1} = P^\top

If P is a permutation matrix then 

(switch to geogebra for the geometric view in 3D)

(Notes: switched co-ods)

Fun with matrix multiplication

(some puzzles)

B =\begin{bmatrix} 1 & 0 & 0 & 0\\ 1 & 1 & 0 & 0\\ 2 & 3 & 1 & 0\\ 4 & 5 & 6 & 1\\ \end{bmatrix}

What would the entries above the diagonal be?

AB =
\begin{bmatrix} ~ & 0 & 0 & 0\\ ~ & ~ & 0 & 0\\ ~ & ~ & ~ & 0\\ ~ & ~ & ~ & ~\\ \end{bmatrix} 
(lower triangular matrix)
(lower triangular matrix)
A =\begin{bmatrix} 1 & 0 & 0 & 0\\ 1 & 1 & 0 & 0\\ 2 & 3 & 1 & 0\\ 4 & 5 & 6 & 1\\ \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

A =\begin{bmatrix} 0 & -1\\ 1 & 0 \end{bmatrix}
\mathbf{x} = \begin{bmatrix} 3\\ 2 \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} 0 & -1\\ 1 & 0 \end{bmatrix}
\begin{bmatrix} 3\\ 2 \end{bmatrix}
= \begin{bmatrix} -2\\ 3 \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2}\\ ~\\ \frac{1}{2} & \frac{\sqrt{3}}{2} \end{bmatrix}
= \begin{bmatrix} \frac{\sqrt{3} - 1}{2}\\ \\ \frac{\sqrt{3} + 1}{2} \end{bmatrix}
\begin{bmatrix} 1\\ 1 \end{bmatrix}

Fun with matrix multiplication

(some puzzles)

A =\begin{bmatrix} cos\theta & -sin\theta\\ sin\theta & cos\theta \end{bmatrix}
A\mathbf{x} =\begin{bmatrix} \frac{\sqrt{3}}{2} & -\frac{1}{2}\\ ~\\ \frac{1}{2} & \frac{\sqrt{3}}{2} \end{bmatrix}
= \begin{bmatrix} \frac{\sqrt{3} - 1}{2}\\ \\ \frac{\sqrt{3} + 1}{2} \end{bmatrix}
\begin{bmatrix} 1\\ 1 \end{bmatrix} 
(rotation matrix)

Fun with matrix multiplication

(some puzzles)

A_{4\times4} \begin{bmatrix} 1\\ 1\\ 1\\ 1\\ \end{bmatrix}
=\begin{bmatrix} 1\\ 1\\ 1\\ 1\\ \end{bmatrix}
Find~(one~such)~A?
\begin{bmatrix} 0.25&0.25&0.25&0.25\\ 0.2&0.3&0.2&0.3\\ 0.1&0.2&0.3&0.4\\ 0.2&0.1&0.2&0.5\\ \end{bmatrix}
\begin{bmatrix} 1\\ 1\\ 1\\ 1\\ \end{bmatrix}
=\begin{bmatrix} 1\\ 1\\ 1\\ 1\\ \end{bmatrix} 
Trivia: 

That is a Markov matrix.

One of its eigen values is 1.

The corresponding eigen vector
is a vector of all 1's
Ax = \lambda x~~~~(\lambda = 1)

Enough Fun: Some Serious Qs now

Properties of matrix multiplication

Associative
Distributive
(Not)Commutative
(AB)C = A(BC)
A(B+C) = AB + AC
AB \neq BA

Proof: homework 1

Proof: homework 1

Can you think of a special case when the two sides will be equal?

Why is this a linear transformation?

A\mathbf{x} = \mathbf{b}
matrix
vector
vector
A(c\mathbf{x} + d\mathbf{y}) 
(distributive property)
= A(c\mathbf{x}) + A(d\mathbf{y})
= cA\mathbf{x} + dA\mathbf{y}
\underbrace{A}_{\mathbb{R}^{m \times n}}\underbrace{\mathbf{x}}_{\mathbb{R}^n} = \underbrace{\mathbf{y}}_{\mathbb{R}^m}
A: \mathbb{R}^n \rightarrow \mathbb{R}^m
T: \mathbb{R}^n \rightarrow \mathbb{R}^m
T(c\mathbf{x} + d\mathbf{y}) = cT(\mathbf{x}) + dT (\mathbf{y})
\equiv

System of linear equations

(and their connection to matrix multiplication)
x_1 + x_2 = 3 \\ x_1 - x_2 = 1
\begin{bmatrix} x_1\\ x_1 \end{bmatrix}
\begin{bmatrix} x_2\\ -x_2 \end{bmatrix}
+
=\begin{bmatrix} 3\\ 1 \end{bmatrix}
x_1\begin{bmatrix} 1\\ 1 \end{bmatrix}
+
x_2\begin{bmatrix} 1\\ -1 \end{bmatrix}
=\begin{bmatrix} 3\\ 1 \end{bmatrix}
\begin{bmatrix} x_1\\ x_2 \end{bmatrix}
\begin{bmatrix} 1&1\\ 1&-1 \end{bmatrix}
=\begin{bmatrix} 3\\ 1 \end{bmatrix}
A
\mathbf{x}
\mathbf{b}
A\mathbf{x} = \mathbf{b}

System of linear equations

x_1 + x_2 + x_3 - 2x_4= 1
\begin{bmatrix} x_1\\ x_2\\ x_3\\ x_4 \end{bmatrix}
\begin{bmatrix} 1&1&1&-2\\ 2&0&2&-1\\ 3&2&-1&0\\ 2&-1&1&-1\\ \end{bmatrix}
=\begin{bmatrix} 1\\ 2\\ 3\\ 2\\ \end{bmatrix}
A
\mathbf{x}
\mathbf{b}
2x_1 + 2x_3 - x_4 = 2
3x_1 + 2x_2 - x_3 = 3
2x_1 -x_2 + x_3 - x_4 = 2

Can you guess x?

Hint: Ax is the linear combination of the columns of A

Learning Objectives

How do you multiply two matrices? (4 ways)

What is the transpose of a matrix?

What is the inverse of a matrix?

What are some of the laws of matrix multiplication?

What is a symmetric matrix?

What are some interesting matrices which lead to special products?

How do you represent a system of linear equations using matrices?

(achieved)

CS6015: Lecture 2

By Mitesh Khapra

Made with Slides.com

CS6015: Lecture 2

Fun with matrix multiplication, System of linear equations

  • 3,897

More from Mitesh Khapra