CS6015: Linear Algebra and Random Processes

Lecture 15:  Formula for determinant, co-factors, Finding \(A^{-1}\), Cramer's rule for solving \(Ax=b\), Determinant=Volume

Learning Objectives

(for today's lecture)

What is the formula for computing a determinant?

What are co-factors?

What is Cramer's rule for computing  \(x\) = \(A^{-1}\)b ?

What is the connection between determinant and volume?

A formula for determinant 

(the 2 x 2 case)

\begin{vmatrix} a & b\\ c & d \end{vmatrix}\\
= \begin{vmatrix} a + 0 & 0 + b\\ c & d \end{vmatrix}\\
= \begin{vmatrix} a & 0\\ c & d \end{vmatrix}+ \begin{vmatrix} 0 & b\\ c & d \end{vmatrix}
= \begin{vmatrix} a & 0\\ c+0 & 0+d \end{vmatrix}+ \begin{vmatrix} 0 & b\\ c+0 & 0+d \end{vmatrix}
= \begin{vmatrix} a & 0\\ c & 0 \end{vmatrix}+ \begin{vmatrix} a & 0\\ 0 & d \end{vmatrix}
+ \begin{vmatrix} 0 & b\\ c & 0 \end{vmatrix}+ \begin{vmatrix} 0 & b\\ 0 & d \end{vmatrix}
0
0
(by prop. 6)
(by prop. 6)
\begin{vmatrix} a & 0\\ 0 & d \end{vmatrix}+ \begin{vmatrix} 0 & b\\ c & 0 \end{vmatrix}
=\begin{vmatrix} a & 0\\ 0 & d \end{vmatrix}+ (-1)\begin{vmatrix} b & 0\\ 0 & c \end{vmatrix}
=a\cdot d\begin{vmatrix} 1 & 0\\ 0 & 1 \end{vmatrix}- b\cdot c\begin{vmatrix} 1 & 0\\ 0 & 1 \end{vmatrix}
=1
=1
=a\cdot d- b\cdot c
(by prop. 2)
(by prop. 1)
(by prop. 1)

A formula for determinant 

(the 3 x 3 case)

\begin{vmatrix} a & b\\ c & d \end{vmatrix}\\
=\begin{vmatrix} a & 0\\ 0 & d \end{vmatrix}+ \begin{vmatrix} 0 & b\\ c & 0 \end{vmatrix}

Only one entry per row, per column 0's everywhere else

\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
=\begin{vmatrix} a_{11} & 0 & 0\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}+ \begin{vmatrix} 0 & a_{12} & 0\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}+ \begin{vmatrix} 0 & 0 & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}+\dots 27~terms
=ad\begin{vmatrix} 1 & 0\\ 0 & 1 \end{vmatrix}+ bc\begin{vmatrix} 0 & 1\\ 1 & 0 \end{vmatrix}

All permutations of the 2x2 identity matrix

=\begin{vmatrix} a_{11} & 0 & 0\\ 0 & a_{22} & 0\\ 0 & 0 & a_{33}\\ \end{vmatrix}+ \begin{vmatrix} a_{11} & 0 & 0\\ 0 & 0 & a_{23}\\ 0 & a_{32} & 0\\ \end{vmatrix}+ \begin{vmatrix} 0 & a_{12} & 0\\ a_{21} & 0 & 0\\ 0 & 0 & a_{33}\\ \end{vmatrix}+ \begin{vmatrix} 0 & a_{12} & 0\\ 0 & 0 & a_{23}\\ a_{31} & 0 & 0\\ \end{vmatrix}+ \begin{vmatrix} 0 & 0 & a_{13}\\ 0 & a_{22} & 0\\ a_{31} & 0 & 0\\ \end{vmatrix}+ \begin{vmatrix} 0 & 0 & a_{13}\\ a_{21} & 0 & 0\\ 0 & a_{32} & 0\\ \end{vmatrix}
(21 of these would have a column of 0s)

A formula for determinant 

(the 3 x 3 case)

\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
=a_{11}a_{22}a_{33}\begin{vmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{vmatrix} + a_{11}a_{23}a_{32}\begin{vmatrix} 1 & 0 & 0\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{vmatrix} + a_{12}a_{23}a_{31}\begin{vmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0\\ \end{vmatrix}
+ a_{12}a_{21}a_{33}\begin{vmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{vmatrix}+ a_{13}a_{21}a_{32}\begin{vmatrix} 0 & 0 & 1\\ 1 & 0 & 0\\ 0 & 1 & 0\\ \end{vmatrix}+ a_{13}a_{22}a_{31}\begin{vmatrix} 0 & 0 & 1\\ 0 & 1 & 0\\ 1 & 0 & 0\\ \end{vmatrix}

All permutations of the 3x3 identity matrix

+1 or -1 depending on number of row exchanges (\(k\)): \((-1)^k\)

+1
-1
+1
-1
+1
-1

A formula for determinant 

(the 3 x 3 case)

\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
=a_{11}a_{22}a_{33}\begin{vmatrix} 1 & 0 & 0\\ 0 & 1 & 0\\ 0 & 0 & 1\\ \end{vmatrix} + a_{11}a_{23}a_{32}\begin{vmatrix} 1 & 0 & 0\\ 0 & 0 & 1\\ 0 & 1 & 0\\ \end{vmatrix} + a_{12}a_{23}a_{31}\begin{vmatrix} 0 & 1 & 0\\ 0 & 0 & 1\\ 1 & 0 & 0\\ \end{vmatrix}
+ a_{12}a_{21}a_{33}\begin{vmatrix} 0 & 1 & 0\\ 1 & 0 & 0\\ 0 & 0 & 1\\ \end{vmatrix}+ a_{13}a_{21}a_{32}\begin{vmatrix} 0 & 0 & 1\\ 1 & 0 & 0\\ 0 & 1 & 0\\ \end{vmatrix}+ a_{13}a_{22}a_{31}\begin{vmatrix} 0 & 0 & 1\\ 0 & 1 & 0\\ 1 & 0 & 0\\ \end{vmatrix}
+1
-1
+1
-1
+1
-1
P \in 3!~Permutations
=\sum
det(P)
a_{1\_\_}a_{2\_\_}a_{3\_\_}
\alpha
\beta
\gamma
\{\alpha, \beta, \gamma\} = some~permutation~of~\{1,2,3\}

A formula for determinant 

(the n x n case)

\begin{vmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{vmatrix}\\
P \in n!~Permutations
=\sum
det(P)
a_{1\alpha}a_{2\beta}a_{3\gamma}\dots a_{n\omega}
\{\alpha, \beta, \gamma\, \dots, \omega\} = some~permutation~of~\{1,2,3, \dots, n\}

Co-factors

\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
=a_{11}a_{22}a_{33}(+1) + a_{11}a_{23}a_{32}(-1) + a_{12}a_{23}a_{31}(+1)
+a_{12}a_{21}a_{33}(-1) + a_{13}a_{21}a_{32}(+1)+ a_{13}a_{22}a_{31}(-1)
=a_{11}(a_{22}a_{33} - a_{23}a_{32})
+~a_{12}(a_{23}a_{31} - a_{21}a_{33})
+~a_{13}(a_{21}a_{32} - a_{22}a_{31})
\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
det~of~a~n-1\times n-1~matrix
det~of~a~n-1\times n-1~matrix
det~of~a~n-1\times n-1~matrix
+
-
+
\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\
\begin{vmatrix} a_{11} & a_{12} & a_{13}\\ a_{21} & a_{22} & a_{23}\\ a_{31} & a_{32} & a_{33}\\ \end{vmatrix}\\

Co-factors

Co-factors

\begin{vmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{vmatrix}\\
=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
C_{ij} = (-1)^{i+j} (det~of~a~n-1\times n-1~matrix)

(obtained after dropping i-th row and j-th column)

=a_{21}C_{21}+a_{22}C_{22}+\cdots+a_{2n}C_{2n}

(in the big formula you can put the brackets wherever you want)

a_{21}(- a_{12}a_{33} + a_{13}a_{32} )
C_{21}

Co-factors

\begin{vmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{vmatrix}\\
=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
=a_{21}C_{21}+a_{22}C_{22}+\cdots+a_{2n}C_{2n}
a_{21}(- a_{12}a_{33} + a_{13}a_{32} )
C_{21}
=a_{31}C_{31}+a_{32}C_{32}+\cdots+a_{3n}C_{3n}
=a_{n1}C_{n1}+a_{n2}C_{n2}+\cdots+a_{nn}C_{nn}
\dots

Formula for \( A^{-1}\)

(2 x 2 case)

(and the n x n case)

A = \begin{bmatrix} a & b\\ c & d \end{bmatrix}\\
A^{-1} = \frac{1}{det(A)}\begin{bmatrix} d & -b\\ -c & a \end{bmatrix}\\
C = \begin{bmatrix} d & -c\\ -b & a \end{bmatrix}\\

(matrix of co-factors)

C^\top
A=\begin{bmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\
A^{-1} = \frac{1}{det(A)}C^\top

Why is this formula correct?

To prove: 

A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
\begin{bmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\
\begin{bmatrix} C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\ C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\ C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\ \end{bmatrix}\\

(1,1) entry = 

a_{11}C_{11} + a_{12}C_{12} + a_{13}C_{13} + \cdots + a_{1n}C_{1n}

(2,2) entry = 

=det(A)
a_{21}C_{21} + a_{22}C_{22} + a_{23}C_{23} + \cdots + a_{2n}C_{2n}
=det(A)

(i,i) entry = 

a_{i1}C_{i1} + a_{i2}C_{i2} + a_{i3}C_{i3} + \cdots + a_{in}C_{in}
=det(A)

All the diagonal entries will be det(A) 

(exactly as we wanted)

Why is this formula correct?

To prove: 

A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
\begin{bmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\
\begin{bmatrix} C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\ C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\ C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\ \end{bmatrix}\\

(2,1) entry = 

a_{21}C_{11} + a_{22}C_{12} + a_{23}C_{13} + \cdots + a_{2n}C_{1n}
=0

What about the off-diagonal entries?

Why?

(not very obvious but we will see on the next slide)

Why is this formula correct?

To prove: 

A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
A=\begin{bmatrix} a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\
B=\begin{bmatrix} a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\ a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\

The last n-1 rows of A and B are equal

Hence, the co-factors of the first row for the two matrices will be equal 

det(B) = 0
(two equal rows)

a similar argument can be made for all off-diagonal entries of \( AC^\top\)

\begin{matrix} C_{11} & C_{12} & C_{13} & \cdots & C_{1n} \end{matrix}
\begin{matrix} C_{11} & C_{12} & C_{13} & \cdots & C_{1n} \end{matrix}

Cramer's rule

(solving \(A\mathbf{x} = \mathbf{b}\) )

\mathbf{x} = A^{-1}\mathbf{b}
\mathbf{x} = \frac{1}{det(A)}C^{\top}\mathbf{b}
\begin{bmatrix} C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\ C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\ C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\ \end{bmatrix}\\
\begin{bmatrix} b_1\\ b_2\\ b_3\\ \cdots\\ b_n \end{bmatrix}
\begin{bmatrix} x_1\\ x_2\\ x_3\\ \cdots\\ x_n \end{bmatrix}=\frac{1}{det(A)}
x_1 = \frac{1}{det(A)}(b_1C_{11} + b_2C_{21} + b_3C_{31} + \cdots + b_nC_{n1})
this looks like a determinant of some matrix
det(A)=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
What is that matrix?
\begin{bmatrix} b_{1} & a_{12} & a_{13}&\cdots&a_{1n}\\ b_{2} & a_{22} & a_{23}&\cdots&a_{2n}\\ b_{3} & a_{32} & a_{33}&\cdots&a_{3n}\\ \cdots & \cdots & \cdots&\cdots&\cdots\\ b_{n} & a_{n2} & a_{n3}&\cdots&a_{nn}\\ \end{bmatrix}\\
B_1
x_1 = \frac{det(B_1)}{det(A)}
The first column of A replaced by B
(transpose)

Cramer's rule

(solving \(A\mathbf{x} = \mathbf{b}\) )

\mathbf{x} = A^{-1}\mathbf{b}
\mathbf{x} = \frac{1}{det(A)}C^{\top}\mathbf{b}
x_i = \frac{det(B_i)}{det(A)}

\( B_i \) is the matrix obtained by replacing the \(i\)-th column of \(A\) by \(\mathbf{b}\)

Practically, not very useful as computing the determinant is expensive (n! terms)

(GE is cheaper and hence preferred)

Determinant = Volume

The determinant is equal to the volume of the parallelepiped formed by the columns of the matrix 

Learning Objectives

(achieved)

What is the formula for computing a determinant?

What are co-factors?

What is Cramer's rule for computing the inverse?

What is the connection between determinant and volume?