CS6015: Linear Algebra and Random Processes
Lecture 15: Formula for determinant, co-factors, Finding \(A^{-1}\), Cramer's rule for solving \(Ax=b\), Determinant=Volume
Learning Objectives
(for today's lecture)
What is the formula for computing a determinant?
What are co-factors?
What is Cramer's rule for computing \(x\) = \(A^{-1}\)b ?
What is the connection between determinant and volume?
A formula for determinant
(the 2 x 2 case)
\begin{vmatrix}
a & b\\
c & d
\end{vmatrix}\\
= \begin{vmatrix}
a + 0 & 0 + b\\
c & d
\end{vmatrix}\\
= \begin{vmatrix}
a & 0\\
c & d
\end{vmatrix}+
\begin{vmatrix}
0 & b\\
c & d
\end{vmatrix}
= \begin{vmatrix}
a & 0\\
c+0 & 0+d
\end{vmatrix}+
\begin{vmatrix}
0 & b\\
c+0 & 0+d
\end{vmatrix}
= \begin{vmatrix}
a & 0\\
c & 0
\end{vmatrix}+
\begin{vmatrix}
a & 0\\
0 & d
\end{vmatrix}
+ \begin{vmatrix}
0 & b\\
c & 0
\end{vmatrix}+
\begin{vmatrix}
0 & b\\
0 & d
\end{vmatrix}
0
0
(by prop. 6)
(by prop. 6)
\begin{vmatrix}
a & 0\\
0 & d
\end{vmatrix}+
\begin{vmatrix}
0 & b\\
c & 0
\end{vmatrix}
=\begin{vmatrix}
a & 0\\
0 & d
\end{vmatrix}+
(-1)\begin{vmatrix}
b & 0\\
0 & c
\end{vmatrix}
=a\cdot d\begin{vmatrix}
1 & 0\\
0 & 1
\end{vmatrix}-
b\cdot c\begin{vmatrix}
1 & 0\\
0 & 1
\end{vmatrix}
=1
=1
=a\cdot d-
b\cdot c
(by prop. 2)
(by prop. 1)
(by prop. 1)
A formula for determinant
(the 3 x 3 case)
\begin{vmatrix}
a & b\\
c & d
\end{vmatrix}\\
=\begin{vmatrix}
a & 0\\
0 & d
\end{vmatrix}+
\begin{vmatrix}
0 & b\\
c & 0
\end{vmatrix}
Only one entry per row, per column 0's everywhere else
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
=\begin{vmatrix}
a_{11} & 0 & 0\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}+
\begin{vmatrix}
0 & a_{12} & 0\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}+
\begin{vmatrix}
0 & 0 & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}+\dots 27~terms
=ad\begin{vmatrix}
1 & 0\\
0 & 1
\end{vmatrix}+
bc\begin{vmatrix}
0 & 1\\
1 & 0
\end{vmatrix}
All permutations of the 2x2 identity matrix
=\begin{vmatrix}
a_{11} & 0 & 0\\
0 & a_{22} & 0\\
0 & 0 & a_{33}\\
\end{vmatrix}+
\begin{vmatrix}
a_{11} & 0 & 0\\
0 & 0 & a_{23}\\
0 & a_{32} & 0\\
\end{vmatrix}+
\begin{vmatrix}
0 & a_{12} & 0\\
a_{21} & 0 & 0\\
0 & 0 & a_{33}\\
\end{vmatrix}+
\begin{vmatrix}
0 & a_{12} & 0\\
0 & 0 & a_{23}\\
a_{31} & 0 & 0\\
\end{vmatrix}+
\begin{vmatrix}
0 & 0 & a_{13}\\
0 & a_{22} & 0\\
a_{31} & 0 & 0\\
\end{vmatrix}+
\begin{vmatrix}
0 & 0 & a_{13}\\
a_{21} & 0 & 0\\
0 & a_{32} & 0\\
\end{vmatrix}
(21 of these would have a column of 0s)
A formula for determinant
(the 3 x 3 case)
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
=a_{11}a_{22}a_{33}\begin{vmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 1\\
\end{vmatrix} +
a_{11}a_{23}a_{32}\begin{vmatrix}
1 & 0 & 0\\
0 & 0 & 1\\
0 & 1 & 0\\
\end{vmatrix} +
a_{12}a_{23}a_{31}\begin{vmatrix}
0 & 1 & 0\\
0 & 0 & 1\\
1 & 0 & 0\\
\end{vmatrix}
+
a_{12}a_{21}a_{33}\begin{vmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{vmatrix}+
a_{13}a_{21}a_{32}\begin{vmatrix}
0 & 0 & 1\\
1 & 0 & 0\\
0 & 1 & 0\\
\end{vmatrix}+
a_{13}a_{22}a_{31}\begin{vmatrix}
0 & 0 & 1\\
0 & 1 & 0\\
1 & 0 & 0\\
\end{vmatrix}
All permutations of the 3x3 identity matrix
+1 or -1 depending on number of row exchanges (\(k\)): \((-1)^k\)
+1
-1
+1
-1
+1
-1
A formula for determinant
(the 3 x 3 case)
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
=a_{11}a_{22}a_{33}\begin{vmatrix}
1 & 0 & 0\\
0 & 1 & 0\\
0 & 0 & 1\\
\end{vmatrix} +
a_{11}a_{23}a_{32}\begin{vmatrix}
1 & 0 & 0\\
0 & 0 & 1\\
0 & 1 & 0\\
\end{vmatrix} +
a_{12}a_{23}a_{31}\begin{vmatrix}
0 & 1 & 0\\
0 & 0 & 1\\
1 & 0 & 0\\
\end{vmatrix}
+
a_{12}a_{21}a_{33}\begin{vmatrix}
0 & 1 & 0\\
1 & 0 & 0\\
0 & 0 & 1\\
\end{vmatrix}+
a_{13}a_{21}a_{32}\begin{vmatrix}
0 & 0 & 1\\
1 & 0 & 0\\
0 & 1 & 0\\
\end{vmatrix}+
a_{13}a_{22}a_{31}\begin{vmatrix}
0 & 0 & 1\\
0 & 1 & 0\\
1 & 0 & 0\\
\end{vmatrix}
+1
-1
+1
-1
+1
-1
P \in 3!~Permutations
=\sum
det(P)
a_{1\_\_}a_{2\_\_}a_{3\_\_}
\alpha
\beta
\gamma
\{\alpha, \beta, \gamma\} = some~permutation~of~\{1,2,3\}
A formula for determinant
(the n x n case)
\begin{vmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{vmatrix}\\
P \in n!~Permutations
=\sum
det(P)
a_{1\alpha}a_{2\beta}a_{3\gamma}\dots a_{n\omega}
\{\alpha, \beta, \gamma\, \dots, \omega\} = some~permutation~of~\{1,2,3, \dots, n\}
Co-factors
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
=a_{11}a_{22}a_{33}(+1) +
a_{11}a_{23}a_{32}(-1) +
a_{12}a_{23}a_{31}(+1)
+a_{12}a_{21}a_{33}(-1) +
a_{13}a_{21}a_{32}(+1)+
a_{13}a_{22}a_{31}(-1)
=a_{11}(a_{22}a_{33} - a_{23}a_{32})
+~a_{12}(a_{23}a_{31} - a_{21}a_{33})
+~a_{13}(a_{21}a_{32} - a_{22}a_{31})
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
det~of~a~n-1\times n-1~matrix
det~of~a~n-1\times n-1~matrix
det~of~a~n-1\times n-1~matrix
+
-
+
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
\begin{vmatrix}
a_{11} & a_{12} & a_{13}\\
a_{21} & a_{22} & a_{23}\\
a_{31} & a_{32} & a_{33}\\
\end{vmatrix}\\
Co-factors
Co-factors
\begin{vmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{vmatrix}\\
=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
C_{ij} = (-1)^{i+j} (det~of~a~n-1\times n-1~matrix)
(obtained after dropping i-th row and j-th column)
=a_{21}C_{21}+a_{22}C_{22}+\cdots+a_{2n}C_{2n}
(in the big formula you can put the brackets wherever you want)
a_{21}(- a_{12}a_{33} + a_{13}a_{32} )
C_{21}
Co-factors
\begin{vmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{vmatrix}\\
=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
=a_{21}C_{21}+a_{22}C_{22}+\cdots+a_{2n}C_{2n}
a_{21}(- a_{12}a_{33} + a_{13}a_{32} )
C_{21}
=a_{31}C_{31}+a_{32}C_{32}+\cdots+a_{3n}C_{3n}
=a_{n1}C_{n1}+a_{n2}C_{n2}+\cdots+a_{nn}C_{nn}
\dots
Formula for \( A^{-1}\)
(2 x 2 case)
(and the n x n case)
A = \begin{bmatrix}
a & b\\
c & d
\end{bmatrix}\\
A^{-1} = \frac{1}{det(A)}\begin{bmatrix}
d & -b\\
-c & a
\end{bmatrix}\\
C = \begin{bmatrix}
d & -c\\
-b & a
\end{bmatrix}\\
(matrix of co-factors)
C^\top
A=\begin{bmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
A^{-1} = \frac{1}{det(A)}C^\top
Why is this formula correct?
To prove:
A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
\begin{bmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
\begin{bmatrix}
C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\
C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\
C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\
\end{bmatrix}\\
(1,1) entry =
a_{11}C_{11} + a_{12}C_{12} + a_{13}C_{13} + \cdots + a_{1n}C_{1n}
(2,2) entry =
=det(A)
a_{21}C_{21} + a_{22}C_{22} + a_{23}C_{23} + \cdots + a_{2n}C_{2n}
=det(A)
(i,i) entry =
a_{i1}C_{i1} + a_{i2}C_{i2} + a_{i3}C_{i3} + \cdots + a_{in}C_{in}
=det(A)
All the diagonal entries will be det(A)
(exactly as we wanted)
Why is this formula correct?
To prove:
A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
\begin{bmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
\begin{bmatrix}
C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\
C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\
C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\
\end{bmatrix}\\
(2,1) entry =
a_{21}C_{11} + a_{22}C_{12} + a_{23}C_{13} + \cdots + a_{2n}C_{1n}
=0
What about the off-diagonal entries?
Why?
(not very obvious but we will see on the next slide)
Why is this formula correct?
To prove:
A^{-1} = \frac{1}{det(A)}C^\top
A (\frac{1}{det(A)}C^\top) = I
i.e.,~AC^\top = det(A)I
A=\begin{bmatrix}
a_{11} & a_{12} & a_{13}&\cdots&a_{1n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
B=\begin{bmatrix}
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{21} & a_{22} & a_{23}&\cdots&a_{2n}\\
a_{31} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
a_{n1} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
The last n-1 rows of A and B are equal
Hence, the co-factors of the first row for the two matrices will be equal
det(B) = 0
(two equal rows)
a similar argument can be made for all off-diagonal entries of \( AC^\top\)
\begin{matrix}
C_{11} & C_{12} & C_{13} & \cdots & C_{1n}
\end{matrix}
\begin{matrix}
C_{11} & C_{12} & C_{13} & \cdots & C_{1n}
\end{matrix}
Cramer's rule
(solving \(A\mathbf{x} = \mathbf{b}\) )
\mathbf{x} = A^{-1}\mathbf{b}
\mathbf{x} = \frac{1}{det(A)}C^{\top}\mathbf{b}
\begin{bmatrix}
C_{11} & C_{21} & C_{31}&\cdots&C_{n1}\\
C_{12} & C_{22} & C_{32}&\cdots&C_{n2}\\
C_{13} & C_{23} & C_{33}&\cdots&C_{n3}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
C_{1n} & C_{2n} & C_{3n}&\cdots&C_{nn}\\
\end{bmatrix}\\
\begin{bmatrix}
b_1\\
b_2\\
b_3\\
\cdots\\
b_n
\end{bmatrix}
\begin{bmatrix}
x_1\\
x_2\\
x_3\\
\cdots\\
x_n
\end{bmatrix}=\frac{1}{det(A)}
x_1 = \frac{1}{det(A)}(b_1C_{11} + b_2C_{21} + b_3C_{31} + \cdots + b_nC_{n1})
this looks like a determinant of some matrix
det(A)=a_{11}C_{11}+a_{12}C_{12}+\cdots+a_{1n}C_{1n}
What is that matrix?
\begin{bmatrix}
b_{1} & a_{12} & a_{13}&\cdots&a_{1n}\\
b_{2} & a_{22} & a_{23}&\cdots&a_{2n}\\
b_{3} & a_{32} & a_{33}&\cdots&a_{3n}\\
\cdots & \cdots & \cdots&\cdots&\cdots\\
b_{n} & a_{n2} & a_{n3}&\cdots&a_{nn}\\
\end{bmatrix}\\
B_1
x_1 = \frac{det(B_1)}{det(A)}
The first column of A replaced by B
(transpose)
Cramer's rule
(solving \(A\mathbf{x} = \mathbf{b}\) )
\mathbf{x} = A^{-1}\mathbf{b}
\mathbf{x} = \frac{1}{det(A)}C^{\top}\mathbf{b}
x_i = \frac{det(B_i)}{det(A)}
\( B_i \) is the matrix obtained by replacing the \(i\)-th column of \(A\) by \(\mathbf{b}\)
Practically, not very useful as computing the determinant is expensive (n! terms)
(GE is cheaper and hence preferred)