COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

8.3 - Matrix Multiplication Method

All-Pairs Shortest Path

8.3 - Matrix Multiplication Method

  • We are now moving to all-pairs shortest path problem:
    • find the shortest paths from every vertex in the graph to every other vertex in the graph (with no negative-weight cycles)
    • as before, we have \(G = \langle V,E \rangle\) with no negative-weight cycle
  • We are going to discuss two different methods which are similar:
    • Matrix Multiplication Method
    • Floyd-Warshall Algorithm
  • One thing you should keep in mind is that, if you perform Single-Source Shortest Path algorithm twice, then you will checking the same edges in both runs
    • so can we somehow store this information, i.e. dynamic programming?

8.3 - Matrix Multiplication Method

Optimal Substructure

ShortestPath(\(u\),\(v\))

\(w(u,y_1)\) + ShortestPath(\(y_1\),\(v\))

\(w(u,v)\) 

\(\dots\)

\(w(u,y_2)\) +ShortestPath(\(y_2\),\(v\))

\(w(u,y_\ell)\) +ShortestPath(\(y_\ell\),\(v\))

  • We have shown before that the shortest path problem exhibits an optimal substructure
    • we are going to modify this slightly

8.3 - Matrix Multiplication Method

Optimal Substructure

  • Let \(p_{ij}\) be the shortest path from vertex \(x_i\) to vertex \(x_j\)
  • Suppose that \(p_{ij}\) contains at most \(m\) edges 
    • we can decompose \(p_{ij}\) into \(p_{ik}\) and the edge \((k,j)\) where \(p_{ik}\) is the shortest path from \(i\) to \(k\) (since the shortest path problem exhibits optimal substructure)
    • \(p_{ik}\) contains at most \((m-1)\) edges

\(x_i\)

\(x_j\)

\(\cdots\)

\(x_k\)

\(p_{ik}\)

\((k,j)\)

8.3 - Matrix Multiplication Method

Optimal Substructure

ShortestPath(\(u\),\(v\)) with at most \(m\) edges

\(w(u,y_1)\) + ShortestPath(\(y_1\),\(v\)) with at most \((m-1)\) edges

\(w(u,v)\) 

\(\dots\)

\(w(u,y_2)\) +ShortestPath(\(y_2\),\(v\)) with at most \((m-1)\) edges

\(w(u,y_\ell)\) +ShortestPath(\(y_\ell\),\(v\)) with at most \((m-1)\) edges

8.3 - Matrix Multiplication Method

Quick Aside: Dynamic Programming

  • The four steps:
    1. Show that there is an optimal substructure
    2. Show the recursive relation that gives optimal solution
    3. Find the VALUE of the optimal solution
    4. (optional) Find the COMBINATION that gives the optimal solution

8.3 - Matrix Multiplication Method

Recursive Relation

  • Let \(l_{ij}^{(m)}\) be the weight of the shortest path from vertex \(i\) to vertex \(j\) with at most \(m\) edges
  • We have:

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty  & \text{otherwise.}\end{cases} \]

  • i.e. when \(m = 0\), \(l_{ij}^{(0)}\) is the weight of the shortest path between vertex \(i\) and vertex \(j\) with no edges

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ ?  & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty  & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ?  & \text{otherwise.}\end{cases} \]

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ w(i,j)  & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty  & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ?  & \text{otherwise.}\end{cases} \]

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ?  & \text{otherwise.}\end{cases} \]

i

j

\[ l_{ij}^{(2)} =\min_{1 \le k \le n} \left(l_{ik}^{(1)} + w(k,j)\right) \]

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ w(i,j)  & \text{otherwise.}\end{cases} \]

  note: \(w(j,j) = 0\)

8.3 - Matrix Multiplication Method

Recursive Relation

  • More generally, for \(m \ge 1\), the recursive solution is given by

\[\ell^{(m)}_{ij} = \min_{1 \le k \le n} \left( l_{ik}^{(m-1)} + w(k,j)\right)\]

  • remember, these are the weights of the shortest path using up to \(m\) edges, not exactly \(m\) edges
    • so it could be the case that \(\ell^{(3)}_{ij} = \ell^{(2)}_{ij} = \ell^{(1)}_{ij} = w(i,j) \)
  • if there are \(n\) vertices, then the shortest path should contain at most \(n-1\) edges
    • we assume that there are no negative cycles, so the shortest path must visit every vertex at most once

8.3 - Matrix Multiplication Method

Algorithm

  • Approach: We compute a series of matrices \(L^{(1)}\), \(L^{(2)}\), \(\dots\), \(L^{(n-1)}\), containing \(l_{ij}^{(1)}\), \(l_{ij}^{(2)}\), \(\dots\), \(l_{ij}^{(n-1)}\) for all pairs of vertices \((i,j)\)
    • ​i.e. this series of matrices record the weights of the shortest paths between \(i\) and \(j\) as we consider more and more edges
    • we can stop at \(L^{(n-1)}\) because the shortest path should visit each vertex at most once
       
  • ​\(L^{(1)}\) is the adjancency matrix of the graph \(G\) (we use only one edge)
  • \(L^{(2)}\) is the weights of the shortest paths between \(i\) and \(j\) if we use up to two edges
  • \(L^{(3)}\) is the weights of the shortest paths between \(i\) and \(j\) if we use up to three edges

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

  • start with \(L_{12}^{(2)} = l_{12}^{(2)}\)
  • \[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
    • ​\(l_{11}^{(1)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(1)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty\)
    • ​\(l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty\)

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & \infty & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{13}^{(2)} = l_{13}^{(2)}\)
  • \[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
    • ​\(l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty\)
    • ​\(l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty\)
    • ​\(l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty\)
    • ​\(l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty\)
    • ​\(l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

  • compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

8.3 - Matrix Multiplication Method

Example

5

2

1

4

3

8

3

9

-5

-4

1

7

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & \infty & -3 & -4 \\ \infty & 0 & 4 & 4 & 3 \\ \infty & 7 & 0 & 16 & 10 \\ \infty & 2 & -5 & 0 & \infty \\ \infty & \infty & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Complexity

\[\left(\begin{matrix}0 & 8 & -8 & -3 & -4\end{matrix}\right)\left(\begin{matrix}8 \\ 0 \\ 7 \\ \infty \\ \infty\end{matrix}\right)\]

  • Let \(W\) be the adjacency matrix
  • How do you compute \(L^{(3)} \times W\) (matrix multiplication)?

\[ = (0\times 8) + (8 \times 0) + (-8 \times 7) + (-3 * \infty) + (-4 * \infty)\]

\[W = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Complexity

  • how do you compute \(L_{12}^{(4)} = l_{12}^{(4)}\)
  • \[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
    • ​\(l_{11}^{(3)} + w(1,2) = 0 + 8 = 8\)
    • ​\(l_{12}^{(3)} + w(2,2) = 8 + 0 = 8\)
    • ​\(l_{13}^{(3)} + w(3,2) = -8 + 7 = -1\)
    • ​\(l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty\)
    • ​\(l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty\)

\[ = (0\times 8) + (8 \times 0) + (-8 \times 7) + (-3 * \infty) + (-4 * \infty)\]

\[W = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

do you see the similarity?

8.3 - Matrix Multiplication Method

Complexity

  • The complexity of multiplying two \((n \times n)\) matrices is \(O(n^3)\), so the cost of computing \(L^{(i+1)}\) from \(L^{(i)}\) and \(W\) is \(O(V^3)\)
    • but we have to do this \(V\) times, so total cost is \(O(V^4)\)
    • how does this compare to doing Bellman-Ford or Dijkstra repeatedly?
  • However, we can improve this running time:
    • if you have to compute \(x^n\), are you going to do \(n\) multiplications?
    • no, you can apply repeated squarings
      • \(x^{19}\):
        • \(x^2 \rightarrow x^4 \rightarrow x^8 \rightarrow x^{16}\)
        • \(x^{16} \times x^2 \times x\)

8.3 - Matrix Multiplication Method

Complexity

  • Because the operation we're doing is associative (as is matrix multiplication), we can compute
    • \(L^{(1)} = W\)
    • \(L^{(2)} = L^{(1)} \times W = W \times W\)
    • \(L^{(4)} = L^{(3)} \times W  = (L^{(2)} \times W) \times W= W^2 \times W^2\)
    • \(L^{(8)} = W^8 = W^4 \times W^4\)
  • For example:
    • \(L^{(19)}\):
      • \(L^{(1)} \rightarrow L^{(2)} \rightarrow L^{(4)} \rightarrow L^{(8)} \rightarrow L^{(16)}\)
      • \(L^{(16)} \times W \times W^2\)

8.3 - Matrix Multiplication Method

Complexity

  • Therefore the complexity of this algorithm is actually \(O(V^3 \log V)\)
  • You can push this further using Strassen's algorithm to \(O(V^{2.807}\log V)\) or use other fast multiplication trick

 

  • Finally, don't forget that this is a dynamic programming method, so we're actually brute forcing this (we did try every single path, right?)

8.3 - Matrix Multiplication Method

Implementation

// Let L = L^(i) and W be the adjacency matrix
// Compute Lp = L^(i+1)

public static int[][] extend(int[][] L, int [][] W) {
  int INF = Integer.MAX_VALUE
  int n = L.length;
  int[][] Lp = new int[n][n];
		
  for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
    
      // set Lp[i][j] to infinity at start
      Lp[i][j] = INF;
      for (int k = 0; k < n; k++) {
        // if either W[k][j] or L[i][k] is infinity,
        // then Lp[i][j] should be infinity as well
        if (W[k][j] != INF && L[i][k] != INF) 
          Lp[i][j] = Math.min(Lp[i][j], L[i][k]+W[k][j]);
      }
    
    }
  }		
  return Lp;
}