COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

8.3 - Matrix Multiplication Method

All-Pairs Shortest Path

8.3 - Matrix Multiplication Method

We are now moving to all-pairs shortest path problem:
- find the shortest paths from every vertex in the graph to every other vertex in the graph (with no negative-weight cycles)
- as before, we have $G = \langle V,E \rangle$ with no negative-weight cycle
We are going to discuss two different methods which are similar:
- Matrix Multiplication Method
- Floyd-Warshall Algorithm
One thing you should keep in mind is that, if you perform Single-Source Shortest Path algorithm twice, then you will checking the same edges in both runs
- so can we somehow store this information, i.e. dynamic programming?

8.3 - Matrix Multiplication Method

Optimal Substructure

ShortestPath($u$,$v$)

$w(u,y_1)$ + ShortestPath($y_1$,$v$)

$w(u,v)$

$\dots$

$w(u,y_2)$ +ShortestPath($y_2$,$v$)

$w(u,y_\ell)$ +ShortestPath($y_\ell$,$v$)

We have shown before that the shortest path problem exhibits an optimal substructure
- we are going to modify this slightly

8.3 - Matrix Multiplication Method

Optimal Substructure

Let $p_{ij}$ be the shortest path from vertex $x_i$ to vertex $x_j$
Suppose that $p_{ij}$ contains at most $m$ edges
- we can decompose $p_{ij}$ into $p_{ik}$ and the edge $(k,j)$ where $p_{ik}$ is the shortest path from $i$ to $k$ (since the shortest path problem exhibits optimal substructure)
- $p_{ik}$ contains at most $(m-1)$ edges

$x_i$

$x_j$

$\cdots$

$x_k$

$p_{ik}$

$(k,j)$

8.3 - Matrix Multiplication Method

Optimal Substructure

ShortestPath($u$,$v$) with at most $m$ edges

$w(u,y_1)$ + ShortestPath($y_1$,$v$) with at most $(m-1)$ edges

$w(u,v)$

$\dots$

$w(u,y_2)$ +ShortestPath($y_2$,$v$) with at most $(m-1)$ edges

$w(u,y_\ell)$ +ShortestPath($y_\ell$,$v$) with at most $(m-1)$ edges

8.3 - Matrix Multiplication Method

Quick Aside: Dynamic Programming

The four steps:
1. Show that there is an optimal substructure
2. Show the recursive relation that gives optimal solution
3. Find the VALUE of the optimal solution
4. (optional) Find the COMBINATION that gives the optimal solution

8.3 - Matrix Multiplication Method

Recursive Relation

Let $l_{ij}^{(m)}$ be the weight of the shortest path from vertex $i$ to vertex $j$ with at most $m$ edges
We have:

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty & \text{otherwise.}\end{cases} \]

i.e. when $m = 0$, $l_{ij}^{(0)}$ is the weight of the shortest path between vertex $i$ and vertex $j$ with no edges

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ ? & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ? & \text{otherwise.}\end{cases} \]

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ w(i,j) & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(0)} = \begin{cases}0 & \text{if $i = j$},\\ \infty & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ? & \text{otherwise.}\end{cases} \]

8.3 - Matrix Multiplication Method

Recursive Relation

\[ l_{ij}^{(2)} = \begin{cases}0 & \text{if $i = j$},\\ ? & \text{otherwise.}\end{cases} \]

\[ l_{ij}^{(2)} =\min_{1 \le k \le n} \left(l_{ik}^{(1)} + w(k,j)\right) \]

\[ l_{ij}^{(1)} = \begin{cases}0 & \text{if $i = j$},\\ w(i,j) & \text{otherwise.}\end{cases} \]

note: $w(j,j) = 0$

8.3 - Matrix Multiplication Method

Recursive Relation

More generally, for $m \ge 1$, the recursive solution is given by

\[\ell^{(m)}_{ij} = \min_{1 \le k \le n} \left( l_{ik}^{(m-1)} + w(k,j)\right)\]

remember, these are the weights of the shortest path using up to $m$ edges, not exactly $m$ edges
- so it could be the case that $\ell^{(3)}_{ij} = \ell^{(2)}_{ij} = \ell^{(1)}_{ij} = w(i,j) $
if there are $n$ vertices, then the shortest path should contain at most $n-1$ edges
- we assume that there are no negative cycles, so the shortest path must visit every vertex at most once

8.3 - Matrix Multiplication Method

Algorithm

Approach: We compute a series of matrices $L^{(1)}$, $L^{(2)}$, $\dots$, $L^{(n-1)}$, containing $l_{ij}^{(1)}$, $l_{ij}^{(2)}$, $\dots$, $l_{ij}^{(n-1)}$ for all pairs of vertices $(i,j)$
- i.e. this series of matrices record the weights of the shortest paths between $i$ and $j$ as we consider more and more edges
- we can stop at $L^{(n-1)}$ because the shortest path should visit each vertex at most once
$L^{(1)}$ is the adjancency matrix of the graph $G$ (we use only one edge)
$L^{(2)}$ is the weights of the shortest paths between $i$ and $j$ if we use up to two edges
$L^{(3)}$ is the weights of the shortest paths between $i$ and $j$ if we use up to three edges

8.3 - Matrix Multiplication Method

Example

\[L^{(1)} = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

\[L^{(2)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

8.3 - Matrix Multiplication Method

Example

start with $L_{12}^{(2)} = l_{12}^{(2)}$
\[l_{12}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,2)\right)\]
- $l_{11}^{(1)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(1)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(1)} + w(3,2) = \infty + 7 = \infty$
- $l_{14}^{(1)} + w(4,2) = \infty + \infty = \infty$
- $l_{15}^{(1)} + w(5,2) = -4 + \infty = \infty$

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & \infty & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{13}^{(2)} = l_{13}^{(2)}$
\[l_{13}^{(2)} = \min_{1 \le k \le n} \left( l_{1k}^{(1)} + w(k,3)\right)\]
- $l_{11}^{(1)} + w(1,3) = 0 + \infty = \infty$
- $l_{12}^{(1)} + w(2,3) = 8 + \infty = \infty$
- $l_{13}^{(1)} + w(3,3) = \infty + 0 = \infty$
- $l_{14}^{(1)} + w(4,3) = \infty + -5 = \infty$
- $l_{15}^{(1)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & ? & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & ? & ? & ? \\ ? & 0 & ? & ? & ? \\ ? & ? & 0 & ? & ? \\ ? & ? & ? & 0 & ? \\ ? & ? & ? & ? & 0\end{matrix}\right)\]

compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

8.3 - Matrix Multiplication Method

Example

-5

-4

\[L^{(2)} = \left(\begin{matrix} 0 & 8 & \infty & -3 & -4 \\ \infty & 0 & 4 & 4 & 3 \\ \infty & 7 & 0 & 16 & 10 \\ \infty & 2 & -5 & 0 & \infty \\ \infty & \infty & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Complexity

\[\left(\begin{matrix}0 & 8 & -8 & -3 & -4\end{matrix}\right)\left(\begin{matrix}8 \\ 0 \\ 7 \\ \infty \\ \infty\end{matrix}\right)\]

Let $W$ be the adjacency matrix
How do you compute $L^{(3)} \times W$ (matrix multiplication)?

\[ = (0\times 8) + (8 \times 0) + (-8 \times 7) + (-3 * \infty) + (-4 * \infty)\]

\[W = \left(\begin{matrix} 0 & 8 & \infty & \infty & -4 \\ \infty & 0 & \infty & 9 & 3 \\ \infty & 7 & 0 & \infty & \infty \\ \infty & \infty & -5 & 0 & \infty \\ \infty & \infty & \infty & 1 & 0\end{matrix}\right)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

8.3 - Matrix Multiplication Method

Complexity

how do you compute $L_{12}^{(4)} = l_{12}^{(4)}$
\[l_{12}^{(4)} = \min_{1 \le k \le n} \left( l_{1k}^{(3)} + w(k,2)\right)\]
- $l_{11}^{(3)} + w(1,2) = 0 + 8 = 8$
- $l_{12}^{(3)} + w(2,2) = 8 + 0 = 8$
- $l_{13}^{(3)} + w(3,2) = -8 + 7 = -1$
- $l_{14}^{(3)} + w(4,2) = -3 + \infty = \infty$
- $l_{15}^{(3)} + w(5,3) = -4 + \infty = \infty$

\[ = (0\times 8) + (8 \times 0) + (-8 \times 7) + (-3 * \infty) + (-4 * \infty)\]

\[L^{(3)} = \left(\begin{matrix} 0 & 8 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

\[L^{(4)} = \left(\begin{matrix} 0 & -1 & -8 & -3 & -4 \\ \infty & 0 & -1 & 4 & 3 \\ \infty & 7 & 0 & 11 & 10 \\ \infty & 2 & -5 & 0 & 5 \\ \infty & 3 & -4 & 1 & 0\end{matrix}\right)\]

do you see the similarity?

8.3 - Matrix Multiplication Method

Complexity

The complexity of multiplying two $(n \times n)$ matrices is $O(n^3)$, so the cost of computing $L^{(i+1)}$ from $L^{(i)}$ and $W$ is $O(V^3)$
- but we have to do this $V$ times, so total cost is $O(V^4)$
- how does this compare to doing Bellman-Ford or Dijkstra repeatedly?
However, we can improve this running time:
- if you have to compute $x^n$, are you going to do $n$ multiplications?
- no, you can apply repeated squarings
  - $x^{19}$:
    - $x^2 \rightarrow x^4 \rightarrow x^8 \rightarrow x^{16}$
    - $x^{16} \times x^2 \times x$

8.3 - Matrix Multiplication Method

Complexity

Because the operation we're doing is associative (as is matrix multiplication), we can compute
- $L^{(1)} = W$
- $L^{(2)} = L^{(1)} \times W = W \times W$
- $L^{(4)} = L^{(3)} \times W = (L^{(2)} \times W) \times W= W^2 \times W^2$
- $L^{(8)} = W^8 = W^4 \times W^4$
For example:
- $L^{(19)}$:
  - $L^{(1)} \rightarrow L^{(2)} \rightarrow L^{(4)} \rightarrow L^{(8)} \rightarrow L^{(16)}$
  - $L^{(16)} \times W \times W^2$

8.3 - Matrix Multiplication Method

Complexity

Therefore the complexity of this algorithm is actually $O(V^3 \log V)$
You can push this further using Strassen's algorithm to $O(V^{2.807}\log V)$ or use other fast multiplication trick

Finally, don't forget that this is a dynamic programming method, so we're actually brute forcing this (we did try every single path, right?)

8.3 - Matrix Multiplication Method

Implementation

// Let L = L^(i) and W be the adjacency matrix
// Compute Lp = L^(i+1)

public static int[][] extend(int[][] L, int [][] W) {
  int INF = Integer.MAX_VALUE
  int n = L.length;
  int[][] Lp = new int[n][n];
		
  for (int i = 0; i < n; i++) {
    for (int j = 0; j < n; j++) {
    
      // set Lp[i][j] to infinity at start
      Lp[i][j] = INF;
      for (int k = 0; k < n; k++) {
        // if either W[k][j] or L[i][k] is infinity,
        // then Lp[i][j] should be infinity as well
        if (W[k][j] != INF && L[i][k] != INF) 
          Lp[i][j] = Math.min(Lp[i][j], L[i][k]+W[k][j]);
      }
    
    }
  }		
  return Lp;
}

COMP3010 - 8.3 - Matrix Multiplication Method

By Daniel Sutantyo

COMP3010 - 8.3 - Matrix Multiplication Method

COMP3010: Algorithm Theory and Design

8.3 - Matrix Multiplication Method

All-Pairs Shortest Path

Optimal Substructure

Optimal Substructure

Optimal Substructure

Quick Aside: Dynamic Programming

Recursive Relation

Recursive Relation

Recursive Relation

Recursive Relation

Recursive Relation

Algorithm

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Example

Complexity

Complexity

Complexity

Complexity

Complexity

Implementation

COMP3010 - 8.3 - Matrix Multiplication Method

More from Daniel Sutantyo