COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
8.2 - Bellman-Ford Algorithm
Prelude
8.2 - Bellman-Ford Algorithm
- Bellman-Ford is a single-source shortest path algorithm that can handle graphs with negative edge weights
- It is generally slower than Dijkstra, \(O(VE)\) vs \(O(V\log V + E)\) (or just \(O(V^2)\))
- As before, we define our graph to be \(G\langle V,E \rangle\), and set \(s\) to be the starting vertex, i.e. we are going to find the shortest path from \(s\) to every other vertex in \(V\)
- we are going to assume that the graph does not have negative-weight cycle
- Definition: for vertex \(v \in V\), define \(\delta(v)\) to be the weight of the shortest path from \(s\) to \(v\) (we have \(\delta(s) = 0\))
Algorithm
8.2 - Bellman-Ford Algorithm
- set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
-
for \(i = 1\) to \(|V|-1\):
- for each edge \((u,v) \in E\), relax \((u,v\))
- for each edge \((u,v)\in E\)
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return false
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return true
(the algorithm is actually very simple, relax every edge \(|V|-1\) times
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(\infty\)
\(\infty\)
\(\infty\)
\(\infty\)
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | inf | inf | inf | inf |
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(\infty\)
\(\infty\)
\(\infty\)
\(-4\)
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | inf | inf | inf | -4 |
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(8\)
\(\infty\)
\(\infty\)
\(-4\)
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | inf | inf | -4 |
\(\delta(b) = 8\)
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(8\)
\(\infty\)
\(\infty\)
\(-4\)
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | inf | inf | -4 |
\(\delta(b) = 8\)
no change
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(8\)
\(17\)
\(\infty\)
\(-4\)
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | inf | 17 | -4 |
\(\delta(b) = 8\)
no change
\(\delta(d) = 17\)
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
\(\delta(b) = 8\)
no change
\(\delta(d) = 17\)
no change
\(8\)
\(17\)
\(\infty\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | inf | 17 | -4 |
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
\(\delta(b) = 8\)
no change
\(\delta(d) = 17\)
no change
\(\delta(c) = 12\)
\(8\)
\(17\)
\(12\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | 12 | 17 | -4 |
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
\(\delta(b) = 8\)
no change
\(\delta(d) = 17\)
no change
\(\delta(c) = 12\)
\(\delta(d) = -3\)
\(8\)
\(-3\)
\(12\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | 12 | -3 | -4 |
Are we done?
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(\delta(e) = -4\)
\(\delta(b) = 8\)
no change
\(\delta(d) = 17\)
no change
\(\delta(c) = 12\)
\(\delta(d) = -3\)
\(8\)
\(-3\)
\(12\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | 12 | -3 | -4 |
Are we done?
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(8\)
\(-3\)
\(12\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | 12 | -3 | -4 |
The weight of the shortest path to from \(s\) to \(c\) is -8, however, this requires us to relax \((s,e)\), \((e,d)\), and then \((d,c)\)
We didn't relax the paths in this order
So what do we do now? Keep going
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(8\)
\(-3\)
\(-8\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | 8 | -8 | -3 | -4 |
no change
no change
\(\delta(c) = -8\)
no change
no change
no change
no change
Example
8.2 - Bellman-Ford Algorithm
e
b
s
d
c
8
3
9
-5
-4
1
7
\(0\)
\((s,e)\)
\((b,e)\)
\((s,b)\)
\((b,d)\)
\((c,b)\)
\((d,c)\)
\((e,d)\)
\(-1\)
\(-3\)
\(-8\)
\(-4\)
| s | b | c | d | e |
|---|---|---|---|---|
| 0 | -1 | -8 | -3 | -4 |
no change
no change
no change
no change
no change
\(\delta(b) = -1\)
no change
Correctness
8.2 - Bellman-Ford Algorithm
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 1, 2, 3?
- Did we try the path using edges 1, 2, 4?
- Did we try the path using edges 4, 3, 1?
- You can see the proof of correctness in CLRS pages 652-654
- Here is an intuition
- let's say that there are 4 vertices, and 4 edges
Correctness
8.2 - Bellman-Ford Algorithm
- You can see the proof of correctness in CLRS pages 652-654
- Here is an intuition
- let's say that there are 4 vertices, and 4 edges
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 1, 2, 3?
- Did we try the path using edges 1, 2, 4?
- Did we try the path using edges 4, 3, 1?
Correctness
8.2 - Bellman-Ford Algorithm
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 1, 2, 3?
- Did we try the path using edges 1, 2, 4?
- Did we try the path using edges 4, 3, 1?
- You can see the proof of correctness in CLRS pages 652-654
- Here is an intuition
- let's say that there are 4 vertices, and 4 edges
Correctness
8.2 - Bellman-Ford Algorithm
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 1, 2, 3?
- Did we try the path using edges 1, 2, 4?
- Did we try the path using edges 4, 3, 1?
- You can see the proof of correctness in CLRS pages 652-654
- Here is an intuition
- let's say that there are 4 vertices, and 4 edges
Correctness
8.2 - Bellman-Ford Algorithm
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 4, 3, 2, 1?
- You can see the proof of correctness in CLRS pages 652-654
- Here is an intuition
- let's say that there are 4 vertices, and 4 edges
Correctness
8.2 - Bellman-Ford Algorithm
- Did we try every possible permutation of edges?
- yes, but wouldn't this cost \(O(E!)\)?
- no, because there are some overlaps as you can see
1
2
3
4
1
2
3
4
1
2
3
4
- Did we try the path using edges 1, 2, 3?
- Did we try the path using edges 1, 2, 4?
- Did we try the path using edges 4, 3, 1?
Algorithm
8.2 - Bellman-Ford Algorithm
- set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
-
for \(i = 1\) to \(|V|-1\):
- for each edge \((u,v) \in E\), relax \((u,v\))
- for each edge \((u,v)\in E\)
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return false
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return true
(the algorithm is actually very simple, relax every edge \(|V|-1\) times
Algorithm
8.2 - Bellman-Ford Algorithm
- set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
-
for \(i = 1\) to \(|V|-1\):
- for each edge \((u,v) \in E\), relax \((u,v\))
- why only do it \(|V|-1\) times?
- because you would have tried all combinations of \(|V|-1\) edges at this point, and the shortest path should not contain more than \(|V|-1\) edges
- you should not visit the same vertex twice, this means there is a cycle
Algorithm
8.2 - Bellman-Ford Algorithm
- for each edge \((u,v)\in E\)
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return false
- if \(\delta(v) > \delta(u) + w(u,v)\)
- return true
- What does this bit do?
- After performing the algorithm \(|V| - 1\) times, just try to relax all the edges one more time.
- If somehow you still get a better path, that means the graph contains a negative weight cycle, and this algorithm does not work in this case
Complexity
8.2 - Bellman-Ford Algorithm
- It is clear that the complexity of the algorithm is \(O(VE)\)
- Contrast this with Dijkstra's \(O(V\log V + E)\)
- \(E = O(V^2)\) (consider a graph with \(n\) vertices, how many edges can you have at most)?
- so \(O(VE)\) can be as bad as \(O(V^3)\)
- You can actually improve Bellman-Ford to \(O(V+E)\) if you have a directed acyclic graph (see CLRS 24.2 if you want more information)
- I am not going to test you on the formal proof of correctness or complexity, but I do need you to understand the basic idea
COMP3010 - 8.2 - Bellman-Ford Algorithm
By Daniel Sutantyo
COMP3010 - 8.2 - Bellman-Ford Algorithm
- 192
