COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

8.2 - Bellman-Ford Algorithm

Prelude

8.2 - Bellman-Ford Algorithm

  • Bellman-Ford is a single-source shortest path algorithm that can handle graphs with negative edge weights
  • It is generally slower than Dijkstra, \(O(VE)\) vs \(O(V\log V + E)\) (or just \(O(V^2)\))

 

  • As before, we define our graph to be \(G\langle V,E \rangle\), and set \(s\) to be the starting vertex, i.e. we are going to find the shortest path from \(s\) to every other vertex in \(V\)
    • we are going to assume that the graph does not have negative-weight cycle

 

  • Definition: for vertex \(v \in V\), define \(\delta(v)\) to be the weight of the shortest path from \(s\) to \(v\) (we have \(\delta(s) = 0\))

Algorithm

8.2 - Bellman-Ford Algorithm

  • set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
  • for \(i = 1\) to \(|V|-1\):
    • for each edge \((u,v) \in E\), relax \((u,v\))
  • for each edge \((u,v)\in E\)
    • if \(\delta(v) > \delta(u) + w(u,v)\)
      • return false
  • return true

 

(the algorithm is actually very simple, relax every edge \(|V|-1\) times

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(\infty\)

\(\infty\)

\(\infty\)

\(\infty\)

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

s b c d e
0 inf inf inf inf

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(\infty\)

\(\infty\)

\(\infty\)

\(-4\)

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

s b c d e
0 inf inf inf -4

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(8\)

\(\infty\)

\(\infty\)

\(-4\)

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

s b c d e
0 8 inf inf -4

\(\delta(b) = 8\)

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(8\)

\(\infty\)

\(\infty\)

\(-4\)

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

s b c d e
0 8 inf inf -4

\(\delta(b) = 8\)

no change

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(8\)

\(17\)

\(\infty\)

\(-4\)

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

s b c d e
0 8 inf 17 -4

\(\delta(b) = 8\)

no change

\(\delta(d) = 17\)

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

\(\delta(b) = 8\)

no change

\(\delta(d) = 17\)

no change

\(8\)

\(17\)

\(\infty\)

\(-4\)

s b c d e
0 8 inf 17 -4

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

\(\delta(b) = 8\)

no change

\(\delta(d) = 17\)

no change

\(\delta(c) = 12\)

\(8\)

\(17\)

\(12\)

\(-4\)

s b c d e
0 8 12 17 -4

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

\(\delta(b) = 8\)

no change

\(\delta(d) = 17\)

no change

\(\delta(c) = 12\)

\(\delta(d) = -3\)

\(8\)

\(-3\)

\(12\)

\(-4\)

s b c d e
0 8 12 -3 -4

Are we done?

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(\delta(e) = -4\)

\(\delta(b) = 8\)

no change

\(\delta(d) = 17\)

no change

\(\delta(c) = 12\)

\(\delta(d) = -3\)

\(8\)

\(-3\)

\(12\)

\(-4\)

s b c d e
0 8 12 -3 -4

Are we done?

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(8\)

\(-3\)

\(12\)

\(-4\)

s b c d e
0 8 12 -3 -4

The weight of the shortest path to from \(s\) to \(c\) is -8, however, this requires us to relax \((s,e)\), \((e,d)\), and then \((d,c)\)

 

We didn't relax the paths in this order


So what do we do now? Keep going

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(8\)

\(-3\)

\(-8\)

\(-4\)

s b c d e
0 8 -8 -3 -4

no change

no change

\(\delta(c) = -8\)

no change

no change

no change

no change

Example

8.2 - Bellman-Ford Algorithm

e

b

s

d

c

8

3

9

-5

-4

1

7

\(0\)

\((s,e)\)

\((b,e)\)

\((s,b)\)

\((b,d)\)

\((c,b)\)

\((d,c)\)

\((e,d)\)

\(-1\)

\(-3\)

\(-8\)

\(-4\)

s b c d e
0 -1 -8 -3 -4

no change

no change

no change

no change

no change

\(\delta(b) = -1\)

no change

Correctness

8.2 - Bellman-Ford Algorithm

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 1, 2, 3?
  • Did we try the path using edges 1, 2, 4?
  • Did we try the path using edges 4, 3, 1?
  • You can see the proof of correctness in CLRS pages 652-654
  • Here is an intuition
    • let's say that there are 4 vertices, and 4 edges

Correctness

8.2 - Bellman-Ford Algorithm

  • You can see the proof of correctness in CLRS pages 652-654
  • Here is an intuition
    • let's say that there are 4 vertices, and 4 edges

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 1, 2, 3?
  • Did we try the path using edges 1, 2, 4?
  • Did we try the path using edges 4, 3, 1?

Correctness

8.2 - Bellman-Ford Algorithm

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 1, 2, 3?
  • Did we try the path using edges 1, 2, 4?
  • Did we try the path using edges 4, 3, 1?
  • You can see the proof of correctness in CLRS pages 652-654
  • Here is an intuition
    • let's say that there are 4 vertices, and 4 edges

Correctness

8.2 - Bellman-Ford Algorithm

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 1, 2, 3?
  • Did we try the path using edges 1, 2, 4?
  • Did we try the path using edges 4, 3, 1?
  • You can see the proof of correctness in CLRS pages 652-654
  • Here is an intuition
    • let's say that there are 4 vertices, and 4 edges

Correctness

8.2 - Bellman-Ford Algorithm

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 4, 3, 2, 1?
  • You can see the proof of correctness in CLRS pages 652-654
  • Here is an intuition
    • let's say that there are 4 vertices, and 4 edges

Correctness

8.2 - Bellman-Ford Algorithm

  • Did we try every possible permutation of edges?
    • yes, but wouldn't this cost \(O(E!)\)?
    • no, because there are some overlaps as you can see

1

2

3

4

1

2

3

4

1

2

3

4

  • Did we try the path using edges 1, 2, 3?
  • Did we try the path using edges 1, 2, 4?
  • Did we try the path using edges 4, 3, 1?

Algorithm

8.2 - Bellman-Ford Algorithm

  • set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
  • for \(i = 1\) to \(|V|-1\):
    • for each edge \((u,v) \in E\), relax \((u,v\))
  • for each edge \((u,v)\in E\)
    • if \(\delta(v) > \delta(u) + w(u,v)\)
      • return false
  • return true

 

(the algorithm is actually very simple, relax every edge \(|V|-1\) times

Algorithm

8.2 - Bellman-Ford Algorithm

  • set \(\delta(s) = 0\) and set \(\delta(v) = \infty\) for all \(v \in V \setminus\{s\}\)
  • for \(i = 1\) to \(|V|-1\):
    • for each edge \((u,v) \in E\), relax \((u,v\))
  • why only do it \(|V|-1\) times?
    • because you would have tried all combinations of \(|V|-1\) edges at this point, and the shortest path should not contain more than \(|V|-1\) edges
    • you should not visit the same vertex twice, this means there is a cycle

Algorithm

8.2 - Bellman-Ford Algorithm

  • for each edge \((u,v)\in E\)
    • if \(\delta(v) > \delta(u) + w(u,v)\)
      • return false
  • return true

 

  • What does this bit do?
  • After performing the algorithm \(|V| - 1\) times, just try to relax all the edges one more time.
    • If somehow you still get a better path, that means the graph contains a negative weight cycle, and this algorithm does not work in this case

Complexity

8.2 - Bellman-Ford Algorithm

  • It is clear that the complexity of the algorithm is \(O(VE)\) 
  • Contrast this with Dijkstra's \(O(V\log V + E)\)
    • \(E = O(V^2)\) (consider a graph with \(n\) vertices, how many edges can you have at most)?
    • so \(O(VE)\) can be as bad as \(O(V^3)\)
       
  • You can actually improve Bellman-Ford to \(O(V+E)\) if you have a directed acyclic graph (see CLRS 24.2 if you want more information)
     
  • I am not going to test you on the formal proof of correctness or complexity, but I do need you to understand the basic idea