The problem

What a shortest path is once edges have weights

← OUTLINE

Weighted, directed graphs

• Each edge (u, v) has a direction and a weight w(u, v): a distance, a time, a cost.
• The weight of a path is the sum of its edge weights.
• Two ways from S to A here: the direct edge costs 5. Going S ⁠→⁠ B ⁠→⁠ A costs 2 + 1 = 3.
• Fewest edges is not the same as lightest. Counting hops gives the wrong answer as soon as weights differ.

THE RUNNING EXAMPLE. ONE EDGE HAS A NEGATIVE WEIGHT.

The single-source shortest paths problem

• Input: a weighted directed graph and a source vertex s.
• Output: δ(s, v) for every vertex v.
• We also store a parent pointer for each vertex: the previous vertex on one shortest path. Following parents back to s recovers the path.
• The parent pointers form a shortest-path tree rooted at s.

TEAL: THE SHORTEST-PATH TREE FROM S, WITH δ(S, v) AT EACH VERTEX.

Negative weights and negative cycles

• A negative edge is fine. Shortest paths are still well defined.
• A negative cycle is not: a cycle whose weights sum to less than zero.
• Here X ⁠→⁠ Y ⁠→⁠ Z ⁠→⁠ X sums to 2 − 5 + 1 = −2. Every lap makes the path cheaper, so there is no smallest weight.
• For any vertex reachable through a negative cycle we say δ(s, v) = −∞.
• A correct algorithm must either rule negative cycles out or detect them.

A CYCLE OF TOTAL WEIGHT −2.

Relaxation

One operation that every shortest-path algorithm repeats

← OUTLINE

Estimates that only improve

• Keep an estimate d[v] for every vertex: the weight of the best path to v found so far.
• Start with d[s] = 0 and d[v] = ∞ for everything else.
• Relaxing an edge (u, v) asks one question: is going through u better than what I have for v?

def relax(u, v, w, d, parent):
    if d[u] + w < d[v]:
        d[v] = d[u] + w
        parent[v] = u

Order matters

• Relaxing edges in a good order finishes fast. Relax S ⁠→⁠ B, then B ⁠→⁠ A, then A ⁠→⁠ C, then C ⁠→⁠ D: four relaxations, all five answers correct.
• Relax C ⁠→⁠ D first and nothing happens: d[C] is still ∞, so there is nothing to extend.
• An estimate becomes correct once the edges of a shortest path to that vertex have been relaxed in path order.
• Different algorithms are different answers to one question: in what order do we relax?

RELAXING THE TEAL EDGES IN PATH ORDER IS ENOUGH.

Bellman-Ford

Relax every edge, |V| − 1 times

← OUTLINE

The algorithm

1. Set d[s] = 0 and every other estimate to ∞.
2. Repeat |V| − 1 times: relax every edge in the graph, in any fixed order.
3. Make one more pass. If any edge can still be relaxed, report a negative cycle.

Walkthrough 1 of 3: round 1

Edge order every round: C⁠→⁠D, A⁠→⁠C, B⁠→⁠C, B⁠→⁠D, B⁠→⁠A, S⁠→⁠A, S⁠→⁠B.

• C ⁠→⁠ D, A ⁠→⁠ C, B ⁠→⁠ C, B ⁠→⁠ D, B ⁠→⁠ A: nothing to extend, every tail is still at ∞.
• S ⁠→⁠ A: 0 + 5 beats ∞. d[A] = 5.
• S ⁠→⁠ B: 0 + 2 beats ∞. d[B] = 2.
• One round moved the estimates one edge away from S.

MAGENTA: ESTIMATES THAT CHANGED THIS ROUND.

Walkthrough 2 of 3: rounds 2 and 3

• Round 2. A ⁠→⁠ C: d[C] = 5 + 3 = 8. B ⁠→⁠ D: d[D] = 2 + 6 = 8. B ⁠→⁠ A: 2 + 1 = 3 beats 5, so d[A] = 3.
• d[C] = 8 was computed from the old d[A] = 5. It is now stale.
• Round 3. C ⁠→⁠ D: d[D] = 8 − 4 = 4. A ⁠→⁠ C: 3 + 3 = 6 beats 8, so d[C] = 6.
• Now d[D] = 4 is stale: it used the old d[C].

THE GRAPH AFTER ROUND 3.

Walkthrough 3 of 3: round 4 and the check

• Round 4 (round |V| − 1). C ⁠→⁠ D: 6 − 4 = 2 beats 4, so d[D] = 2. Every estimate now equals δ.
• Check pass. No edge can be relaxed: no negative cycle.
• The shortest path to D, S ⁠→⁠ B ⁠→⁠ A ⁠→⁠ C ⁠→⁠ D, has four edges. It took four rounds.

Why |V| − 1 rounds are enough

• With no negative cycle, some shortest path to each vertex is simple (no repeated vertex), so it has at most |V| − 1 edges.
• Claim: after round k, d[v] is correct for every vertex whose shortest path uses at most k edges.
• Base: k = 0. Only s, and d[s] = 0.
• Step: let the shortest path to v end with the edge (u, v). The path to u has at most k − 1 edges, so d[u] was correct after round k − 1. Round k relaxes (u, v), which sets d[v] = δ(s, u) + w(u, v) = δ(s, v).
• Safety says d[v] never drops below δ(s, v), so once correct it stays correct. ■

Bellman-Ford in Python

def bellman_ford(vertices, edges, s):    # edges: (u, v, w)
    d = {v: float('inf') for v in vertices}
    parent = {v: None for v in vertices}
    d[s] = 0
    for _ in range(len(vertices) - 1):
        for u, v, w in edges:
            if d[u] + w < d[v]:          # relax
                d[v] = d[u] + w
                parent[v] = u
    for u, v, w in edges:                # check pass
        if d[u] + w < d[v]:
            raise ValueError('negative cycle')
    return d, parent

• Two nested loops: O(V · E).
• Space: O(V) for the estimates and parents.
• Early exit: if a whole round changes nothing, stop. The answers are final.
• On a dense graph E is about V², so this is O(V³). We can do better when no weight is negative.

Wrap-up

What to keep, and what comes next

← OUTLINE

The one-slide version

• The problem: from one source, the lightest path to every vertex. Lightest is not fewest edges.
• Negative cycles make shortest paths undefined. Negative edges alone do not.
• Relaxation: try to improve d[v] through an edge (u, v). Estimates are never too low.
• Bellman-Ford: relax every edge |V| − 1 times, then one check pass. O(V · E), negative weights allowed.
• Next: Dijkstra's algorithm. With no negative weights, a greedy order and a priority queue bring the cost down to O(E log V).

References

1. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 4th ed. MIT Press, 2022. Chapter 22 introduction and Section 22.1 (the Bellman-Ford algorithm).
2. R. Bellman, “On a routing problem,” Quarterly of Applied Mathematics, vol. 16, no. 1, 1958.
3. L. R. Ford Jr., “Network flow theory,” RAND Corporation, Paper P-923, 1956.