← OUTLINE

The greedy method

Take the best-looking choice now, and never revisit it

What greedy means

• Build the solution one choice at a time.
• At each step, take the choice that looks best by some local rule.
• Commit. No backtracking, no reconsidering.
• Usually fast and short: often one sort plus one pass.
• The hard part is never the code. It is showing that local choices add up to a globally optimal answer.

A scheduling problem

• One room, eight requests. Each request has a start and a finish time.
• Two requests are compatible if they do not overlap (one may start exactly when the other finishes).
• Goal: schedule as many requests as possible.
• Brute force would test all 2ⁿ subsets.

EIGHT REQUESTS FOR ONE ROOM, LISTED BY FINISH TIME.

Two tempting rules that fail

• Earliest start first. One long early request can block everything else.

• Shortest first. A short request can sit across the boundary of two long ones.

The rule that works: earliest finish first

def schedule(requests):          # (start, finish) pairs
    chosen, free_at = [], 0
    for s, f in sorted(requests, key=lambda r: r[1]):
        if s >= free_at:         # compatible with all chosen
            chosen.append((s, f))
            free_at = f
    return chosen

• Sort by finish time, then one pass.
• Picks a1, a4, a7, a8: four requests.
• Cost: O(n log n) for the sort, O(n) for the pass.

TEAL: CHOSEN. PALE: SKIPPED BECAUSE IT OVERLAPS A CHOSEN REQUEST.

Why it works: the exchange argument

• Let greedy choose g₁, g₂, …, g_k, and let some optimal schedule be o₁, o₂, …, o_m, both in order of finish time.
• g₁ finishes no later than o₁, because greedy took the earliest finish of all. So swap o₁ for g₁: the schedule is still compatible and still has m requests.
• Repeat at position 2, 3, … Each swap keeps the schedule valid and the same size (induction on the position).
• If m were larger than k, request o_k+1 would start after g_k finishes, and greedy would have taken it. So m = k.

Greedy is not always right

• Making change with the fewest coins. Greedy rule: take the largest coin that fits.
• Coins 25, 10, 5, 1: greedy is optimal.
• Coins 4, 3, 1, amount 6: greedy takes 4 + 1 + 1, three coins. The optimum is 3 + 3, two coins.
• Same rule, same code, different input family, wrong answer.

← OUTLINE

Minimum spanning trees

Connect everything, as cheaply as possible

Graph vocabulary in one slide

• A graph G = (V, E): vertices and edges. Here it is undirected and weighted: each edge has a cost w(u, v).
• We store it as adjacency lists: each vertex keeps a list of (neighbor, weight) pairs. Space O(V + E).
• Connected: a path exists between every pair of vertices.
• A tree is connected and has no cycle. On n vertices it has exactly n − 1 edges.
• Our running example: 7 vertices, 12 edges, all weights different.

THE RUNNING EXAMPLE FOR THE REST OF THE DECK.

The problem

• Every spanning tree has exactly |V| − 1 edges.
• A graph can have a huge number of spanning trees: the complete graph on n vertices has nⁿ⁻².
• Uses: laying cable or pipe, circuit wiring, clustering, and as a building block in approximation algorithms.
• If all weights are different, the MST is unique.

THE MST OF THE EXAMPLE: TOTAL WEIGHT 23.

The cut property

• A cut splits the vertices into two non-empty groups, S and V − S.
• An edge crosses the cut if it has one endpoint in each group.
• Here S = {A, D, F}. Crossing edges: A⁠–⁠B (4), B⁠–⁠D (5), D⁠–⁠E (9), E⁠–⁠F (10), F⁠–⁠G (12).

DOTTED LINE: THE CUT. AMBER: ITS LIGHTEST CROSSING EDGE, A⁠–⁠B.

Proof of the cut property: exchange again

• Let e be the lightest edge crossing the cut, and suppose some MST T does not contain e.
• Add e to T. A tree plus one edge has exactly one cycle, and that cycle goes through e.
• The cycle leaves S along e, so it must come back: it crosses the cut along some other edge f of T.
• e is the lightest crossing edge, so w(e) < w(f).
• Exchange: remove f, keep e. The result is still connected, still has |V| − 1 edges, so it is a spanning tree, and it is lighter than T.
• That contradicts T being minimum. So every MST contains e. ■

← OUTLINE

Kruskal's algorithm

Sort the edges, keep each one that does not close a cycle

The algorithm

1. Start with every vertex as its own one-vertex tree (a forest).
2. Sort the edges by weight, lightest first.
3. For each edge (u, v) in that order: if u and v are in different trees, add the edge and merge the two trees. If they are in the same tree, skip it (it would close a cycle).
4. Stop after |V| − 1 edges have been added.

Walkthrough 1 of 3: the three lightest edges

• Sorted weights: 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12.
• E⁠–⁠G (1): different trees. Add.
• A⁠–⁠D (2): different trees. Add.
• B⁠–⁠E (3): different trees. Add.
• The forest now has trees {A, D}, {B, E, G}, {C}, {F}.

TEAL: IN THE FOREST. PALE: NOT LOOKED AT YET.

Walkthrough 2 of 3: a merge, then a rejection

• A⁠–⁠B (4): A is in {A, D}, B is in {B, E, G}. Different trees. Add, and merge them into {A, B, D, E, G}.
• B⁠–⁠D (5): B and D are now in the same tree. Adding it would close the cycle A, B, D. Skip.
• A skipped edge is never looked at again.

MAGENTA DASHED: REJECTED, BOTH ENDPOINTS ALREADY CONNECTED.

Walkthrough 3 of 3: done after six edges

• C⁠–⁠E (6): C is alone. Add.
• D⁠–⁠F (7): F is alone. Add.
• Six edges on seven vertices: stop. Edges 8 through 12 are never examined.
• Total weight: 1 + 2 + 3 + 4 + 6 + 7 = 23.

Same tree or not? Union-find

class UnionFind:
    def __init__(self, items):
        self.parent = {x: x for x in items}
        self.rank = {x: 0 for x in items}

    def find(self, x):               # the set's representative
        if self.parent[x] != x:
            self.parent[x] = self.find(self.parent[x])
        return self.parent[x]

    def union(self, a, b):           # False if already together
        ra, rb = self.find(a), self.find(b)
        if ra == rb:
            return False
        if self.rank[ra] < self.rank[rb]:
            ra, rb = rb, ra
        self.parent[rb] = ra
        if self.rank[ra] == self.rank[rb]:
            self.rank[ra] += 1
        return True

• Each set is a tree of parent pointers. The root is the set's name.
• find walks to the root. union hangs one root under the other.
• Union by rank: the shorter tree goes under the taller one, so heights stay O(log n).
• Path compression: on the way back, point every visited node straight at the root.
• With both, any sequence of m operations costs O(m α(n)), where α(n) ≤ 4 for any n you will ever meet.

Kruskal in Python

def kruskal(vertices, edges):          # edges: (w, u, v) triples
    uf = UnionFind(vertices)
    tree = []
    for w, u, v in sorted(edges):       # lightest first
        if uf.union(u, v):              # different trees: safe edge
            tree.append((u, v, w))
    return tree

• Sorting: O(E log E), which is O(E log V) because E ≤ V².
• At most E unions and 2E finds: O(E α(V)), nearly linear.
• Total: O(E log V). The sort dominates.

← OUTLINE

Prim's algorithm

Grow one tree, always by the cheapest edge leaving it

The algorithm

1. Pick any start vertex s. The tree is {s}.
2. Among all edges with one endpoint in the tree and one outside, take the lightest. Add it and its outside endpoint to the tree.
3. Repeat until every vertex is in the tree.

Walkthrough 1 of 2: start at A

• Tree {A}. Crossing edges: A⁠–⁠B (4), A⁠–⁠D (2). Take A⁠–⁠D (2).
• Tree {A, D}. Crossing: A⁠–⁠B (4), B⁠–⁠D (5), D⁠–⁠F (7), D⁠–⁠E (9). Take A⁠–⁠B (4).
• Tree {A, B, D}. Crossing: B⁠–⁠E (3), D⁠–⁠F (7), B⁠–⁠C (8), D⁠–⁠E (9). Take B⁠–⁠E (3).
• Next: the crossing edges now include E⁠–⁠G (1).

TEAL VERTICES: IN THE TREE. AMBER: THE LIGHTEST EDGE LEAVING IT.

Walkthrough 2 of 2: the same tree, a different order

• Take E⁠–⁠G (1), then C⁠–⁠E (6), then D⁠–⁠F (7).
• Prim's order: 2, 4, 3, 1, 6, 7. Kruskal's order: 1, 2, 3, 4, 6, 7.
• Same six edges, total weight 23. With distinct weights there is only one MST, so both must find it.
• Prim's tree is connected at every step. Kruskal's forest is not.

Prim in Python

import heapq

def prim(adj, s):                 # adj[u]: list of (v, w)
    in_tree = {s}
    tree = []
    pq = [(w, s, v) for v, w in adj[s]]
    heapq.heapify(pq)
    while pq and len(in_tree) < len(adj):
        w, u, v = heapq.heappop(pq)   # lightest candidate
        if v in in_tree:
            continue                  # no longer crosses the cut
        in_tree.add(v)
        tree.append((u, v, w))
        for x, wx in adj[v]:
            if x not in in_tree:
                heapq.heappush(pq, (wx, v, x))
    return tree

• The heap holds candidate edges. Stale ones (both endpoints inside) are discarded when popped.
• Each edge is pushed at most twice and popped at most twice.
• Heap size is O(E), so each push or pop is O(log E) = O(log V).
• Total: O(E log V).

Kruskal or Prim?

← OUTLINE

Activity: Span It

Both algorithms, by hand, on one graph

Your turn, in pairs

• One worksheet per pair, both names on it.
• Side 1: run Kruskal. List the edges in the order considered and mark each one added or skipped.
• Side 1: run Prim from S. List the edges in the order added.
• Side 2: one weight changes. Use the cut property to find the new MST without starting over.

Debrief: Kruskal

• V⁠–⁠X (1), T⁠–⁠Y (2), V⁠–⁠Y (3): add, add, add.
• T⁠–⁠V (4): skip. T and V are already connected through Y.
• S⁠–⁠U (5), U⁠–⁠W (6): add, add.
• X⁠–⁠Y (7): skip. X and Y are already connected through V.
• U⁠–⁠Y (8): add. Six edges: stop.
• Total: 1 + 2 + 3 + 5 + 6 + 8 = 25.

Debrief: Prim from S

• {S}: take S⁠–⁠U (5).
• Then U⁠–⁠W (6), then U⁠–⁠Y (8): the tree has to pay 8 to reach the middle.
• From Y the cheap edges open up: T⁠–⁠Y (2), V⁠–⁠Y (3), V⁠–⁠X (1).
• Order added: 5, 6, 8, 2, 3, 1. Total 25, the same tree as Kruskal.

Debrief: U⁠–⁠Y goes from 8 to 13

• Remove U⁠–⁠Y from the old MST. It falls into two pieces: {S, U, W} and {T, V, X, Y}.
• That is a cut. The other five MST edges are still safe (their cuts did not change).
• Edges crossing the cut: S⁠–⁠T (9), W⁠–⁠Y (10), W⁠–⁠X (11), S⁠–⁠Y (12), U⁠–⁠Y (13).
• The lightest is S⁠–⁠T (9). Swap it in.
• New total: 25 − 8 + 9 = 26. No re-run needed.

← OUTLINE

Wrap-up

What to keep from today

The one-slide version

• Greedy: commit to the locally best choice. Fast, short, and only correct when you can prove it.
• Exchange argument: swap an optimal solution's choice for the greedy choice and show nothing gets worse.
• Cut property: the lightest edge across any cut is in the MST.
• Kruskal: sorted edges plus union-find. O(E log V).
• Prim: one growing tree plus a priority queue. O(E log V).
• Next: weighted shortest paths, where the same greedy instinct needs more care.

References

1. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 4th ed. MIT Press, 2022. Section 15.1 (activity selection), Chapter 19 (disjoint sets), Chapter 21 (minimum spanning trees).
2. J. B. Kruskal, “On the shortest spanning subtree of a graph and the traveling salesman problem,” Proceedings of the American Mathematical Society, vol. 7, no. 1, 1956.
3. R. C. Prim, “Shortest connection networks and some generalizations,” Bell System Technical Journal, vol. 36, no. 6, 1957.
4. R. E. Tarjan, “Efficiency of a good but not linear set union algorithm,” Journal of the ACM, vol. 22, no. 2, 1975.