Graphs

Vocabulary, representation, and the questions we ask about paths

← OUTLINE

Graphs are everywhere

• A graph G = (V, E) is a set of vertices V and a set of edges E, each edge a pair of vertices.
• Road networks, computer networks, social networks: anything with things and connections.
• The state space of a puzzle or game: vertices are positions, edges are legal moves.
• Dependencies: course prerequisites, build steps, spreadsheet cells. One thing must happen before another.
• Today's algorithms are the first thing anyone runs on a graph, and the building block of everything after.

THE RUNNING EXAMPLE: 8 VERTICES, 8 EDGES, UNDIRECTED. H IS ON ITS OWN.

Vocabulary

• Undirected edge {u, v}: both directions. Directed edge (u, v): from u to v only.
• We assume graphs are simple: no repeated edges, no self-loops. So |E| ≤ |V|², and |E| can be as small as 0.
• Neighbors of u: Adj(u) = {v : (u, v) ∈ E}. In a directed graph, outgoing neighbors Adj⁺(u) and incoming Adj⁻(u).
• Degree deg(u) = |Adj(u)|. Every undirected edge is counted twice across all degrees, so Σ_u deg(u) = 2|E|.
• A path is a sequence of vertices with an edge between each consecutive pair. Its length is its number of edges.

Adj(A) = {B, C}, deg(A) = 2.  Adj(F) = {D, E, G}, deg(F) = 3.

Representation: adjacency lists

• Store Adj(u) for every vertex u: a map from each vertex to a list of its neighbors.
• The map is a direct-access array (vertices 0 to |V| − 1) or a hash table (any labels). Each list is an array or linked list, since we mostly just iterate over it.
• Space: Θ(|V|) for the map plus Θ(deg(u)) per list, so Θ(|V| + |E|) in total.
• Linear time for a graph algorithm means O(|V| + |E|): the size of the input.

 1  G = {
 2      'A': ['B', 'C'],
 3      'B': ['A', 'D'],
 4      'C': ['A', 'D', 'E'],
 5      'D': ['B', 'C', 'F'],
 6      'E': ['C', 'F'],
 7      'F': ['D', 'E', 'G'],
 8      'G': ['F'],
 9      'H': [],
10  }

EVERY UNDIRECTED EDGE APPEARS IN TWO LISTS.

Distance, and three questions about paths

• Reachability(G, s, t): is there any path from s to t?
• Single-pair shortest path(G, s, t): return δ(s, t) and a path that achieves it.
• Single-source shortest paths(G, s): return δ(s, v) for every v.
• Each problem solves the ones above it. We will solve the hardest one in O(|V| + |E|).

δ(A, v) FOR EVERY VERTEX v. H IS UNREACHABLE.

Parent pointers and the shortest-path tree

• Returning a full path to every vertex can take Ω(|V|²) space. Instead store one thing per vertex.
• parent[v] = the vertex just before v on some shortest path from s. parent[s] = None.
• To recover the path to v: follow parent pointers back to s, then reverse.
• All the parent pointers together form the shortest-path tree: O(|V|) size, every root-to-vertex path is a shortest path.

TEAL: THE SHORTEST-PATH TREE FROM A. parent[G] = F, parent[F] = D, parent[D] = B, parent[B] = A.

Breadth-first search

Explore in order of distance: level by level

← OUTLINE

The idea: level sets

• Explore vertices in increasing order of distance from s.
• Level L_i = all vertices at distance exactly i. L₀ = {s}.
• Claim 1. Every vertex in L_i is a neighbor of some vertex in L_i−1.
• Claim 2. No vertex appears in two different levels.
• So: to build L_i, look at every neighbor of every vertex in L_i−1, and keep the ones not seen before. Set d[v] = i and parent[v] = the vertex that found it.
• Stop when a level comes out empty.

BUILDING L₃ FROM L₂ = {D, E}: THE AMBER EDGES ARE THE ONES BEING CHECKED.

Walkthrough 1 of 2: levels 0, 1, 2

• L₀ = {A}. d[A] = 0, parent[A] = None.
• Neighbors of A: B and C, both new. L₁ = {B, C}, parents A and A.
• Neighbors of B: A (seen), D (new). Neighbors of C: A (seen), D (seen now), E (new). L₂ = {D, E}, parent[D] = B, parent[E] = C.
• D was found through B, because B is listed before C in L₁. Through C would have been just as short.

MAGENTA: THE LEVEL JUST BUILT.

Walkthrough 2 of 2: levels 3, 4, and the end

• Neighbors of D: B, C (seen), F (new). Neighbors of E: C (seen), F (seen now). L₃ = {F}, parent[F] = D.
• Neighbors of F: D, E (seen), G (new). L₄ = {G}, parent[G] = F.
• Neighbors of G: F (seen). L₅ = {}. Stop.
• H was never found: d[H] = ∞, and it has no parent.

THE FINISHED SEARCH: FIVE LEVELS, ONE UNREACHABLE VERTEX.

Try it: breadth-first search, one event at a time

Try it yourself: https://courses.lpcordova.phd/tools/graph-search.html?algo=bfs&start=A&controls=0

BFS in Python: levels become a queue

 1  def bfs(adj, s):
 2      d, parent = {s: 0}, {s: None}
 3      queue = [s]                 # FIFO
 4      while queue:
 5          u = queue.pop(0)        # dequeue
 6          for v in adj[u]:
 7              if v not in d:      # not seen yet
 8                  d[v] = d[u] + 1
 9                  parent[v] = u
10                  queue.append(v)
11      return d, parent

• A first-in, first-out queue holds the discovered-but-unexplored vertices. It always contains the rest of one level followed by the start of the next, so dequeuing visits levels in order.
• v not in d is the seen check: d doubles as the visited set.
• In practice use collections.deque; list.pop(0) is O(n).
• Vertices never reached are simply absent from d: treat that as ∞.

Cost and correctness

• Each vertex is enqueued at most once (the seen check), so dequeued at most once.
• When u is dequeued we do O(1) work per neighbor: O(deg(u)) in total for u.
• Summing over all vertices: O(Σ deg(u)) = O(|E|) by the handshake fact, plus O(|V|) to set up d and report ∞ for the rest. O(|V| + |E|): linear.
• Correctness, by induction on i: suppose d and parent are right for every vertex in L₀, …, L_i−1. A vertex v at distance i has a neighbor u at distance i − 1 (the one before it on a shortest path), so v is found while scanning u's list, and gets d[v] = i. A vertex at distance less than i is already seen, so it is not added to L_i. ■

Depth-first search

Follow a path until stuck, then back up

← OUTLINE

The idea: go deep, then back up

• Searches from s like BFS, but with a different rule for which discovered vertex to explore next: the most recently discovered one.
• Follow any path forward until every neighbor has been seen, then backtrack to the most recent vertex with an unexplored neighbor.
• Naturally recursive: visit(u) looks at each neighbor v, and if v is new, sets parent[v] = u and calls visit(v).
• Returns a parent tree (the DFS tree) and the order in which visits finish. It does not compute distances: DFS paths are not shortest.

THE DFS TREE FROM A. t = DISCOVERY TIME (THE CLOCK ALSO TICKS AT EACH FINISH). THE PATH TO G HAS 6 EDGES; BFS FOUND ONE WITH 4.

Walkthrough

• visit(A): B is new ⁠→⁠ visit(B). visit(B): A seen; D new ⁠→⁠ visit(D).
• visit(D): B seen; C new ⁠→⁠ visit(C). visit(C): A, D seen; E new ⁠→⁠ visit(E).
• visit(E): C seen; F new ⁠→⁠ visit(F). visit(F): D, E seen; G new ⁠→⁠ visit(G).
• visit(G): F seen. G finishes. Back in F: done, F finishes. Then E, C, D finish.
• Back in B: done, B finishes. Back in A: C already seen. A finishes. H was never reached.

Discovery order

Finishing order

NEIGHBORS ARE ALWAYS TAKEN IN ALPHABETICAL ORDER.

Try it: depth-first search, one event at a time

Try it yourself: https://courses.lpcordova.phd/tools/graph-search.html?algo=dfs&start=A&controls=0

DFS in Python, cost, and correctness

 1  def dfs(adj, s):
 2      parent = {s: None}
 3      finished = []
 4      def visit(u):
 5          for v in adj[u]:
 6              if v not in parent:
 7                  parent[v] = u
 8                  visit(v)
 9          finished.append(u)      # u is done
10      visit(s)
11      return parent, finished

• Cost. Each vertex is visited at most once, with O(1) work per neighbor: O(Σ deg(u)) = O(|E|) for the search itself, O(|V| + |E|) with setup.
• Correctness. Every vertex reachable from s is visited: induct on distance. If v is at distance k + 1, its predecessor u (distance k) is visited; while visiting u, either v is already in parent or visit(v) is called.
• The recursion depth can reach |V|. For large graphs use an explicit stack.
• The finishing order is the real output of DFS; it is what the next section runs on.

Searching the whole graph: Full-DFS and Full-BFS

• One search finds only what is reachable from s. H was left out every time.
• Full-A: repeat until every vertex is visited: pick any unvisited vertex, run A from it.
• Each vertex is still visited once, so Full-BFS and Full-DFS run in O(|V| + |E|).
• Connected components of an undirected graph: the vertices found by each run form one component. Here: {A, B, C, D, E, F, G} and {H}.
• This also gives a connectivity test: connected if and only if the first run reaches everything.

SECOND RUN: visit(H) FINDS NOTHING NEW. TWO COMPONENTS.

BFS and DFS, side by side

Topological sort

Ordering a DAG with DFS finishing times

← OUTLINE

DAGs and topological order

• Edges are dependencies: “u must happen before v.” A topological order is a schedule that respects all of them.
• Two valid orders here: A, B, C, D, E, F and A, C, E, B, D, F. There are more.
• Claim. A directed graph has a topological order if and only if it is a DAG. (A cycle has no first vertex; and we are about to construct an order for every DAG.)

A TASK DAG: A BEFORE B AND C, D NEEDS B AND C, F NEEDS D AND E.

Reverse finishing order is a topological order

• Full-DFS from A, neighbors alphabetical: visit A ⁠→⁠ B ⁠→⁠ D ⁠→⁠ F. F finishes, then D, then B.
• Back in A: C ⁠→⁠ D is seen, E ⁠→⁠ F is seen. E finishes, C finishes, A finishes.
• Finishing order: F, D, B, E, C, A.
• Reversed: A ⁠→⁠ C ⁠→⁠ E ⁠→⁠ B ⁠→⁠ D ⁠→⁠ F. Check every edge: each points left to right.

t = DISCOVERY TIME. FINISHING ORDER F, D, B, E, C, A.

Why it works

• We need: for every edge (u, v), v finishes before u (so u comes first after reversing).
• Case 1: u is visited before v. While visit(u) is running, v is unvisited, so v will be visited (through (u, v) or some other way) and finishes inside the visit of u. So v finishes first.
• Case 2: v is visited before u. The graph is acyclic, so u is not reachable from v. visit(v) therefore finishes without ever touching u, and u is visited later. So v finishes first.
• Either way v finishes before u. Reversing the finishing order puts u before v for every edge. ■

Cycle detection

• If the graph has a cycle, the claim fails: the reverse finishing order will violate some edge.
• Check: compute the order, then test every edge (u, v): is u before v? O(|E|) with a position table.
• Better: find the cycle during the search. Keep the set of vertices currently on the recursion stack (the ancestors of u).
• Claim. If G has a cycle, Full-DFS will see an edge (u, v) with v still on the stack: a back edge. The stack from v to u, plus the edge, is a cycle.
• Why. Let v₀ be the first cycle vertex visited. Every other cycle vertex is reachable from v₀, so all are visited inside visit(v₀); the last one on the cycle has an edge back to v₀, which is still on the stack.

F ⁠→⁠ A IS A BACK EDGE: A IS ON THE STACK. CYCLE A ⁠→⁠ B ⁠→⁠ D ⁠→⁠ F ⁠→⁠ A.

Try it: topological sort, and a graph with a cycle

Try it yourself: https://courses.lpcordova.phd/tools/graph-search.html?algo=topo&graph=cycle&controls=0

Topological sort in Python

 1  def topological_sort(adj):
 2      parent, finished, on_stack = {}, [], set()
 3      def visit(u):
 4          on_stack.add(u)
 5          for v in adj[u]:
 6              if v in on_stack:
 7                  raise ValueError(f'cycle through {v}')
 8              if v not in parent:
 9                  parent[v] = u
10                  visit(v)
11          on_stack.remove(u)
12          finished.append(u)
13      for s in adj:                 # Full-DFS
14          if s not in parent:
15              parent[s] = None
16              visit(s)
17      return list(reversed(finished))

• on_stack is the set of ancestors of the current vertex: exactly the vertices whose visit has started but not finished.
• An edge to a vertex in on_stack is a back edge, and a cycle.
• An edge to a finished vertex is harmless: that vertex is already later in the order.
• The outer loop makes it Full-DFS, so vertices with no incoming edges are not missed.
• Another linear method, Kahn's algorithm, repeatedly removes a vertex with in-degree 0. Same result, no recursion.

Activity: Search Party

Trace all three by hand, in pairs

← OUTLINE

Search Party

• Write the level table before the parent table; the parents fall out of it.
• For DFS, keep a running call stack in the margin. Finishing order is the order the stack pops.
• Side 2 (f): the first vertex of the cycle you find may not be the first vertex you visited.

Wrap-up

What to keep, and what comes next

← OUTLINE

The one-slide version

• Graphs are stored as adjacency lists in Θ(|V| + |E|) space; linear time means O(|V| + |E|).
• BFS explores by distance using a queue. It computes δ(s, v) for every v and a shortest-path tree. O(|V| + |E|).
• DFS explores by recency using recursion (a stack). It computes a parent tree and a finishing order. O(|V| + |E|).
• Full-DFS and Full-BFS cover every vertex; the runs are the connected components.
• Topological sort: reverse the DFS finishing order. A back edge to a vertex on the stack means a cycle, so no order exists.
• Next: edges get weights. Prim's algorithm finishes the minimum spanning tree story, and Bellman-Ford finds shortest paths when fewest edges is no longer lightest.

References

1. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 4th ed. MIT Press, 2022. Sections 20.1 to 20.4 (representations, breadth-first search, depth-first search, topological sort).
2. E. F. Moore, “The shortest path through a maze,” in Proceedings of the International Symposium on the Theory of Switching, Harvard University Press, 1959, pp. 285–292.
3. R. Tarjan, “Depth-first search and linear graph algorithms,” SIAM Journal on Computing, vol. 1, no. 2, pp. 146–160, 1972.
4. A. B. Kahn, “Topological sorting of large networks,” Communications of the ACM, vol. 5, no. 11, pp. 558–562, 1962.