The exam

What is on it and how it runs

← OUTLINE

The midterm at a glance

How to prove something

A framework

← OUTLINE

What a proof is

• A proof is an argument a skeptical reader can check, one step at a time.
• Every step is either something given, a definition, or follows from earlier steps by a rule you can name.
• Examples are not proofs. Checking n = 1, 2, 3 shows the claim is plausible, not that it is true.
• You do not need to be clever. You need to pick the right template and fill it in.

Step 2: the shape of the claim picks the technique

Induction: the template

1. State P(n). Write the claim as a statement about one number n. Say what n measures (size, height, number of inserts).
2. Base case. Show P holds for the smallest n. Actually check it.
3. Inductive hypothesis. “Assume P(k) holds for some k ≥ base.” (Strong form: assume P(j) for every j ≤ k.)
4. Inductive step. Show P(k + 1). Break the size k + 1 object into smaller pieces, apply the hypothesis to the pieces, and combine.
5. Conclude. “By induction, P(n) holds for all n ≥ base.”

Worked proof 1: induction

1. P(h): a perfect binary tree of height h has 2^h+1 − 1 nodes. Induction on h.
2. Base, h = 0: a single node. 2¹ − 1 = 1. True.
3. Hypothesis: assume a perfect tree of height k has 2^k+1 − 1 nodes.
4. Step: a perfect tree of height k + 1 is a root plus two perfect subtrees of height k. By the hypothesis each has 2^k+1 − 1 nodes. Total: 1 + 2(2^k+1 − 1) = 2^k+2 − 1.
5. Conclude: by induction, P(h) holds for every h ≥ 0. ■

HEIGHT 2: A ROOT PLUS TWO PERFECT TREES OF HEIGHT 1. 1 + 2(3) = 7.

Loop invariants: the template

1. State the invariant. “At the start of iteration i, …”
2. Initialization. It is true before the first iteration.
3. Maintenance. If it is true before an iteration, it is true before the next one.
4. Termination. When the loop ends, the invariant plus the exit condition give you what you wanted.

Worked proof 2: a loop invariant

1  def insertion_sort(A):
2      for i in range(1, len(A)):
3          key = A[i]
4          j = i - 1
5          while j >= 0 and A[j] > key:
6              A[j + 1] = A[j]
7              j -= 1
8          A[j + 1] = key

1. Initialization, i = 1: A[0:1] is one item. One item is sorted.
2. Maintenance: assume A[0:i] is sorted. The inner loop shifts every item larger than key one place right and drops key into the gap. Now A[0:i+1] is sorted, which is the invariant for i + 1.
3. Termination: the loop stops when i = n. The invariant says A[0:n] is sorted. That is the whole array. ■

Worked proof 3: contradiction

1. Assume the opposite: the largest key sits at some node x that is not the root.
2. x is not the root, so x has a parent p.
3. The max-heap property says key(p) ≥ key(x). Keys are distinct, so key(p) > key(x).
4. But key(x) is the largest key in the heap. Contradiction.
5. So the assumption was false: the largest key is at the root. ■

Five ways proofs fall short

• Assuming what you are proving. If your step starts from P(k + 1), you are going backwards.
• No base case, or a base case that is asserted and not checked.
• The hypothesis is never used. Then the step is either wrong or not induction.
• Proof by example. “It works for n = 4” proves the claim for n = 4.
• Undefined letters. Say what n, k, and h are before you use them.

Recurrences

A recipe that solves almost all of them

← OUTLINE

Where a recurrence comes from

1  def merge_sort(A):
2      if len(A) <= 1:
3          return A
4      mid = len(A) // 2
5      L = merge_sort(A[:mid])     # one call, size n/2
6      R = merge_sort(A[mid:])     # one call, size n/2
7      return merge(L, R)          # O(n) work here

• Count the calls: that is a.
• Look at the argument: that is n / b.
• Everything else in the function body is f(n).

The recipe: three steps

For T(n) = a T(n / b) + n^d  (a ≥ 1, b > 1):

1. Read off a, b, and d (the exponent of the work term).
2. Compute the critical exponent c = log_b a.
3. Compare d with c.

Who does most of the work?

For T(n) = a T(n / b) + Θ(n^d), the comparison is between two exponents.

The recursion tree: root, levels, leaves

1  def find_max(A, lo, hi):          # max of A[lo:hi]
2      if hi - lo == 1:
3          return A[lo]
4      mid = (lo + hi) // 2
5      left = find_max(A, lo, mid)    # size n/2
6      right = find_max(A, mid, hi)   # size n/2
7      return max(left, right)        # O(1) work

• Every box is one execution of the function, labelled with its range.
• Root: the call you made, find_max(A, 0, 8).
• Levels: the calls created by lines 5 and 6.
• Leaves: calls on one item. They return on line 3 and make no calls.
• Own work: line 7 (or line 3), 1 unit per call. So work per level = calls per level: 1, 2, 4, 8.

find_max(A, 0, 8). TEAL: THE ROOT. AMBER: THE LEAVES.

Leaves do most of the work: d < c

1  def find_max(A, lo, hi):          # max of A[lo:hi]
2      if hi - lo == 1:
3          return A[lo]
4      mid = (lo + hi) // 2
5      left = find_max(A, lo, mid)    # size n/2
6      right = find_max(A, mid, hi)   # size n/2
7      return max(left, right)        # O(1) work

Finding the maximum by halves: T(n) = 2 T(n / 2) + 1

• Two calls, so a = 2. Each on half, so b = 2. One comparison outside the calls, so d = 0. c = log₂ 2 = 1. d < c.
• Root: 1 call of size n. Its own work: 1.
• Level 1: 2 calls, 1 unit each. Total 2. Level 2: 4 calls. Total 4.
• Leaves: n calls on single items, 1 unit each. Total n.
• Each level does twice the work of the one above.

Every level does the same work: d = c

1  def merge_sort(A):
2      if len(A) <= 1:
3          return A
4      mid = len(A) // 2
5      L = merge_sort(A[:mid])     # size n/2
6      R = merge_sort(A[mid:])     # size n/2
7      return merge(L, R)          # O(n) work here

Merge sort: T(n) = 2 T(n / 2) + n

• Two calls, so a = 2. Each on half, so b = 2. The merge is linear, so d = 1. c = log₂ 2 = 1. d = c.
• Root: 1 call of size n. Its own work: n (the final merge).
• Level 1: 2 calls, each merging n / 2 items. Total n.
• Level 2: 4 calls, each merging n / 4 items. Total n.
• Leaves: n calls on single items. Total n.

The root does most of the work: d > c

1  def search(A, k):               # binary search
2      if not A:
3          return False
4      mid = len(A) // 2
5      if A[mid] == k:
6          return True
7      if k < A[mid]:
8          return search(A[:mid], k)   # copies n/2
9      return search(A[mid + 1:], k)

Binary search that slices: T(n) = T(n / 2) + n

• One call runs, so a = 1. On half, so b = 2. The slice copies about n / 2 items, so d = 1. c = log₂ 1 = 0. d > c.
• Root: 1 call of size n. Its own work: n.
• Level 1: 1 call of size n / 2. Work n / 2. Level 2: work n / 4.
• Leaf: 1 call on an empty or one-item list. Work 1.
• n + n/2 + n/4 + … is less than 2n. The root alone is more than half of it.

From code to a recurrence, step by step

 1  def blend(A, lo, hi):
 2      n = hi - lo
 3      if n <= 1:
 4          return A[lo] if n == 1 else 0
 5      h = n // 2
 6      q = h // 2
 7      x = blend(A, lo, lo + h)          # left half
 8      y = blend(A, lo + h, hi)          # right half
 9      z = blend(A, lo + q, lo + q + h)  # middle half
10      s = 0
11      for i in range(lo, hi):           # one pass
12          s += A[i]
13      return max(x, y, z, s)

1. What is n? The size of the input this call works on. Here n = hi − lo.
2. Base case. When n ≤ 1 the function returns at once: T(1) = Θ(1).
3. Count the recursive calls that run in one execution. Three: x, y, z. So a = 3.
4. How big is each call's input? Each range has length h = n / 2. So b = 2.
5. Everything else is f(n). Cross out the recursive calls. What is left is a loop of n iterations plus constant work. So f(n) = n, and d = 1.

Then solve it with the recipe

 1  def blend(A, lo, hi):
 2      n = hi - lo
 3      if n <= 1:
 4          return A[lo] if n == 1 else 0
 5      h = n // 2
 6      q = h // 2
 7      x = blend(A, lo, lo + h)          # left half
 8      y = blend(A, lo + h, hi)          # right half
 9      z = blend(A, lo + q, lo + q + h)  # middle half
10      s = 0
11      for i in range(lo, hi):           # one pass
12          s += A[i]
13      return max(x, y, z, s)

T(n) = 3 T(n / 2) + n    so a = 3, b = 2, d = 1.

1. Critical exponent: c = log_b a = log₂ 3.
2. Estimate it with the definition of a log: 2¹ = 2 and 2² = 4, and 3 sits between them. So 1 < c < 2 (it is about 1.58).
3. Compare: d = 1 is less than c. The leaves do most of the work.
4. Answer: T(n) = Θ(n^{log₂ 3}), about n^1.58.

Two traps when reading code

Trap 1: calls that are written but do not all run.

1  def search(A, lo, hi, k):
2      if lo >= hi:
3          return -1
4      mid = (lo + hi) // 2
5      if A[mid] == k:
6          return mid
7      if k < A[mid]:
8          return search(A, lo, mid, k)
9      return search(A, mid + 1, hi, k)

• Two calls appear, but one execution makes one of them. a = 1, not 2.
• T(n) = T(n / 2) + 1 = Θ(log n).

Trap 2: work hidden inside one line.

1  def search(A, k):
2      if not A:
3          return False
4      mid = len(A) // 2
5      if A[mid] == k:
6          return True
7      if k < A[mid]:
8          return search(A[:mid], k)
9      return search(A[mid + 1:], k)

• A slice copies its elements: A[:mid] costs Θ(n).
• T(n) = T(n / 2) + n = Θ(n). The same algorithm, no longer logarithmic.

Six recurrences

Logarithm rules 

When the recipe does not apply

The recipe needs subproblems of size n / b. If the size shrinks by subtraction, unroll or draw the tree.

• Recursion tree, in three questions: how much work on level i? How many levels? What do the levels sum to?
• Geometric sums: shrinking terms sum to the first term, growing terms to the last term, equal terms to (one term) × (number of levels).
• Arithmetic sum: 1 + 2 + … + n = n(n + 1) / 2 = Θ(n²).

Choosing a data structure

The question type that matters most

← OUTLINE

The method: five steps

1. List the operations the problem needs. Underline the ones that happen most often.
2. Classify each operation. Is it by key, by position, by order (next, previous, range, smallest), or only the extreme (max or min)?
3. Write down the candidates. Usually three: one cheap and simple, one hash-based, one tree-based.
4. Fill in the cost table. One row per candidate, one column per operation.
5. Choose and justify. Name the operation that rules out each loser, and state the trade-off you accepted.

Step 2: the questions that decide it

The cost table to know cold

(a) amortized    (e) expected    All others are worst case.

Worked example: the print shop

1. Operations: add, cancel, next_after. All three are frequent.
2. Classify: add and cancel are by key. next_after is by order.

Three quick ones

How to write the answer

1. Name the structure in the first sentence.
2. Give the cost of every operation the problem asked for, with (a) or (e) where it applies.
3. Rule out the alternatives, one sentence each, naming the operation that sinks them.
4. State the trade-off you accepted, if there is one.
5. If the problem asks for it, describe how each operation works in two or three lines. Pseudocode or precise prose.

AVL trees

Review: the facts, the cases, one full insert

← OUTLINE

AVL trees: what to know

• Height: a leaf has height 0, an empty subtree −1. Skew(x) = h(right) − h(left).
• AVL property: every node has skew −1, 0, or +1. It forces h ≤ 1.44 log₂ n, so every operation is O(log n).
• Insert or delete as in a BST, then walk up recomputing heights. At the lowest node with skew ±2, rotate.

One full insert: add 22

• The tree holds 30, 20, 40, 10, 25. Insert 22: it goes left of 25.
• Walk up from the new leaf, recomputing heights.
• skew(25) = −1. skew(20) = +1. skew(30) = −2.
• Lowest unbalanced node: 30.
• Its left child 20 has skew +1, the opposite sign. That is the zigzag shape.
• Fix: rotate_left(20), then rotate_right(30).

LABELS: HEIGHT (SKEW). MAGENTA: UNBALANCED. AMBER: JUST INSERTED.

After the double rotation

• 25 was lifted to the root. 20 and 30 are its children.
• 22 changed parents: it was 25's left child and is now 20's right child.
• In-order before and after: 10, 20, 22, 25, 30, 40. Rotations never reorder keys.
• Height went from 3 back to 2.

TEAL: THE NODE THE ROTATION LIFTED INTO PLACE.

Heaps

Review: the array, the two moves, build

← OUTLINE

Binary heaps: what to know

• A complete binary tree stored in an array, level by level. No pointers.
• parent(i) = (i - 1) // 2
• left(i) = 2i + 1, right(i) = 2i + 2
• Max-heap property: every parent is at least its children. The max is at index 0.
• It is not a BST: there is no order between siblings.
• Height is ⌊log₂ n⌋, always.

THE ARRAY [9, 7, 8, 2, 5, 4] AS A TREE. LABELS ARE INDICES.

Build and delete_max, by hand

build on [2, 9, 4, 7, 5, 8]: sink down at index 2, then 1, then 0.

• delete_max: swap 9 with the last item (4), pop the 9, sink the 4. It swaps with 8. Result: [8, 7, 4, 2, 5].
• Sink down swaps with the larger child. Swapping with the smaller one breaks the property.

Sorts

Review: one table, then which one when

← OUTLINE

Every sort so far, in one table

Which sort when

Wrap-up

How to use the week

← OUTLINE

Between now and the exam

• Work the study guide's sample problems on paper, timed, about 12 minutes each.
• Rebuild the cost table for the five structures from memory, with a reason for each cell.
• Write out the three templates once: induction, loop invariant, contradiction.
• Redo one homework problem of each kind from a blank page, then compare with the posted solution.
• Trace by hand: an AVL insert with a double rotation, a heap build, one recurrence from each of the three cases.

References

1. T. H. Cormen, C. E. Leiserson, R. L. Rivest, and C. Stein, Introduction to Algorithms, 4th ed. MIT Press, 2022. Section 2.1 (loop invariants), Sections 4.3 to 4.5 (recurrences and the master method), Chapter 6 (heapsort), Chapter 8 (sorting in linear time), Chapters 11 to 13 (hash tables and search trees).
2. The CS 351 Midterm Study Guide: topics, what you may use, and sample problems.