COMP3010 - 6.1 - Recursion Tree Method

COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

6.1 - Recursion-Tree Method

Recursion-Tree Method

6.1 - Recursion-Tree Method

In the previous lecture, we learnt about recurrence equations
In this lecture, we're going to derive the complexity of the algorithm based on this recurrence equations
The steps:
- Draw out the recursion tree until you can see a pattern (2-3 levels should be enough)
  - put the cost of each subproblem in the nodes
- Count the number of operations on each level
- Count the number of levels (work out the height of the tree)
- Count the total number of operations

Mergesort

6.1 - Recursion-Tree Method

$\text{mergeSort}(0,n)$

$\text{mergeSort}(0,\lfloor\frac{n}{2}\rfloor)$

$\text{mergeSort}(\lceil\frac{n}{2}\rceil,n)$

$\text{mergeSort}(0,\lfloor\frac{n}{4}\rfloor)$

$\text{mergeSort}(\lceil\frac{n}{4}\rceil,\lfloor\frac{n}{2}\rfloor)$

$\text{mergeSort}(\lceil\frac{n}{2}\rceil,\lfloor\frac{3n}{4}\rfloor)$

$\text{mergeSort}(\lceil\frac{3n}{4}\rceil,n)$

Mergesort

6.1 - Recursion-Tree Method

$\text{mergeSort}(n)$

$\text{mergeSort}(\frac{n}{2})$

$\text{mergeSort}(\frac{n}{4})$

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Mergesort

6.1 - Recursion-Tree Method

$\text{mergeSort}(n)$

$\text{mergeSort}(\frac{n}{2})$

$\text{mergeSort}(\frac{n}{4})$

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

assume:
- $n$ is a power of 2
- we are doing $cn$ operations on each subproblem of size $n$
- we are doing 1 operation on the leaves of the tree

Mergesort

6.1 - Recursion-Tree Method

Step 1: Draw the recursion tree

$T(n)$

\[T\left(\frac{n}{2}\right)\]

\[T\left(\frac{n}{4}\right)\]

\[T\left(\frac{n}{2}\right)\]

. . .

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

\[T\left(\frac{n}{4}\right)\]

Mergesort

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Step 2: Count the number of operations on each level

$cn$

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

. . .

\[cn\]

\[\Theta(n)\]

Mergesort

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Step 3: Count the number of levels (height of the tree)

$cn$

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

. . .

\[cn\]

\[\Theta(n)\]

Mergesort

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Step 3: Count the number of levels (height of the tree)

on each level we divide the problem size by 2
- sizes: $\quad n \quad\rightarrow \quad n/2 \quad\rightarrow\quad n/4 \quad\rightarrow\quad\cdots\quad\rightarrow\quad n/2^i$
- remember, we assume that $n$ is a power of 2
the final level is when $n /2^i = 1$
- i.e. when $\quad 2^i = n \quad\rightarrow\quad i = \log_2n$
- so there are $\log_2 n+1$ levels
just to check: level $i$ has $2^i$ nodes, so the final level has $2^{\log_2 n}$ nodes = $n$ nodes (as we expected)

Mergesort

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Step 4: Count the total number of operations
- number of levels: $\log_2 n$
- number of operations on each level: $n$
- total number of operations: $n \log_2 n$

$T(n) = \Theta(n \log n)$

Mergesort

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n = 1$}\\ 2T(n/2) + cn &\text{if $n > 1$} \end{cases}$

Step 4: Count the total number of operations
- number of levels: $\log_2 n$
- number of operations on each level: $n$
- total number of operations: $n \log_2 n$

$T(n) = \Theta(n \log n)$

\[\underbrace{cn + cn+ \cdots + cn}_{\text{$(\log_2 n)$ terms}} + \Theta (n) = \Theta(n \log n)\]

Linear Search

6.1 - Recursion-Tree Method

public static int linearSearch(int[] a, int i, int j, int target) {
	if (i > j) 
		return -1;
	int m = (i+j)/2;
	if (a[m] == target)
		return m;
	return Math.max(linearSearch(a,i,m-1,target), linearSearch(a,m+1,j,target));
}

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 1: Draw the recursion tree

$T(n)$

\[T\left(\frac{n}{2}\right)\]

. . .

assume:
- $n$ is a power of 2
- we are doing $1$ operation on each node (including the leaves)

\[T\left(\frac{n}{2}\right)\]

\[T\left(\frac{n}{4}\right)\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 2: Count the number of operations on each level

$cn$

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

\[\frac{cn}{2}\]

\[\frac{cn}{4}\]

. . .

\[cn\]

\[\Theta(n)\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 2: Count the number of operations on each level

$1$

\[1\]

. . .

\[4\]

\[2\]

\[1\]

\[???\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 2: Count the number of operations on each level

as before, the final level is when $n /2^i = 1$
- i.e. when $\quad 2^i = n \quad\rightarrow\quad i = \log_2n$
level $i$ has $2^i$ nodes, so the final level has $2^{\log_2 n}$ nodes = $n$ nodes

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 2: Count the number of operations on each level

\[4\]

\[2\]

\[1\]

\[\Theta(2^{\log_2 n}) = \Theta(n)\]

$1$

\[1\]

. . .

Step 3: Count the number of levels (height of the tree)

as in the case of mergesort, there are $\log_2 n$ levels

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 4: Count the total number of operations

unfortunately, we cannot just multiply the number of levels with the number of operations on each level (because it changes)
we need to find the sum of
- $1 + 2 + 4 + \dots + \Theta(n) = 2^0 + 2^1 + 2^2 + \dots + 2^{\log_2 n-1} + \Theta(n)= \displaystyle\sum_{k=0}^{\log_2 n-1} 2^k + \Theta(n)$

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 4: Count the total number of operations

we need to find the sum of
- $1 + 2 + 4 + \dots + n = 2^0 + 2^1 + 2^2 + \dots + 2^{\log_2 n-1} = \displaystyle\sum_{k=0}^{\log_2 n-1} 2^k$

see CLRS Appendix A for the formula for geometric series

\[\sum_{k=0}^m x^k = 1 + x + x^2 + \cdots + x^m = \frac{x^{m+1}-1}{x-1}\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

Step 4: Count the total number of operations

\[\sum_{k=0}^{\log_2 n-1} 2^k = 1 + 2 + 2^2 + \cdots + 2^{\log_2 n-1}= \frac{2^{\log_2 n}-1}{2-1} = n - 1\]

\[\sum_{k=0}^m x^k = 1 + x + x^2 + \cdots + x^m = \frac{x^{m+1}-1}{x-1}\]

so linear search is $\Theta(n)$

Total number of operation is
- $\displaystyle\sum_{k=0}^{\log_2 n-1} 2^k + \Theta(n) = (n-1) + \Theta(n) = \Theta(n)$

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

assume:
- $n$ is a power of 3
- we are doing $\Theta(1)$ operation on each node

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

$1$

. . .

\[9\]

\[3\]

\[1\]

\[\Theta(3^{\log_3 n}) = \Theta(n)\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

there are $\log_3 n$ levels
we need to find the sum of
- $3^0 + 3^1 + 3^2 + \dots + 3^{\log_3 n-1} = \displaystyle\sum_{k=0}^{\log_3 n-1} 3^k$

the formula for geometric series:

\[\sum_{k=0}^m x^k = 1 + x + x^2 + \cdots + x^m = \frac{x^{m+1}-1}{x-1}\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

Step 4: Count the total number of operations

\[\sum_{k=0}^m x^k = 1 + x + x^2 + \cdots + x^m = \frac{x^{m+1}-1}{x-1}\]

\[\sum_{k=0}^{\log_3 n-1} x^k = 1 + 3 + 3^2 + \cdots + 3^{\log_3 n -1} = \frac{3^{\log_3 n}-1}{3-1} = \frac{n-1}{2}\]

so linear search is again $\Theta(n)$
but can you see why we need less than $n$ operations?
- (it's a mistake!)

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

$2$

. . .

\[9\]

\[3\]

\[1\]

\[\Theta(3^{\log_3 n}) = \Theta(n)\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

\[\sum_{k=0}^{\log_3 n-1} x^k = 1 + 3 + 3^2 + \cdots + 3^{\log_3 n -1} = \frac{3^{\log_3 n}-1}{3-1} = \frac{n-1}{2}\]

\[\sum_{k=0}^{\log_3 n-1} x^k = 2(1) + 2(3) + 2(3^2) + \cdots + 2(3^{\log_3 n -1}) = \frac{2(3^{\log_3 n}-1)}{3-1} =n-1\]

Linear Search

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

Total number of operation is
- $\displaystyle\sum_{k=0}^{\log_3 n-1} 3^k + \Theta(n) = (n-1) + \Theta(n) = \Theta(n)$

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

CLRS page 89-90

$T(n)$

\[T\left(\frac{n}{4}\right)\]

Step 1: Draw the recursion tree

we assume $n$ is a power of 4

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

$cn^2$

\[\frac{cn^2}{16}\]

\[\frac{cn^2}{256}\]

. . . . . . . . . . . . . . . . . . . . .

Step 2: Count the number of operations on each level

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

$cn^2$

\[\frac{cn^2}{16}\]

\[\frac{cn^2}{256}\]

. . . . . . . . . . . . . . . . . . . . .

\[\frac{cn^2}{256}\]

$cn^2$

\[\frac{3}{16}cn^2\]

\[\frac{9}{256}cn^2\]

???

Step 2: Count the number of operations on each level

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 3: Count the number of levels

on each level we divide the problem size by 4
the subproblem sizes:
- $n \quad\rightarrow \quad n/4 \quad\rightarrow\quad n/16 \quad\rightarrow\quad\cdots\quad\rightarrow\quad n/4^i$
the final level is when $n /4^i = 1$
- i.e. when $\quad 4^i = n \quad\rightarrow\quad i = \log_4 n$
level $i$ has $3^i$ nodes, so the final level has $3^{\log_4 n}$ nodes

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 3: Count the number of levels

$cn^2$

\[\frac{cn^2}{16}\]

\[\frac{cn^2}{256}\]

. . . . . . . . . . . . . . . . . . . . .

\[\frac{cn^2}{256}\]

$cn^2$

\[\frac{3}{16}cn^2\]

\[\frac{9}{256}cn^2\]

$\Theta(3^{\log_4 n})$

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

$cn^2$

\[\frac{3}{16}cn^2\]

\[\frac{9}{256}cn^2\]

$\Theta(3^{\log_4 n})$

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

$cn^2$

\[\frac{3}{16}cn^2\]

\[\frac{9}{256}cn^2\]

$\Theta(3^{\log_4 n})$

$+$

$+ \dots +$

\[cn^2 + \frac{3}{16}cn^2 + \left(\frac{3}{16}\right)^2cn^2+\cdots+ \left(\frac{3}{16}\right) ^{\log_4 n - 1}cn^2+ \Theta(3^{\log_4 n})\]

\[cn^2\sum_{i=0}^{\log_4n-1}\left(\frac{3}{16}\right)^i+ \Theta(3^{\log_4 n})\]

\[= \frac{(3/16)^{\log_4 n}-1}{(3/16)-1}cn^2+ \Theta(3^{\log_4 n})\]

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

\[cn^2\sum_{i=0}^{\log_4n-1}\left(\frac{3}{16}\right)^i+ \Theta(3^{\log_4 n})\]

\[= \frac{(3/16)^{\log_4 n}-1}{(3/16)-1}cn^2+ \Theta(3^{\log_4 n})\]

this is ugly ...
but here we can be a bit sloppy again:
- use geometric sum of an infinite series, with $|x| < 1$

\[\sum_{k=0}^\infty x^k = \frac{1}{1-x}\]

\[\sum_{k=0}^m x^k = 1 + x + x^2 + \cdots + x^m = \frac{x^{m+1}-1}{x-1}\]

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

\[\sum_{k=0}^n x^k <\sum_{k=0}^\infty x^k = \frac{1}{1-x}\]

if $|x| < 1$ and each term is positive:

\[\sum_{i=0}^{\log_4n-1}\left(\frac{3}{16}\right)^i cn^2+ \Theta(3^{\log_4 n})\]

\[< \sum_{i=0}^{\infty}\left(\frac{3}{16}\right)^i cn^2+ \Theta(3^{\log_4 n})\]

\[=\frac{1}{1-(3/16)} cn^2+ \Theta(3^{\log_4 n})\]

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

One more rule: $3^{\log_4 n} = n^{\log_4 3}$

\[\frac{1}{1-(3/16)} cn^2+ \Theta(3^{\log_4 n})\]

\[=\frac{1}{1-(3/16)} cn^2+ \Theta(n^{\log_4 3})\]

$\log_4 3$ is about 0.79248125, so we have

\[\frac{1}{1-(3/16)} cn^2+ \Theta(n^{0.79}) = O(n^2)\]

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

\[cn^2 + \frac{3}{16}cn^2 + \left(\frac{3}{16}\right)^2cn^2+\cdots+ \left(\frac{3}{16}\right) ^{\log_4 n - 1}cn^2+ \Theta(3^{\log_4 n})\]

\[\left(\frac{3}{16}\right)^{\log_4 n} cn^2 = \frac{3^{\log_4 n}}{16^{\log_4 n}}cn^2\]

\[\left(\frac{3}{16}\right)^{\log_4 n} cn^2\]

$16^{\log_4 n} = (4^2)^{\log_4 n} = (4^{\log_4 n})^2 = n^2$

\[\left(\frac{3}{16}\right)^{\log_4 n} cn^2 = \frac{3^{\log_4 n}}{n^2}cn^2 = c\cdot 3^{\log_4 n}\]

Textbook Example

6.1 - Recursion-Tree Method

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2&\text{otherwise} \end{cases}$

Step 4: Count the number of operations

\[cn^2 + \frac{3}{16}cn^2 + \left(\frac{3}{16}\right)^2cn^2+\cdots+ \left(\frac{3}{16}\right) ^{\log_4 n - 1}cn^2+ \Theta(3^{\log_4 n})\]

\[cn^2 + \frac{3}{16}cn^2 + \left(\frac{3}{16}\right)^2cn^2+\cdots+ \left(\frac{3}{16}\right) ^{\log_4 n - 1}cn^2+ \left(\frac{3}{16}\right)^{\log_4 n}cn^2 \]

\[cn^2\sum_{i=0}^{\log_4n-1}\left(\frac{3}{16}\right)^i+ \Theta(3^{\log_4 n})\]

\[cn^2\sum_{i=0}^{\log_4n}\left(\frac{3}{16}\right)^i\]

The End

6.1 - Recursion-Tree Method

we will do more in the workshop and there will be a submission question as well for this week
it is going to show up in the exam, but the formulas for geometric sums will be provided

6.1 - Recursion-Tree Method

I told you that recursion-tree will be in the exam

I just want a pass