COMP3010 - 6.3 - Master Method

COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

6.3 - Master Method

Master Theorem

6.3 - Master Method

The Master Theorem is a formula that describes the running time of some divide-and-conquer algorithm
The algorithm must be of the form
- $T(n) = aT(n/b) + f(n)$ with $a \ge 1$, $b > 1$, and $f(n) > 0$
First presented by Bentley, Haken, and Saxe in 1980
- the term 'master theorem' was popularised by CLRS
- the proof can be found in CLRS, but we are not going to go through it

Master Theorem

6.3 - Master Method

Given a recurrence of the form

$T(n) = aT(n/b) + f(n) $

if $f(n) = O(n^{\log_b a-\epsilon})$ for some constant $\epsilon > 0$ then
- $T(n) =\Theta(n^{\log_b a})$
if $f(n) = \Theta(n^{\log_b a})$ then
- $T(n) = \Theta(n^{\log_b a}\log n)$
if $f(n) = \Omega(n^{\log_b a + \epsilon})$ for some constant $\epsilon > 0$, and if $a f(n/b) \le cf(n)$ for some $c < 1$ then
- $T(n) = \Theta(f(n))$

Master Theorem

6.3 - Master Method

$T(n) = aT(n/b) + f(n) $

$f(n) = O(n^{\log_b a-\varepsilon})$

$f(n) = \Theta(n^{\log_b a})$

$f(n) = \Omega(n^{\log_b a + \varepsilon})$

$T(n) = aT(n/b) + cn^k $

which formula you use depends on $f(n)$

notice that $f(n)$ is polynomial in all three cases, so we can write $f(n) = cn^k$

$T(n) = \Theta(n^{\log_ba})$

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta({f(n)})$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + f(n) $

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$f(n) = O(n^{\log_b a-\varepsilon})$

for some constant $\varepsilon>0$

this means $k = \log_b a - \varepsilon$

$ b^k = a / b^\varepsilon$

$ b^k < a $

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}$

$a = 2$

$b = 2$

$k = 0$

is $ b^k < a $?

so $T(n) = \Theta(n^{\log_2 2}) = \Theta(n)$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}$

$a = 3$

$b = 3$

$k = 0$

is $ b^k < a $?

so $T(n) = \Theta(n^{\log_3 3}) = \Theta(n)$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

Master Theorem (simplification)

6.3 - Master Method

\[T(n)=\begin{cases}\Theta(n^{\log_b a}),&\text{if $f(n) = O(n^{\log_b a - \epsilon})$ for $\epsilon > 0$}\\\Theta(n^{\log_ba}\log n),&\text{if $f(n)=\Theta(n^{\log_b a})$}\\\Theta(f(n)),&\text{if $f(n) = \Omega(n^{\log_ba+\epsilon})$ for $\epsilon > 0$}\end{cases}\]

\[T(n)=\begin{cases}\Theta(n^{\log_ba}),&\text{if $a > b^k$ }\\\Theta(n^{\log_ba}\log n),&\text{if $a = b^k$}\\\Theta(n^k),&\text{if $a < b^k$}\end{cases}\]

\[T(n) = aT(n/b) + cn^k \]

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}$

$a = 2$

$b = 2$

$k = 1$

$ b^k = a $?

so $T(n) = \Theta(n^{\log_2 2}\log n) = \Theta(n\log n)$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}$

Master Theorem (simplification)

6.3 - Master Method

$T(n) = aT(n/b) + cn^k $

$T(n) = \Theta(n^{\log_ba})$

$ b^k < a $

and

then

$T(n) = \Theta(n^{\log_ba}\log n)$

$T(n) = \Theta(n^k)$

$T(n) = aT(n/b) + cn^k $

$ b^k = a $

and

then

$T(n) = aT(n/b) + cn^k $

$ b^k > a $

and

then

$T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}$

$a = 3$

$b = 4$

$k = 2$

$ b^k > a $?

so $T(n) = \Theta(n^2)$

Problem description

6.3 - Master Method

Why didn't we just do Master method?
- there are some situations where you cannot apply the Master method
- plus, it is still good to be able to do the other methods
In the exam, the formula will be provided (both version), but be mindful that if you're asked to use the recursion-tree + substitution method, then you must
However, you can use master method to 'cheat', in that, you can know what the answer is