COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
6.3 - Master Method
Master Theorem
6.3 - Master Method
- The Master Theorem is a formula that describes the running time of some divide-and-conquer algorithm
- The algorithm must be of the form
- \(T(n) = aT(n/b) + f(n)\) with \(a \ge 1\), \(b > 1\), and \(f(n) > 0\)
- First presented by Bentley, Haken, and Saxe in 1980
- the term 'master theorem' was popularised by CLRS
- the proof can be found in CLRS, but we are not going to go through it
Master Theorem
6.3 - Master Method
Given a recurrence of the form
\(T(n) = aT(n/b) + f(n) \)
- if \(f(n) = O(n^{\log_b a-\epsilon})\) for some constant \(\epsilon > 0\) then
- \(T(n) =\Theta(n^{\log_b a})\)
- if \(f(n) = \Theta(n^{\log_b a})\) then
- \(T(n) = \Theta(n^{\log_b a}\log n)\)
- if \(f(n) = \Omega(n^{\log_b a + \epsilon})\) for some constant \(\epsilon > 0\), and if \(a f(n/b) \le cf(n)\) for some \(c < 1\) then
- \(T(n) = \Theta(f(n))\)
Master Theorem
6.3 - Master Method
\(T(n) = aT(n/b) + f(n) \)
\(f(n) = O(n^{\log_b a-\varepsilon})\)
\(f(n) = \Theta(n^{\log_b a})\)
\(f(n) = \Omega(n^{\log_b a + \varepsilon})\)
\(T(n) = aT(n/b) + cn^k \)
- which formula you use depends on \(f(n)\)
- notice that \(f(n)\) is polynomial in all three cases, so we can write \(f(n) = cn^k\)
\(T(n) = \Theta(n^{\log_ba})\)
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta({f(n)})\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + f(n) \)
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\(f(n) = O(n^{\log_b a-\varepsilon})\)
for some constant \(\varepsilon>0\)
- this means \(k = \log_b a - \varepsilon\)
\( b^k = a / b^\varepsilon\)
\( b^k < a \)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}\)
\(a = 2\)
\(b = 2\)
\(k = 0\)
is \( b^k < a \)?
so \(T(n) = \Theta(n^{\log_2 2}) = \Theta(n)\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}\)
\(a = 3\)
\(b = 3\)
\(k = 0\)
is \( b^k < a \)?
so \(T(n) = \Theta(n^{\log_3 3}) = \Theta(n)\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
Master Theorem (simplification)
6.3 - Master Method
\[T(n)=\begin{cases}\Theta(n^{\log_b a}),&\text{if $f(n) = O(n^{\log_b a - \epsilon})$ for $\epsilon > 0$}\\\Theta(n^{\log_ba}\log n),&\text{if $f(n)=\Theta(n^{\log_b a})$}\\\Theta(f(n)),&\text{if $f(n) = \Omega(n^{\log_ba+\epsilon})$ for $\epsilon > 0$}\end{cases}\]
\[T(n)=\begin{cases}\Theta(n^{\log_ba}),&\text{if $a > b^k$ }\\\Theta(n^{\log_ba}\log n),&\text{if $a = b^k$}\\\Theta(n^k),&\text{if $a < b^k$}\end{cases}\]
\[T(n) = aT(n/b) + cn^k \]
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}\)
\(a = 2\)
\(b = 2\)
\(k = 1\)
\( b^k = a \)?
so \(T(n) = \Theta(n^{\log_2 2}\log n) = \Theta(n\log n)\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}\)
Master Theorem (simplification)
6.3 - Master Method
\(T(n) = aT(n/b) + cn^k \)
\(T(n) = \Theta(n^{\log_ba})\)
\( b^k < a \)
and
then
\(T(n) = \Theta(n^{\log_ba}\log n)\)
\(T(n) = \Theta(n^k)\)
\(T(n) = aT(n/b) + cn^k \)
\( b^k = a \)
and
then
\(T(n) = aT(n/b) + cn^k \)
\( b^k > a \)
and
then
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}\)
\(a = 3\)
\(b = 4\)
\(k = 2\)
\( b^k > a \)?
so \(T(n) = \Theta(n^2)\)
Problem description
6.3 - Master Method
- Why didn't we just do Master method?
- there are some situations where you cannot apply the Master method
- plus, it is still good to be able to do the other methods
- In the exam, the formula will be provided (both version), but be mindful that if you're asked to use the recursion-tree + substitution method, then you must
- However, you can use master method to 'cheat', in that, you can know what the answer is
COMP3010 - 6.3 - Master Method
By Daniel Sutantyo
COMP3010 - 6.3 - Master Method
The master method formula
- 130
