COMP3010: Algorithm Theory and Design

Daniel Sutantyo,  Department of Computing, Macquarie University

6.3 - Master Method

Master Theorem

6.3 - Master Method

  • The Master Theorem is a formula that describes the running time of some divide-and-conquer algorithm
  • The algorithm must be of the form
    • \(T(n) = aT(n/b) + f(n)\) with \(a \ge 1\), \(b > 1\), and \(f(n) > 0\)
  • First presented by Bentley, Haken, and Saxe in 1980
    • the term 'master theorem' was popularised by CLRS
    • the proof can be found in CLRS, but we are not going to go through it

Master Theorem

6.3 - Master Method

Given a recurrence of the form

\(T(n) = aT(n/b) + f(n) \)

  • if \(f(n) = O(n^{\log_b a-\epsilon})\) for some constant \(\epsilon > 0\) then
    • \(T(n) =\Theta(n^{\log_b a})\)
  • if \(f(n) = \Theta(n^{\log_b a})\) then
    • \(T(n) = \Theta(n^{\log_b a}\log n)\)
  • if \(f(n) = \Omega(n^{\log_b a + \epsilon})\) for some constant \(\epsilon > 0\), and if \(a f(n/b) \le cf(n)\) for some \(c < 1\) then
    • \(T(n) = \Theta(f(n))\)

Master Theorem

6.3 - Master Method

\(T(n) = aT(n/b) + f(n) \)

\(f(n) = O(n^{\log_b a-\varepsilon})\)

\(f(n) = \Theta(n^{\log_b a})\)

\(f(n) = \Omega(n^{\log_b a + \varepsilon})\)

\(T(n) = aT(n/b) + cn^k \)

  • which formula you use depends on \(f(n)\)
  • notice that \(f(n)\) is polynomial in all three cases, so we can write \(f(n) = cn^k\)

\(T(n) = \Theta(n^{\log_ba})\)

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta({f(n)})\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + f(n) \)

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\(f(n) = O(n^{\log_b a-\varepsilon})\)

for some constant \(\varepsilon>0\)

  • this means \(k = \log_b a - \varepsilon\)

\( b^k =  a / b^\varepsilon\)

\( b^k < a \)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}\)

\(a = 2\)

\(b = 2\)

\(k = 0\)

is \( b^k < a \)?

so \(T(n) = \Theta(n^{\log_2 2}) = \Theta(n)\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}\)

\(a = 3\)

\(b = 3\)

\(k = 0\)

is \( b^k < a \)?

so \(T(n) = \Theta(n^{\log_3 3}) = \Theta(n)\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

Master Theorem (simplification)

6.3 - Master Method

\[T(n)=\begin{cases}​\Theta(n^{\log_b a}),&\text{if $f(n) = O(n^{\log_b a - \epsilon})$ for $\epsilon > 0$}\\\Theta(n^{\log_ba}\log n),&\text{if $f(n)=\Theta(n^{\log_b a})$}\\\Theta(f(n)),&\text{if $f(n) = \Omega(n^{\log_ba+\epsilon})$ for $\epsilon > 0$}\end{cases}\]

\[T(n)=\begin{cases}​\Theta(n^{\log_ba}),&\text{if $a > b^k$ }\\\Theta(n^{\log_ba}\log n),&\text{if $a = b^k$}\\\Theta(n^k),&\text{if $a < b^k$}\end{cases}\]

\[T(n) = aT(n/b) + cn^k \]

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}\)

\(a = 2\)

\(b = 2\)

\(k = 1\)

\( b^k = a \)?

so \(T(n) = \Theta(n^{\log_2 2}\log n) = \Theta(n\log n)\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}\)

Master Theorem (simplification)

6.3 - Master Method

\(T(n) = aT(n/b) + cn^k \)

\(T(n) = \Theta(n^{\log_ba})\)

\( b^k < a \)

and

then

\(T(n) = \Theta(n^{\log_ba}\log n)\)

\(T(n) = \Theta(n^k)\)

\(T(n) = aT(n/b) + cn^k \)

\( b^k = a \)

and

then

\(T(n) = aT(n/b) + cn^k \)

\( b^k > a \)

and

then

\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/4) + cn^2 &\text{otherwise} \end{cases}\)

\(a = 3\)

\(b = 4\)

\(k = 2\)

\( b^k > a \)?

so \(T(n) = \Theta(n^2)\)

Problem description

6.3 - Master Method

  • Why didn't we just do Master method?
    • there are some situations where you cannot apply the Master method
    • plus, it is still good to be able to do the other methods
  • In the exam, the formula will be provided (both version), but be mindful that if you're asked to use the recursion-tree + substitution method, then you must
  • However, you can use master method to 'cheat', in that, you can know what the answer is