COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
6.0 - Divide and Conquer - Recurrence Equation
The topics so far ...
6.0 - Divide and Conquer
- Complexity
- Correctness
- Brute force
- Dynamic Programming
- Greedy Algorithm
Reference for this week: CLRS Chapter 4
Divide and Conquer
Optimal Substructure
The topics so far ...
6.0 - Divide and Conquer
Divide and Conquer
Optimal Substructure
- Complexity
- Correctness
- Brute force
- Dynamic Programming
- Greedy Algorithm
Reference for this week: CLRS Chapter 4
6.0 - Divide and Conquer
Complexity of Divide and Conquer Algorithm
- How did you work out the complexity of divide and conquer algorithm?
- in the first week, we mostly talked about the complexity notations, i.e. the \(O\), \(\Theta\), and \(\Omega\) notations
- it was assumed that you know the complexity of some simple algorithms like binary search, mergesort, and quicksort
- how did you work it out?
6.0 - Divide and Conquer
Mergesort
- number of levels: \(\log n\)
- number of operations on each level : \(n\)
- total number of operations: \(n\log n\)
13
4
19
11
32
22
7
32
22
7
32
22
7
13
4
19
11
13
4
19
11
6.0 - Divide and Conquer
Binary Search
- number of levels: \(\log n\)
- number of operations on each level : \(1\)
- total number of operations: \(\log n\)
13
4
19
11
32
7
22
13
19
32
22
13
19
13
6.0 - Divide and Conquer
Recursion Tree
6.0 - Divide and Conquer
Recursion Tree - Mergesort
\(T(n) = 2T(n/2) + n\)
the number of operations:
\(T(1) = 1\)
6.0 - Divide and Conquer
Recursion Tree - Binary Search
\(T(n) = T(n/2) + 1\)
the number of operations:
\(T(1) = 1\)
Recurrence Equation
6.0 - Divide and Conquer
- We characterise the running times of divide-and-conquer algorithms using a recurrence equation (or just recurrence)
- Formally, if \(T(n)\) is the running time of the algorithm:
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- where
- \(D(n)\) is the cost of dividing the problem into subproblems
- \(C(n)\) is the cost of combining solutions of the subproblems to create a solution for the problem
6.0 - Divide and Conquer
Mergesort
13
4
19
11
32
22
7
- number of levels: \(\log n\)
- number of operations on each level : \(n\)
- total number of operations: \(n\log n\)
- recurrence equation
\(T(n) = 2T(n/2) + n\)
6.0 - Divide and Conquer
Mergesort
13
4
19
11
32
22
7
32
22
7
13
4
19
11
- number of levels: \(\log n\)
- number of operations on each level : \(n\)
- total number of operations: \(n\log n\)
- recurrence equation
\(T(n) = 2T(n/2) + n\)
6.0 - Divide and Conquer
Mergesort
13
4
19
11
32
22
7
32
22
7
32
22
7
13
4
19
11
13
4
19
11
- number of levels: \(\log n\)
- number of operations on each level : \(n\)
- total number of operations: \(n\log n\)
- recurrence equation
\(T(n) = 2T(n/2) + n\)
6.0 - Divide and Conquer
Mergesort
19
32
7
22
11
13
4
13
4
19
11
32
7
22
22
7
32
11
19
4
13
\(T(n) = 2T(n/2) + n\)
- number of levels: \(\log n\)
- number of operations on each level : \(n\) (cost of combining)
- total number of operations: \(n\log n\)
- recurrence equation
6.0 - Divide and Conquer
Mergesort
19
32
7
22
11
13
4
13
4
19
11
32
7
22
22
7
32
11
19
4
13
\(T(n) = 2T(n/2) + n\)
- number of levels: \(\log n\)
- number of operations on each level : \(n\) (cost of combining)
- total number of operations: \(n\log n\)
- recurrence equation
6.0 - Divide and Conquer
Mergesort
- number of levels: \(\log n\)
- number of operations on each level : \(n\) (cost of combining)
- total number of operations: \(n\log n\)
- recurrence equation
19
32
7
22
11
13
4
13
4
19
11
32
7
22
22
7
32
11
19
4
13
\(T(n) = 2T(n/2) + n\)
Recurrence Equation
6.0 - Divide and Conquer
- We characterise the running times of divide-and-conquer algorithms using a recurrence equation (or just recurrence)
- Formally, if \(T(n)\) is the running time of the algorithm:
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- where
- \(D(n)\) is the cost of dividing the problem into subproblems
- \(C(n)\) is the cost of combining solutions of the subproblems to create a solution for the problem
6.0 - Divide and Conquer
Binary Search
- number of levels: \(\log n\)
- number of operations on each level : \(1\) (cost of dividing)
- total number of operations: \(\log n\)
- recurrence equation:
13
4
19
11
32
7
22
13
19
32
22
13
19
13
\(T(n) = T(n/2) + 1\)
... or is it?
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- we don't have to use equality because sometimes we can't say for sure, so it is okay to have
Recurrence Equation
6.0 - Divide and Conquer
\(T(n) \le aT(n/b) + f(n)\)
\(T(n) \ge aT(n/b) + f(n)\)
or
- it should be okay to just use equality most of the time in this unit, we don't have to be too precise
- we also ignore the details on boundary conditions and non-exact division
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
Recurrence Equation
6.0 - Divide and Conquer
- we also ignore the details on boundary conditions and non-exact division
- e.g. merge sort - \(T(n) = 2T(n/2) + n\)
- what if \(n\) is odd?
- shouldn't we have something like \(T(n) = T(\lfloor{n/2}\rfloor) + T(\lceil{n/2}\rceil + n \)
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
Recurrence Equation
6.0 - Divide and Conquer
- we'll keep it simple:
\(n\)
\[\frac{n}{2}\]
\[\frac{n}{2}\]
\(n\)
\[\frac{n}{3}\]
\[\frac{n}{3}\]
\[\frac{n}{3}\]
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- how does this relate to the complexity notation?
- \(T(n)\) is the running time of the algorithm
- the complexity notation is the bound for the running time of the algorithm
- e.g.
- \(T(n) = \frac{1}{2}n^2 + \frac{1}{2}n\)
- \(T(n) = \Theta(n^2)\)
- \(T(n) \le aT(n/b) + f(n)\) \(\rightarrow\) \(T(n) = O(g(n))\)
- \(T(n) \ge aT(n/b) + f(n)\) \(\rightarrow\) \(T(n) = \Omega(g(n))\)
Recurrence Equation
6.0 - Divide and Conquer
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- you can also use complexity notation to describe \(D(n)\) and/or \(C(n)\):
- e.g.
- \(T(n) = 2T(n/2) + n\)
- \(T(n) = 2T(n/2) + \Theta(n)\) or \(T(n) = 2T(n/2) + O(n)\)
- e.g.
- remember our goal is to find the asymptotic notation for \(T(n)\), that is eventually we want to be able to say that
- \(T(n) = O(g(n))\) or \(T(n) = \Omega(g(n))\) or \(T(n) = \Theta(g(n))\)
Recurrence Equation
6.0 - Divide and Conquer
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le c$}\\ aT(n/b) + D(n) + C(n) &\text{otherwise} \end{cases}\)
- summary:
- you can use inequality as well \(<, \le, =, \ge, >)\)
- don't worry about floor or ceiling when it comes to division
- don't forget that we want an asymptotic bound out of this (we're not trying to be exact)
Recurrence Equation
6.0 - Divide and Conquer
Example - Linear Search
6.0 - Divide and Conquer
public static int linearSearch(int[] a, int i, int j, int target){
if (i > j)
return -1;
if (a[i] == target)
return i;
return linearSearch(a,i+1,j,target);
}public static int linearSearch(int[] a, int i, int j, int target) {
if (i > j)
return -1;
int m = (i+j)/2;
if (a[m] == target)
return m;
return Math.max(linearSearch(a,i,m-1,target), linearSearch(a,m+1,j,target));
}Example - Linear Search
6.0 - Divide and Conquer
public static int linearSearch(int[] a, int i, int j, int target) {
if (i > j)
return -1;
int m = (i+j)/2;
if (a[m] == target)
return m;
return Math.max(linearSearch(a,i,m-1,target), linearSearch(a,m+1,j,target));
}linearSearch(a[], 0, 8, target)
linearSearch(a[], 0, 3, target)
linearSearch(a[], 5, 8, target)
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}\)
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(1)&\text{otherwise} \end{cases}\)
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 3T(n/3) + \Theta(1)&\text{otherwise} \end{cases}\)
- If we divide the array into two parts:
- If we divide the array into three parts:
Example - Linear Search
6.0 - Divide and Conquer
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
22
7
19
4
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
22
7
19
4
Example - Quicksort
6.0 - Divide and Conquer
22
7
32
11
19
4
13
22
7
32
11
19
4
22
7
19
4
Example - Quicksort
6.0 - Divide and Conquer
13
32
11
22
7
19
4
Example - Quicksort
6.0 - Divide and Conquer
13
32
11
22
7
19
4
13
32
11
22
7
19
13
32
11
22
7
19
Example - Quicksort
6.0 - Divide and Conquer
13
32
11
22
7
19
4
13
32
11
22
7
19
13
32
11
22
7
19
13
32
11
22
19
13
32
11
22
19
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ 2T(n/2) + \Theta(n)&\text{otherwise} \end{cases}\)
\(T(n) = \begin{cases} \Theta(1) &\text{if $n \le 1$}\\ T(n-1) + \Theta(n)&\text{otherwise} \end{cases}\)
- Average case:
- Worst case:
Example - Quicksort
6.0 - Divide and Conquer
- We have discussed how to derive the recursive relation for \(T(n)\), and we mentioned that this is going to lead to an asymptotic bound for the running time of the algorithm, but we haven't actually showed how this can be done
- We are going to discuss three methods:
- recursion-tree method
- substitution method
- the master method
What's next
6.0 - Divide and Conquer
COMP3010 - 6.0 - Divide and Conquer
By Daniel Sutantyo
COMP3010 - 6.0 - Divide and Conquer
Recurrence equation for divide and conquer algorithm
- 152
