COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
4.0 - Dynamic Programming and Overlapping Subproblems
4.0 - DP and Overlapping Subproblems
This week
- Overlapping subproblems
- Optimal substructure
- Examples
- See CLRS Chapter 15, as we're going to follow it pretty closely
4.0 - DP and Overlapping Subproblems
From last week
- We discussed brute-force method last week:
- problems and subproblems, search tree, and backtracking
[ 50 , 2 , 18 , 11 , 9 , 23 , 5 , 10 , 30 , 6 , 17 ] L = 27
[ 2 , 18 , 11 , ... , 17 ] L = -23
[ 2 , 18 , 11 , ... , 17 ] L = 27
[18 , 11 , ... , 17 ] L = -25
pick 2
don't pick 2
...
...
...
...
...
...
[ 18 , 11, ... , 17] L = -23
[18 , 11, ... , 17 ] L = 25
[ 18 , 11 , ... , 17 ] L = 27
...
...
pick 50
pick 2
don't pick 50
don't pick 2
4.0 - DP and Overlapping Subproblems
Quick Intro
4.0 - DP and Overlapping Subproblems
Quick Intro
4.0 - DP and Overlapping Subproblems
Fibonacci
- Let's discuss Fibonacci ... again ... since this is the obligatory dynamic programming example
public static int fib(int n) {
if (n <= 2)
return 1;
else return fib(n-1) + fib(n-2);
}1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...
4.0 - DP and Overlapping Subproblems
Fibonacci
public static int fib(int n) {
if (n <= 2)
return 1;
else return fib(n-1) + fib(n-2);
}fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(7)
fib(6)
fib(6)
fib(6)
fib(5)
fib(6)
fib(5)
fib(5)
fib(4)
fib(7)
fib(6)
fib(5)
...
...
...
...
4.0 - DP and Overlapping Subproblems
Fibonacci
| n | fib(n) | no. of recursive calls | |
|---|---|---|---|
| 10 | 55 | 109 | 123 |
| 20 | 6,765 | 13,529 | 15,121 |
| 30 | 832,040 | 1,664,079 | 1,859,326 |
| 40 | 102,334,155 | 204,668,309 | 228,633,935 |
| 50 | 12,586,269,025 | 25,172,538,049 | 28,114,208,661 |
| 60 | ??? | ??? | 3,457,092,791,653 |
1.62
public static int fib(int n) {
if (n <= 2)
return 1;
else return fib(n-1) + fib(n-2);
}n
4.0 - DP and Overlapping Subproblems
Fibonacci - Top Down DP (memoisation)
public static long table[] = new long[1000];
public static int fib(int n) {
if (table[n] != 0)
return table[n];
else {
table[n] = fib2(n-1) + fib2(n-2);
return table[n];
}
}fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(6)
fib(6)
fib(5)
fib(7)
...
...
| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
4.0 - DP and Overlapping Subproblems
Fibonacci - Bottom-Up DP (tabulation)
| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
public static long fib(int n) {
long a = 0;
long b = 1;
for (int i = 0; i < n; i++) {
b = a + b;
a = b - a;
}
return a;
}4.0 - DP and Overlapping Subproblems
Fibonacci
| 1 | 1 | 2 | 3 | 5 | 8 | 13 | 21 | 34 | 55 |
|---|---|---|---|---|---|---|---|---|---|
| 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 |
fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(7)
fib(6)
fib(6)
fib(6)
fib(5)
fib(6)
fib(5)
fib(5)
fib(4)
fib(7)
fib(6)
fib(5)
...
...
...
...
4.0 - DP and Overlapping Subproblems
Fibonacci
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
- Given two sequences X and Y, find their longest common subsequence
GTAACCATGACTGACG
ATTCGATCGATCATCGT
GTAACCATGACTGACG
ATTCGATCGATCATCGT
ACATGATACG
- You can find the code under Week 4 folder on the COMP3010 GitHub
- You can also see it here: https://www.cs.usfca.edu/~galles/visualization/DPLCS.html
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
- Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
T
A
G
T
T
G
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
- Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
T
A
G
T
T
G
T
A
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
- Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
T
A
T
T
A
G
T
T
G
T
A
T
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T
A
T
A
T
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T
A
T
A
T
A
T
A
T
T
T
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
\(X_1X_2X_3\dots X_m\)
\(Y_1Y_2Y_3\dots Y_n\)
\(X_1X_2X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_1Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_1X_2X_3\dots X_m\)
\(Y_3\dots Y_n\)
\(X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_3\dots Y_n\)
\(X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
\(X_1X_2X_3\dots X_m\)
\(Y_1Y_2Y_3\dots Y_n\)
\(X_1X_2X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_1Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_1X_2X_3\dots X_m\)
\(Y_3\dots Y_n\)
\(X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
\(X_2X_3\dots X_m\)
\(Y_3\dots Y_n\)
\(X_3\dots X_m\)
\(Y_2Y_3\dots Y_n\)
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G
0
0
0
0
0
0
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G
0
0
0
0
0
0
1
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 0 | ||||||
| AGTTG | 0 | ||||||
| GTTG | 0 | ||||||
| TTG | 0 | ||||||
| TG | 0 | ||||||
| G | 0 | ||||||
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 1 | 0 | |||||
| AGTTG | 1 | 0 | |||||
| GTTG | 1 | 0 | |||||
| TTG | 1 | 0 | |||||
| TG | 1 | 0 | |||||
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 2 | 1 | 0 | ||||
| AGTTG | 2 | 1 | 0 | ||||
| GTTG | 1 | 1 | 0 | ||||
| TTG | 1 | 1 | 0 | ||||
| TG | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 2 | 2 | 1 | 0 | |||
| AGTTG | 2 | 2 | 1 | 0 | |||
| GTTG | 1 | 1 | 1 | 0 | |||
| TTG | 2 | 2 | 1 | 1 | 1 | 1 | 0 |
| TG | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 2 | 2 | 2 | 1 | 0 | ||
| AGTTG | 2 | 2 | 2 | 1 | 0 | ||
| GTTG | 3 | 2 | 1 | 1 | 1 | 1 | 0 |
| TTG | 2 | 2 | 1 | 1 | 1 | 1 | 0 |
| TG | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 3 | 2 | 2 | 2 | 1 | 0 | |
| AGTTG | 3 | 2 | 2 | 2 | 2 | 1 | 0 |
| GTTG | 3 | 2 | 1 | 1 | 1 | 1 | 0 |
| TTG | 2 | 2 | 1 | 1 | 1 | 1 | 0 |
| TG | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
| GTACAT | TACAT | ACAT | CAT | AT | T | ||
| TAGTTG | 3 | 3 | 2 | 2 | 2 | 1 | 0 |
| AGTTG | 3 | 2 | 2 | 2 | 2 | 1 | 0 |
| GTTG | 3 | 2 | 1 | 1 | 1 | 1 | 0 |
| TTG | 2 | 2 | 1 | 1 | 1 | 1 | 0 |
| TG | 1 | 1 | 1 | 1 | 1 | 1 | 0 |
| G | 1 | 0 | 0 | 0 | 0 | 0 | 0 |
| 0 | 0 | 0 | 0 | 0 | 0 | 0 |
4.0 - DP and Overlapping Subproblems
Longest Common Subsequence (LCSS)
recursion
top-down
DP
bottom-up
DP
???