COMP3010: Algorithm Theory and Design

Daniel Sutantyo,  Department of Computing, Macquarie University

4.0 - Dynamic Programming and Overlapping Subproblems

4.0 - DP and Overlapping Subproblems

This week

  • Overlapping subproblems
  • Optimal substructure
  • Examples
  • See CLRS Chapter 15, as we're going to follow it pretty closely

4.0 - DP and Overlapping Subproblems

From last week

  • We discussed brute-force method last week:
    • problems and subproblems, search tree, and backtracking

[ 50 , 2 , 18 , 11 , 9 , 23 , 5 , 10 , 30 , 6 , 17 ]   L = 27

[ 2 , 18 , 11 , ... , 17 ]   L = -23

[ 2 , 18 , 11 , ... , 17 ]   L = 27

[18 , 11 , ... , 17 ] L = -25

pick 2

don't pick 2

...

...

...

...

...

...

[ 18 , 11, ... , 17] L = -23

[18 , 11, ... , 17 ] L = 25

[ 18 , 11 , ... , 17 ] L = 27

...

...

pick 50

pick 2

don't pick 50

don't pick 2

4.0 - DP and Overlapping Subproblems

Quick Intro

4.0 - DP and Overlapping Subproblems

Quick Intro

4.0 - DP and Overlapping Subproblems

Fibonacci

  • Let's discuss Fibonacci ... again ... since this is the obligatory dynamic programming example
public static int fib(int n) {
  if (n <= 2)
    return 1;
  else return fib(n-1) + fib(n-2);
}

1, 1, 2, 3, 5, 8, 13, 21, 34, 55, 89, ...

4.0 - DP and Overlapping Subproblems

Fibonacci

public static int fib(int n) {
  if (n <= 2)
    return 1;
  else return fib(n-1) + fib(n-2);
}
fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(7)
fib(6)
fib(6)
fib(6)
fib(5)
fib(6)
fib(5)
fib(5)
fib(4)
fib(7)
fib(6)
fib(5)
...
...
...
...

4.0 - DP and Overlapping Subproblems

Fibonacci

n fib(n) no. of recursive calls
10 55 109 123
20 6,765 13,529 15,121
30 832,040 1,664,079 1,859,326
40 102,334,155 204,668,309 228,633,935
50 12,586,269,025 25,172,538,049 28,114,208,661
60 ??? ??? 3,457,092,791,653

1.62

public static int fib(int n) {
  if (n <= 2)
    return 1;
  else return fib(n-1) + fib(n-2);
}

n

4.0 - DP and Overlapping Subproblems

Fibonacci - Top Down DP (memoisation)

public static long table[] = new long[1000];
public static int fib(int n) {
  if (table[n] != 0)
    return table[n];
  else {
    table[n] = fib2(n-1) + fib2(n-2);
    return table[n];
  }
}
fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(6)
fib(6)
fib(5)
fib(7)
...
...
1 1 2 3 5 8 13 21 34 55
1 2 3 4 5 6 7 8 9 10

4.0 - DP and Overlapping Subproblems

Fibonacci - Bottom-Up DP (tabulation)

1 1 2 3 5 8 13 21 34 55
1 2 3 4 5 6 7 8 9 10
public static long fib(int n) {
  long a = 0;
  long b = 1;
  for (int i = 0; i < n; i++) {
    b = a + b;
    a = b - a;
  }
  return a;
}

4.0 - DP and Overlapping Subproblems

Fibonacci

1 1 2 3 5 8 13 21 34 55
1 2 3 4 5 6 7 8 9 10
fib(10)
fib(9)
fib(8)
fib(8)
fib(7)
fib(7)
fib(6)
fib(6)
fib(6)
fib(5)
fib(6)
fib(5)
fib(5)
fib(4)
fib(7)
fib(6)
fib(5)
...
...
...
...

4.0 - DP and Overlapping Subproblems

Fibonacci

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

  • Given two sequences X and Y, find their longest common subsequence
GTAACCATGACTGACG
ATTCGATCGATCATCGT
GTAACCATGACTGACG
ATTCGATCGATCATCGT
ACATGATACG

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

  • Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
T
A
G
T
T
G

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

  • Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
T
A
G
T
T
G
T
A

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

  • Given two sequences X and Y, find their longest common subsequence
G
T
A
C
A
T
T
A
G
T
T
G
T
A
T
T
A
G
T
T
G
T
A
T

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T
A
T
A
T

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
A
G
T
T
G
T
A
C
A
T
T
A
G
T
T
G
G
T
A
C
A
T
G
T
T
G
T
A
C
A
T
A
G
T
T
G
A
C
A
T
A
G
T
T
G
G
T
G
T
A
T
A
T
A
T
A
T
T
T

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

\(X_1X_2X_3\dots X_m\)

\(Y_1Y_2Y_3\dots Y_n\)

\(X_1X_2X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_1Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_1X_2X_3\dots X_m\)

\(Y_3\dots Y_n\)

\(X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_3\dots Y_n\)

\(X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

\(X_1X_2X_3\dots X_m\)

\(Y_1Y_2Y_3\dots Y_n\)

\(X_1X_2X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_1Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_1X_2X_3\dots X_m\)

\(Y_3\dots Y_n\)

\(X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

\(X_2X_3\dots X_m\)

\(Y_3\dots Y_n\)

\(X_3\dots X_m\)

\(Y_2Y_3\dots Y_n\)

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G

0

0

0

0

0

0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

A
T
T
G
T
T
G
T
G
A
T
G
T
G
A
T
T
G

0

0

0

0

0

0

1

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 0
AGTTG 0
GTTG 0
TTG 0
TG 0
G 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 1 0
AGTTG 1 0
GTTG 1 0
TTG 1 0
TG 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 2 1 0
AGTTG 2 1 0
GTTG 1 1 0
TTG 1 1 0
TG 1 1 1 1 1 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 2 2 1 0
AGTTG 2 2 1 0
GTTG 1 1 1 0
TTG 2 2 1 1 1 1 0
TG 1 1 1 1 1 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 2 2 2 1 0
AGTTG 2 2 2 1 0
GTTG 3 2 1 1 1 1 0
TTG 2 2 1 1 1 1 0
TG 1 1 1 1 1 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 3 2 2 2 1 0
AGTTG 3 2 2 2 2 1 0
GTTG 3 2 1 1 1 1 0
TTG 2 2 1 1 1 1 0
TG 1 1 1 1 1 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

GTACAT TACAT ACAT CAT AT T
TAGTTG 3 3 2 2 2 1 0
AGTTG 3 2 2 2 2 1 0
GTTG 3 2 1 1 1 1 0
TTG 2 2 1 1 1 1 0
TG 1 1 1 1 1 1 0
G 1 0 0 0 0 0 0
0 0 0 0 0 0 0

4.0 - DP and Overlapping Subproblems

Longest Common Subsequence (LCSS)

recursion

top-down

DP

bottom-up

DP

???