Dynamic

Programming

UMD Competitive Programming Club Week 4

What is Dynamic Programming

1. Overlapping Subproblems

What is Dynamic Programming

1. Overlapping Subproblems

What is Dynamic Programming

f_n = f_{n-1} + f_{n-2}

1. Overlapping Subproblems

What is Dynamic Programming

f_n
f_{n-1}
f_{n-2}
f_{n-2}
f_{n-3}

1. Overlapping Subproblems

What is Dynamic Programming

f_n
f_{n-1}
f_{n-2}
f_{n-2}
f_{n-3}

They are same!

1. Overlapping Subproblems

What is Dynamic Programming

int f(int n) {
    if (n <= 1) return 1;
    else return f(n-1) + f(n-2);
}

You will compute some states more than once!!

\mathcal{O}(\phi^n)

1. Overlapping Subproblems

What is Dynamic Programming

int dp[MAXN];
int f(int n) {
    if (dp[n]) return dp[n];
    else if (n <= 1) return dp[n] = 1;
    else return dp[n] = f(n-1) + f(n-2);
}

Every state will now only be compute once

This is called Memoization

\mathcal{O}(n)

2. Optimal Substructures

What is Dynamic Programming

2. Optimal Substructures

What is Dynamic Programming

There is a stairs with \(n\) levels, there are \(a_i\) coins on each level, you can walk \(1\) or \(2\) steps each time. What is the maximum coins you can get?

f(n) = \max(f(n-1),f(n-2)) + a_i

Let \(f(n)\) be the maximum coins you get when you get to \(n\)

Two Different Ways

Two Different Ways

Top-Down Approach (Recursion)

Bottom-Up Approach (For Loop)

Two Different Ways

Let's look at Fibonacci again!!!

Two Different Ways

Top-Down (Recursion with Memoization)

int dp[MAXN];
int f(int n) {
    if (dp[n]) return dp[n];
    else if (n <= 1) return dp[n] = 1;
    else return dp[n] = f(n-1) + f(n-2);
}

Two Different Ways

Bottom-Up (For Loop)

int dp[MAXN];
dp[0] = dp[1] = 1;
for (int i = 2; i <= n; i++){
    dp[i] = dp[i - 1] + dp[i - 2];
}

We prefer Bottom-Up since the constant is smaller than recursion

How to Solve DP Problems

How to Solve DP Problems

Define States \(\Rightarrow\) Find The Transition \(\Rightarrow\) Handle Base Cases

The concept is abtract

Let's look at the problems!

Classic DP Problems

Classic DP Problems 1 - Frogs 1

Classic DP Problems 1 - Frogs 1

\text{Let } dp[i] \text{ be the minimum cost for the frog to jump to stone } i

Define The States

Classic DP Problems 1 - Frogs 1

Find The Transition

dp[i] = \min(dp[i-1] + |h_i - h_{i-1}|, dp[i-2] + |h_i - h_{i-2}|)

Classic DP Problems 1 - Frogs 1

Base Case

There is no cost for Frog to be at stone \(1\)

\Rightarrow dp[1] = 0

Classic DP Problems 1 - Frogs 1

Time complexity

\mathcal{O}(n)

Classic DP Problems 2 - Frogs 2

What if now a frog can jump at most \(k\) step forward?

\displaystyle dp[i] = \max_{1 \le j \le k} (dp[i-j] + |h_i - h_{i-j}|)

Each transition is \(\mathcal{O}(k)\)

\(\Rightarrow \mathcal{O}(nk)\)

Classic DP Problems 3 - Vacation

You are on a vacation for \(n\) days,

each day, you can do three activities

  1. Swim in the sea, and get \(a_i\) happiness
  2. Catch bugs, and get \(b_i\) happiness
  3. Do homework, and get \(c_i\) happiness

However, you cannot do same activities on consecutive days

What is the maximum happiness you can have?

Classic DP Problems 3 - Vacation

Define the state!

\text{ Let } dp[i] \text{ be the maximum happiness for the first } i \text{ days }

You want the state to be easy to do transition!

\text{ Let } dp[i][j] \text{ be the maximum happiness for the first } i \text{ days} \\ \text{where you do } jth \text{ activity on } ith \text{ day}

The answer is

\max(dp[n][0], dp[n][1], dp[n][2])

Classic DP Problems 3 - Vacation

Find The Transition!

dp[i][0] = \max(dp[i-1][1], \ dp[i-1][2]) + a_i \\ dp[i][1] = \max(dp[i-1][0], \ dp[i-1][2]) + b_i \\ dp[i][2] = \max(dp[i-1][0], \ dp[i-1][1]) + c_i

Each transition is \(\mathcal{O}(1)\)

Classic DP Problems 3 - Vacation

Base Case

dp[0][0] = dp[0][1] = dp[0][2] = 0
\mathcal{O}(n)

Time

Classic DP Problems 4 - LCS

You are given two strings \(s,t\)

Output their Longest Common Subsequence (LCS)

Classic DP Problems 4 - LCS

Define the states

\text{ Let } dp[i][j] \text{ be the LCS of } s_{0..i-1} \text{ and } t_{0..j-1}

You can also think of \(i,j\) as lengths

You can also think of \(i,j\) as lengths

Classic DP Problems 4 - LCS

Find The Transition

dp[i][j] = \max \begin{cases} dp[i-1][j-1] + 1 &, \text{ if } s[i-1] = t[j-1] \\ dp[i-1][j-1] \\ dp[i][j-1] \end{cases}

Each Transition is \(\mathcal{O}(1)\)

Classic DP Problems 4 - LCS

Base Case

dp[0][i] = dp[i][0] = 0

Time

\mathcal{O}(nm)

Classic DP Problems 4 - LCS

How to construct answer?

Classic DP Problems 4 - LCS

How to construct answer?

Let \(from[i][j]\) be \(\{x,y\}\) where the value of \(dp[i][j]\) is from

Classic DP Problems 4 - LCS

How to construct answer?

for (int i = 1; i <= n; i++) {
    for (int j = 1; j <= n; j++) {
    	if (s[i-1] == s[j-1]) {
            dp[i][j] = dp[i-1][j-1] + 1;
            from[i][j] = {i-1, j-1};
        } 
        if (dp[i-1][j] > dp[i][j]) {
	    dp[i][j] = dp[i-1][j];
            from[i][j] = dp[i-1][j];
        }
        if (dp[i][j-1] > dp[i][j]) {
	    dp[i][j] = dp[i][j-1];
            from[i][j] = dp[i][j-1];
        }
    }
}

Classic DP Problems 5 - MCS

You are given an array with \(n\) integers

Find the maximum sum over all continuous subarray

Classic DP Problems 5 - MCS

Define the states

\text{Let } dp[i] \text{ be the maximum subarray sum ending at } i

The answer will be \(\max(dp[i])\) over all \(i\)

Classic DP Problems 5 - MCS

Find The Transitions

dp[i] = \max(dp[i-1] + a[i], a[i])

Either continue the previous sum, or start a new one

Classic DP Problems 5 - MCS

Base Case

dp[0] = 1

Classic DP Problems 6 - LIS

You are given an array of \(n\) numbers

Find the Longest Increasing Subsequence

Classic DP Problems 6 - LIS

Define The States

\text{Let } dp[i] \text{ be the length of LIS ends with } a_i

Classic DP Problems 6 - LIS

Find Transition

\displaystyle dp[i] = \max_{j \le i, \ a_j < a_i}(dp[j] + 1)

Classic DP Problems 6 - LIS

Base Case

dp[i] = 1

Time

\mathcal{O}(n^2)

Classic DP Problems 6 - LIS

Can we do better?

Classic DP Problems 6 - LIS

There are two ways to speed this up

  1. RMQ + Point Update
  2. Binary Search

We will only talk about 2. here

Classic DP Problems 6 - LIS

Consider the sequence

1, \ 3, \ 7, \ 5, \ 2, \ 6, \ 1

Let's look at it's DP Table

Classic DP Problems 6 - LIS

Let's define \(v[j]\) as the Minimum Value \(a_i\) such that \(dp[i] = j\)

Classic DP Problems 6 - LIS

Classic DP Problems 6 - LIS

You can easily prove that \(v\) is increasing

Find the largest number less than \(a_i\)

Binary Search!!

Classic DP Problems 6 - LIS

1
1

Classic DP Problems 6 - LIS

1
1
2
3

Classic DP Problems 6 - LIS

1
1
2
3
3
7

Classic DP Problems 6 - LIS

1
1
2
3
3
5
3

Classic DP Problems 6 - LIS

1
1
2
3
3
4
3
3

Classic DP Problems 6 - LIS

1
1
2
3
3
4
3
3
4
6

Classic DP Problems 6 - LIS

1
1
2
3
3
4
3
3
4
6
1

Classic DP Problems 6 - LIS

Code

int dp[n];
vector<int> v;

for(int i = 0; i < n; i++){
    int idx = lower_bound(v.begin(), v.end(), a[i]) - v.begin();
    if(idx == v.size()) v.push_back(a[i]);
    else v[idx] = a[i];
    dp[i] = idx + 1;
}
    
cout << v.size() << "\n";
\mathcal{O}(n \log n)

Knapsack Problem

0/1 Knapsack

There are \(n\) items, each has weight \(w_i\) and value \(v_i\)

Now, you have a knapsack with capacity \(C\), find the maximum sum of values of items you can put in the knapsack

0/1 Knapsack

Define The States

\text{Let } dp[i][j] \text{ be the sum of value such that} \\ \text{The first } i \text{ items has sum of weight } j

Answer would be \(dp[n][W]\)

0/1 Knapsack

Find The Transition

For each item, you can choose to take or not take

Therefore, we can derive the transition as

dp[i][j] = \max(dp[i-1][j], dp[i-1][j-w[i]] +v[i])

0/1 Knapsack

Base Case

dp[0][j] = 0

(This can prevent some border cases)

0/1 Knapsack

Do we really need 2D?

NO!

0/1 Knapsack

There is a trick called Rolling DP

0/1 Knapsack

There is a trick called Rolling DP

dp[i\bmod 2][j]

We can define the states as this

since every time we do transition from \(i-1\)

0/1 Knapsack

dp[i\bmod 2][j] = \max(dp[(i-1) \mod 2][j], dp[(i-1) \mod 2][j - W] + v[i])

Transition becomes

0/1 Knapsack

dp[i\bmod 2][j] = \max(dp[(i-1) \mod 2][j], dp[(i-1) \mod 2][j - W] + v[i])

Transition becomes

Actually, we can even get rid of the first dimension!

0/1 Knapsack

for(int i = 1; i <= n; i++){
    for(int j = W; j >= 0; j--){
        if(j >= w[i])
            dp[j] = max(dp[j], dp[j-w[i]] + v[i]);
    }
}

1D Version!

You may ask "Why is \(j\) from \(W\) to \(0\)?"

0/1 Knapsack

This is the original transition

0/1 Knapsack

You want the arrows to be exactly once

Infinite Knapsack

What if every item has infinitely many?

\(k\) from \(0\) to \(W\)!

Infinite Knapsack

for(int i = 1; i <= n; i++){
    for(int j = 0; j <= W; j++){
        if(j >= w[i])
            dp[j] = max(dp[j], dp[j-w[i]] + v[i]);
    }
}

Infinite Knapsack

for(int i = 1; i <= n; i++){
    for(int j = 0; j <= W; j++){
        if(j >= w[i])
            dp[j] = max(dp[j], dp[j-w[i]] + v[i]);
    }
}
O(nW)

Knapsack with Constraints

for(int i = 1; i <= n; i++){
    for(int k = 0; k < num[i]; k++){
        for(int j = W; j >= 0; j--){
            if(j >= w[i])
                dp[j] = max(dp[j], dp[j-w[i]] + v[i]);
        }
    }
}
O(W\sum k)

Knapsack with Constraints

Can we do better?

Knapsack with Constraints

Can we do better?

Do we really need to go through \(k\) for each item?

Knapsack with Constraints

There is actually a faster way in

O(W\sum \lg k)

Range DP

Range DP

AtCoder DP Contest pN - Slimes

There are \(n\) slimes in a line each with a size \(a_i\), every time, you can choose \(2\) adjacent slimes with size \(a,b\) , and combine them into a slime with size \(a+b\), creating a cost of \(a+b\). What is the minimum cost required to combine all slimes?

Range DP

AtCoder DP Contest pN - Slimes

How should we define the state?

Range DP

AtCoder DP Contest pN - Slimes

Let \(dp[i]\) be the minimum cost to combine the first \(i\) slimes?

This will not work!!

Range DP

AtCoder DP Contest pN - Slimes

Define the states

\text{Let } dp[l][r] \text{ be the minimum cost to combine all simes in } [l,r]

Answer will be \(dp[1][n]\)

Range DP

AtCoder DP Contest pN - Slimes

Find The Transition

\displaystyle dp[l][r] = \min_{l \le k \le r}(dp[l][k] + dp[k][r] + \text{sum}(l,r))

Range DP

AtCoder DP Contest pN - Slimes

Find The Transition

\displaystyle dp[l][r] = \min_{l \le k \le r}(dp[l][k] + dp[k][r] + \text{sum}(l,r))

Consider this range

Range DP

AtCoder DP Contest pN - Slimes

Either \(10 + 3\) or \(4 + 9\) is possible

Combining \(6+3\), then combine \(4+9\) gives smaller cost!

Range DP

AtCoder DP Contest pN - Slimes

Base Case

dp[i][i] = 0

(No cost)

Range DP

AtCoder DP Contest pN - Slimes

Runtime

\displaystyle dp[l][r] = \min_{l \le k \le r}(dp[l][k] + dp[k][r] + \text{sum}(l,r))

The transition takes \(O(n)\) if you precompute the prefix sum

Therefore, the total time is \(O(n^3)\)

Range DP

Important thing about Range DP!

Let's look at two different codes, and let's vote on which one is correct!

Range DP

for (int l = 1; l <= n; l++) {
    for(int r = 1; r <= n; r++) {
        for (int k = l; k <= r; k++) {
            dp[l][r] = min(dp[l][k] + dp[k][r] + sum(l, r));
        }
    }
}
for (int l = n; l >= 1; l--) {
    for(int r = l; r <= n; r++) {
        for (int k = l; k <= r; k++) {
            dp[l][r] = min(dp[l][k] + dp[k][r] + sum(l, r));
        }
    }
}

Range DP

Why is the first one incorrect?

for (int l = 1; l <= n; l++) {
    for(int r = 1; r <= n; r++) {
        for (int k = l; k <= r; k++) {
            dp[l][r] = min(dp[l][k] + dp[k][r] + sum(l, r));
        }
    }
}

It is possible to use the data from ranges that we have not computed!

Range DP

The order of transition is important for Range DP!

Range DP

The order of transition is important for Range DP!

Range DP

AtCoder DP Contest pL - Deque

Range DP

AtCoder DP Contest pL - Deque

Define the states

\text{Let } dp[l][r] \text{ be the maximum score the first player can get in } [l,r]

Range DP

AtCoder DP Contest pL - Deque

Find The Transition

dp[l][r] = \max \begin{cases}arr[l] + (\text{sum}(l+1,r) - dp[l+1][r]) \\ (\text{sum}(l,r-1) - dp[l][r-1]) + arr[r] \end{cases}

This is because second player becomes first after first player's operation!

Range DP

AtCoder DP Contest pL - Deque

Find The Transition

dp[l][r] = \text{sum}(l,r) - \min(dp[l+1][r],dp[l][r-1])

This is because second player becomes first after first player's operation!

Range DP

AtCoder DP Contest pL - Deque

Base Case

dp[i][i] = a[i]

If there is only one stone, then first player takes the stone!

\mathcal{O}(n^2)

Bitmask DP

Bitmask DP

Travelling Salesman Problem

You are given a weighted graph with \(n\) vertices and \(m\) directed edges. You want to know the minimum sum of weights to start from \(s\), going through all vertices exactly once, then return to \(s\)

Bitmask DP

Travelling Salesman Problem

Define the states

\text{Let } dp[i][0/1]\ldots[0/1] \text{ be the minimum sum to visit the cities and ends at } i

If there are \(20\) vertices, then it means you have to do 20D DP!

Bitmask DP

Travelling Salesman Problem

Define the states

\text{Let } dp[i][mask] \text{ be the minimum sum to visit the cities and ends at } i

Compress the \(n\)D into a single integer!

Then answer will be \(\min(dp[i][2^n-1] + dis[i][s])\)

Bitmask DP

Travelling Salesman Problem

Find the transitions

dp[i][mask] = \min_{j \not \in mask}(dp[j][mask \oplus 2^j] + dis[j][i])

We go through all pairs of vertices for each mask!

Bitmask DP

Travelling Salesman Problem

Base Case

dp[s][2^s] = 0

Bitmask DP

Travelling Salesman Problem

Base Case

dp[s][2^s] = 0

Bitmask DP

for(int mask = 0;mask < (1<<n);mask++){
    for(int u = 0;u < n;u++){
        if(mask&(1<<u)){
            for(int v = 0;v < n;v++){
                if(mask&(1<<v)) continue;
                dp[v][mask^(1<<v)] = min(dp[v][mask^(1<<v)], dp[u][mask] + dis[u][v]);
            }
        }
    }
}
\mathcal{O}(n^2 2^n)

Some Cool Problems

ABC 236 pE - Average and Median

Codeforces 895C - Square Subsets

Codeforces 895C - Square Subsets

Made with Slides.com