Dynamic

Programming

What is Dynamic Programming?

Dynamic programming Is a method for solving complex problems by breaking them down into simpler subproblems.

a DP problem usually can be solved optimally by breaking it into sub-problems and then recursively finding the optimal solutions to the sub-problems

Dynamic Programming

VS Recursion

Both are using sub-problems to solve the main problem
Recursion is much more general
If it can be solved by dynamic programming, DP is much faster
Usually DP requires more space
SPACE VS TIME

When we use DP?

Recursion is making some redundant computation
Optimal sub-structures:
optimal solutions to the original problem contains optimal solutions to sub-problems

How to use DP?

Think about the recursion solution
Find sub problems that have been computed multiple times
See if the sub problems are optimal sub-structure
Try to memorize them to avoid the redundant computing

Fibonacci Numbers

int Fabonacci(int n) {
    if(n == 0 || n == 1) return 1;
    return F(n-1) + F(n-2);
}

def fibonacci(n):
    if n == 0 or n == 1:
        return 1
    return fibonacci(n-1) + fibonacci(n-2)

F(6)

F(5)

F(4)

F(3)

F(2)

F(1)

F(0)

F(1)

F(0)

F(1)

F(0)

F(1)

F(0)

F(2)

F(1)

F(0)

F(2)

Need some space to avoid the computation

int Fabonacci(int n) {
    if(n <= 1) return 1;
    int[] result = new int[n + 1];
    result[0] = 1;
    result[1] = 1;
    for(int i = 2; i < n + 1; i ++) {
        result[i] = result[i-1] + result[i-2];
    }
    return result[n];
}

We use an Array to store the temp result

What is the optimal sub-structure?

def fibonacci(n):
    if n <= 1:
        return 1
    result = [0] * (n + 1)
    result[0] = 1
    result[1] = 1
    for i in range(2, n + 1):
        result[i] = result[i-1] + result[i-2]
    return result[n]

Unique Paths

int uniquePaths(int m, int n) {
      if(m==1 || n==1) return 1;
      return uniquePaths(m-1, n) + uniquePaths(m, n-1);
}

Where is the redundancy?

What is the optimal sub-structure?

def uniquePaths(m, n):
    if m == 1 or n == 1:
        return 1
    return uniquePaths(m-1, n) + uniquePaths(m, n-1)

Unique Paths

Optimal Sub-structure

a(i,j) = a(i-1,j) + a(i, j-1)

Unique Paths

public int uniquePaths(int m, int n) {  
    int[][] a = new int[m][n];  
    for (int i = 0; i < m; i++) {  
        a[i][0] = 1;  
    }  
    for (int i = 0; i < n; i++) {  
        a[0][i] = 1;  
    }  
    for (int i = 1; i < m; i++) {  
        for (int j = 1; j < n; j++) {  
            a[i][j] = a[i-1][j] + a[i][j-1];  
        }  
    }  
    return a[m-1][n-1];  
}

def uniquePaths(m, n):
    a = [[0] * n for _ in range(m)]
    for i in range(m):
        a[i][0] = 1
    for i in range(n):
        a[0][i] = 1
    for i in range(1, m):
        for j in range(1, n):
            a[i][j] = a[i-1][j] + a[i][j-1]
    return a[m-1][n-1]

Minimum Path Sum

1	3	4	2
3	5	2	3
2	1	2	3
2	2	4	2

Best optimal sub-structure?

Given a grid filled with non-negative numbers, find a path from top left to bottom right, which minimizes the sum of all numbers along its path.

Note: You can only move either down or right at any point in time.

Minimum Path Sum

1	3	4	2
3	5	2	3
2	1	2	3
2	2	4	2

PathSum(m,n) = MIN(PathSum(m,n-1),PathSum(m-1,n)) + matrix(m,n)

Minimum Path Sum

public int minPathSum(int[][] grid) {
    if(grid == null || grid.length==0)
        return 0;
    int m = grid.length;
    int n = grid[0].length;
    int[][] dp = new int[m][n];
    dp[0][0] = grid[0][0];    
    for(int i=1; i<n; i++){
        dp[0][i] = dp[0][i-1] + grid[0][i];
    }
    for(int j=1; j<m; j++){
        dp[j][0] = dp[j-1][0] + grid[j][0];
    }
    for(int i=1; i<m; i++){
        for(int j=1; j<n; j++){
            if(dp[i-1][j] > dp[i][j-1]){
                dp[i][j] = dp[i][j-1] + grid[i][j];
            }else{
                dp[i][j] = dp[i-1][j] + grid[i][j];
            }
        }
    }
    return dp[m-1][n-1];
}

def minPathSum(grid):
    if not grid or len(grid) == 0:
        return 0
    m = len(grid)
    n = len(grid[0])
    dp = [[0] * n for _ in range(m)]
    dp[0][0] = grid[0][0]
    for i in range(1, n):
        dp[0][i] = dp[0][i-1] + grid[0][i]
    for j in range(1, m):
        dp[j][0] = dp[j-1][0] + grid[j][0]
    for i in range(1, m):
        for j in range(1, n):
            if dp[i-1][j] > dp[i][j-1]:
                dp[i][j] = dp[i][j-1] + grid[i][j]
            else:
                dp[i][j] = dp[i-1][j] + grid[i][j]
    return dp[m-1][n-1]

Minimum Path Sum

Any improvement?

We can use less space to get the same result without hurting time complexity

Minimum Path Sum II

public static int minPathSum(int[][] grid) {
    if(grid == null || grid.length==0)
        return 0;
    int m = grid.length;
    int n = grid[0].length;
     int[] newline = new int[n];
     int[] oldline = new int[n];
     oldline[0] = grid[0][0];    
     for(int i=1; i<n; i++){
    	 oldline[i] = oldline[i-1] + grid[0][i];
     }
     for(int i=1; i<m; i++){
    	 newline[0] = grid[i][0] + oldline[0];
         for(int j=1; j<n; j++){
             if(oldline[j] > newline[j-1]){
                 newline[j] = newline[j-1] + grid[i][j];
             }else{
                 newline[j] = oldline[j] + grid[i][j];
             }
         }
         oldline = newline;
     }
     return newline[n-1];
}

def minPathSum(grid):
    if not grid or len(grid) == 0:
        return 0
    m = len(grid)
    n = len(grid[0])
    newline = [0] * n
    oldline = [0] * n
    oldline[0] = grid[0][0]
    for i in range(1, n):
        oldline[i] = oldline[i-1] + grid[0][i]
    for i in range(1, m):
        newline[0] = grid[i][0] + oldline[0]
        for j in range(1, n):
            if oldline[j] > newline[j-1]:
                newline[j] = newline[j-1] + grid[i][j]
            else:
                newline[j] = oldline[j] + grid[i][j]
        oldline = newline.copy()
    return newline[n-1]

Minimum Path Sum II

We can even only use one array

def minPathSum(self, grid: List[List[int]]) -> int:
        """
        64. Minimum Path Sum
        https://leetcode.com/problems/minimum-path-sum/
        滚动数组
        time complexity: O(row * col)  
        space complexity: O(row)   
        """
       
        if not grid or len(grid) == 0: 
            return 0
        row, col = len(grid), len(grid[0])
        
        pre = [(2 ** 31 - 1)] * col
        
        for i in range(row):
            cur = [(2 ** 31 - 1)] * col
            for j in range(col):
                if i == 0 and j == 0:
                    cur[j] = grid[i][j]
                elif i == 0:
                    cur[j] = cur[j - 1] + grid[i][j]
                elif j == 0:
                    cur[j] = pre[j] + grid[i][j]
                else:
                    cur[j] = min(pre[j], cur[j - 1]) + grid[i][j]
            pre = cur
         
        
        return cur[col - 1]

0-1 Knapsack

Given a knapsack which can hold w pounds of items, and a set of items with weight w1, w2, ... wn. Each item has its value s1,s2,...,sn. Try to select the items that could put in knapsack and contains most value.

What is the optimal sub-structure?

0-1 Knapsack

w[i][j]: for the previous total i items, the max value it can have for capacity j

Which two we need to use to compare?

w[i][j]: for the previous total i items, the max value it can have for capacity j

When you iterate i, and j, you need to try:

If the new i could be added into j
if it could and added, could it be better

	1	2	3	4	5	6	7
0	0	0	0	0	0	0	0
1	3	3	3	3	3	3	3
2	3	3	8	11	11	11	11
3	3	3	8	11	11	11	12
4	3	3	8	11	11	11	12

w[i][j]: for the previous total i items, the max value it can have for capacity j

Example: weights{1,3,4,5} values{3,8,4,7}

public int knapsack(int capacity, int[] weights, int[] values) {
	int length = weights.length;
	if (capacity == 0 || length == 0)
		return 0;
	int[][] w = new int[length + 1][capacity + 1];
	for (int i = 1; i <= length; i++) {
		int index = i - 1;
		for (int j = 1; j <= capacity; j++) {
			if (j < weights[index]) {
				w[i][j] = w[i - 1][j];
			} else if (w[i - 1][j - weights[index]] + values[index] > w[i - 1][j]) {
				w[i][j] = w[i - 1][j - weights[index]] + values[index];
			} else {
				w[i][j] = w[i - 1][j];
			}
		}
	}
	return w[length][capacity];
}

def knapsack(capacity, weights, values):
    length = len(weights)
    if capacity == 0 or length == 0:
        return 0
    w = [[0] * (capacity + 1) for _ in range(length + 1)]
    for i in range(1, length + 1):
        index = i - 1
        for j in range(1, capacity + 1):
            if j < weights[index]:
                w[i][j] = w[i - 1][j]
            elif w[i - 1][j - weights[index]] + values[index] > w[i - 1][j]:
                w[i][j] = w[i - 1][j - weights[index]] + values[index]
            else:
                w[i][j] = w[i - 1][j]
    return w[length][capacity]

Coin Change

We mentioned this question in DFS and use DFS will be TLE, even with the Greedy idea.

The question is similar to knapsack

coin and amount will be the two dimension

How we update the result?

public int coinChange(int[] coins, int amount) {
    Arrays.sort(coins);
    int length = coins.length;
    int[][] dp = new int[length][amount + 1];
    for (int j = 0; j <= amount; j ++) {
        if (j % coins[0] == 0) {
            dp[0][j] = j / coins[0];
        } else {
            dp[0][j] = -1;
        }
    }
    for (int i = 1; i < length; i ++) {
        for (int j = 0; j <= amount; j ++) {
            if (j < coins[i]) {
                dp[i][j] = dp[i-1][j];
            } else {
                int temp = Integer.MAX_VALUE;
                for (int k = 0; k <= j / coins[i]; k ++) {
                    int remaining = j - coins[i] * k;
                    if (dp[i-1][remaining] != -1 && dp[i-1][remaining] + k < temp) {
                        temp = dp[i-1][remaining] + k;
                    } 
                }
                dp[i][j] = temp < Integer.MAX_VALUE ? temp : -1;
            }
        }
    }
    return dp[length - 1][amount];
}

def coinChange(coins, amount):
    coins.sort()
    length = len(coins)
    dp = [[0] * (amount + 1) for _ in range(length)]
    for j in range(amount + 1):
        if j % coins[0] == 0:
            dp[0][j] = j // coins[0]
        else:
            dp[0][j] = -1
    for i in range(1, length):
        for j in range(amount + 1):
            if j < coins[i]:
                dp[i][j] = dp[i-1][j]
            else:
                temp = float('inf')
                for k in range(j // coins[i] + 1):
                    remaining = j - coins[i] * k
                    if dp[i-1][remaining] != -1 and dp[i-1][remaining] + k < temp:
                        temp = dp[i-1][remaining] + k
                dp[i][j] = temp if temp < float('inf') else -1
    return dp[length - 1][amount]

public int coinChange(int[] coins, int amount) {
        int max = amount + 1;             
        int[] dp = new int[amount + 1];  
        Arrays.fill(dp, max);  
        dp[0] = 0;   
        for (int i = 1; i <= amount; i++) {
            for (int j = 0; j < coins.length; j++) {
                if (coins[j] <= i) {
                    dp[i] = Math.min(dp[i], dp[i - coins[j]] + 1);
                }
            }
        }
        return dp[amount] > amount ? -1 : dp[amount];
    }

Optimized bottom-up

def coinChange(coins, amount):
    max_value = amount + 1
    dp = [max_value] * (amount + 1)
    dp[0] = 0
    
    for i in range(1, amount + 1):
        for coin in coins:
            if coin <= i:
                dp[i] = min(dp[i], dp[i - coin] + 1)
    
    return -1 if dp[amount] > amount else dp[amount]

Dynamic Programming VS Greedy

Greedy algorithm: an algorithmic paradigm that builds up a solution piece by piece, always choosing the next piece that offers the most obvious and immediate benefit. So the problems where choosing locally optimal also leads to a global solution are best fit for Greedy.

Dynamic Programming VS Greedy

Dynamic programming: it's mainly an optimization over plain recursion. Wherever we see a recursive solution that has repeated calls for the same inputs, we can optimize it using DP. The idea is to simply store the results of subproblems so that we do not have to re-compute them when needed later. This simple optimization reduces time complexities from exponential to polynomial.

Algorithm relationships

Brute Force Search (DFS/Backtrace) <= For each stage, the optimal state is defined all prior stage states.
Greedy <= For each stage, the optimal state is defined by optimal state from prior stage.
Dynamic programing <= For each stage, the optimal state is defined by states from some stages, and won't care about how those states are created. (每个阶段的最优状态可以从之前某个阶段的某个或某些状态直接得到，而不管之前这个状态是如何得到的)

How to solve DP problem

Steps to solve a DP:

Identify if it is a DP problem
Decide a state expression (with least parameters)
Create state transition function
Find the boundary and build the dp table

Longest Increasing Subsequence

3, 1, 4, 5, 7, 6, 8, 2

1, 4, 5, 6, 8 (Or 1, 4, 5, 7, 8)

Longest Increasing Subsequence

What is the optimal sub-structure?

Longest Increasing Subsequence

We store lis[i] for the LIS by i?

We store lis[i] for the LIS end with sequence[i]

Longest Increasing Subsequence

  public int lengthOfLIS(int[] nums) {
        if (nums == null || nums.length == 0) {
            return 0;
        }
        
        int n = nums.length;
        int[] dp = new int[n];
        
        // Initialize dp array with 1, since the length of the LIS at each element is at least 1
        for (int i = 0; i < n; i++) {
            dp[i] = 1;
        }
        
        // Iterate through the array to find the longest increasing subsequence
        for (int i = 0; i < n; i++) {
            for (int j = 0; j < i; j++) {
                if (nums[i] > nums[j]) {
                    dp[i] = Math.max(dp[i], dp[j] + 1);
                }
            }
        }
        
        // Find the maximum value in dp array, which will be the length of the LIS
        int maxLength = 0;
        for (int i = 0; i < n; i++) {
            maxLength = Math.max(maxLength, dp[i]);
        }
        
        return maxLength;
    }

站在i的位置向之前看

看看之前的 j，寻找更长subsequence

Longest Increasing Subsequence

def longestIncreasingSubsequence(nums):
    if not nums:
        return 0
    
    n = len(nums)
    dp = [1] * n
    
    # Iterate through the array to find the longest increasing subsequence
    for i in range(n):
        for j in range(i):
            if nums[i] > nums[j]:
                dp[i] = max(dp[i], dp[j] + 1)
    
    # Return the maximum value in dp array, which is the length of the LIS
    return max(dp)

State mathematical description

Given a series of numbers of size N,

Assume F(k): the LIS length, which end with kth number in the series.

Q: Find the max in F(1)...F(N)

Longest Common Sequence

Example: abcfbc abfcab

return 4 (abcb)

Longest Common Sequence

Example: abcfbc abfcab

return 4 (abcb)

Longest Common Sequence

What is the optimal sub-structure?

maxCommon(i,j): longest common string for String A(0,i) and String B(0,j)

We finally need to get maxCommon(stringA.length, stringB.length)

Longest Common Sequence

What is the relationship between maxCommon(i,j) and maxCommon(i-1,j-1)?

If(A[i-1] = B[j-1]) ?

If(A[i-1] != B[j-1])?

Longest Common Sequence

What is the relationship between maxCommon(i,j) and maxCommon(i-1,j-1)?

If(A[i-1] = B[j-1]) ?

If(A[i-1] != B[j-1])?

maxCommon(i,j) = maxCommon(i-1,j-1) + 1

maxCommon(i,j) = max(maxCommon(i-1,j), maxCommon(i,j-1))

Longest Common Sequence

public static int longestCommonString(String a, String b) {
	int m = a.length();
	int n = b.length();
	int[][] maxCommon = new int[m+1][n+1];
	for(int i = 0; i <= m; i ++) {
		maxCommon[i][0] = 0;
	}
	for(int j = 0; j <= n; j ++) {
		maxCommon[0][j] = 0;
	}
	for(int i = 1; i <= m; i ++) {
		for(int j = 1; j <= n; j ++) {
		  if(a.charAt(i-1) == b.charAt(j-1)) {
		    maxCommon[i][j] = maxCommon[i-1][j-1] + 1;
		  }
		  else {
		    maxCommon[i][j] = Math.max(maxCommon[i][j-1], maxCommon[i-1][j]);
		  }
		}
	}
	return maxCommon[m][n];
}

Longest Common Sequence

def longestCommonString(a, b):
    m = len(a)
    n = len(b)
    maxCommon = [[0] * (n+1) for _ in range(m+1)]
    for i in range(1, m+1):
        for j in range(1, n+1):
            if a[i-1] == b[j-1]:
                maxCommon[i][j] = maxCommon[i-1][j-1] + 1
            else:
                maxCommon[i][j] = max(maxCommon[i][j-1], maxCommon[i-1][j])
    return maxCommon[m][n]