Basic Techniques

UMD CP Club Summer CP Week 3

Greedy Algorithms

Let's look at a simple task

Suppose you have unlimited number of coins with values $\{\$1,\$5,\$10,\$50,\$100\}$ and you want to buy a phone that costs $\$567$. What is the minimum number of coins you have to take to pay exactly $\$567$?

A Simple Task

You take $5 \times \$100$, $1 \times \$50$, $1 \times \$10$, $1 \times \$5$, and $2 \times \$1$.

This is also the minimum number of coins required

A Simple Task

If you think that is intuitive, then what we just did is called a "Greedy Algorithm"

The Greedy thought is that

Always taking the maximum value$\implies$ minimum number of coins

So what exactly is Greedy Algorithm

Last week, we learned about searching

So let's write a code that searches all possibilities

int values[] = {1, 5, 10, 50, 100};

int search(int x){
    if(x < 0) return 1e9;
    if(x == 0) return 0;
    int mn = 1e9;

    for(int i = 0; i < 5; i++){
        mn = min(mn, search(x - values[i]) + 1);
    }

    return mn;
}

code for coins problem

So what exactly is Greedy Algorithm

So what exactly is Greedy Algorithm

So what exactly is Greedy Algorithm

So, greedy algorithm is to find a way to solve an optimization problem faster!

Is Greedy always correct?

A different example

Suppose you have unlimited number of coins with values $\{\$1,\$5, \$8\}$ and you want to buy something that costs $\$x$. What is the minimum number of coins you have to take to pay exactly $\$x$?

What if $x = 15$?

Greedy Can Sometimes be Incorrect!

We have to prove the correctness of our algorithm!

Let's prove "Coin Change" problem

Coin Change

Suppose you have unlimited number of coins with $n$ different denominations, each with $c_1, c_2, \cdots, c_n$. You want to buy a product with price $x$. Find the minimum number of coins required to pay exactly $x$

$1 \le n \le 10^6$
$c_i < c_{i+1} \text{ and } c_i \mid c_{i+1}$

Text

To start with, let's think about why greedy works?

Coin Change

Text

To start with, let's think about why greedy works?

Coin Change

Text

To start with, let's think about why greedy works?

If the denominations includes $\$1$ and $\$5$

and you have 5 $\$1$. What would you do?

Convert them into 1 $\$5$!

Text

Coin Change

Claim 1. In the optimal solution, we won't take more than $\frac{c_{i+1}}{c_i}$ coins with denomination $c_i$

Coin Change

Text

Proof.

Suppose you have $\ge \frac{c_{i+1}}{c_i}$ coins with denomination $c_i$. We can convert $\frac{c_{i+1}}{c_i}$ ($> 1$) of the coins into one $c_i$ coin, and get a better result.

Claim 1. In the optimal solution, we won't take more than $\frac{c_{i+1}}{c_i}$ coins with denomination $c_i$

Text

Claim 2. In the optimal solution, if $c_i \le x < c_{i+1}$. Then $c_i$ must be taken.

Coin Change

Text

Proof (Part of).

(\frac{c_2}{c_1}-1)c_1 + (\frac{c_3}{c_2}-1)c_2 + \cdots + (\frac{c_i}{c_{i-1}}-1)c_i \\= (c_2 - c_1) + (c_3 - c_2) + \cdots + (c_i - c_{i-1}) = c_i - c_1 < c_i

Claim 2. In the optimal solution, if $c_i \le x < c_{i+1}$. Then $c_i$ must be taken.

Coin Change

Text

From these two claims, you can show that the greedy algorithm is actually optimal

Text

CSES - Movie Festival (Interval Scheduling)

There are $n$ movies, each starts at $l_i$ and ends at $r_i$.

You want to find the maximum number of movies that you can watch completely

$1 \le n \le 10^6$

Text

CSES - Movie Festival (Interval Scheduling)

There are $n$ movies, each starts at $l_i$ and ends at $r_i$.

You want to find the maximum number of movies that you can watch completely

Text

CSES - Movie Festival (Interval Scheduling)

What if we start watching movies with the earliest starting time?

Text

CSES - Movie Festival (Interval Scheduling)

What if we start watching movies with the earliest starting time?

Text

CSES - Movie Festival (Interval Scheduling)

The optimal solution is actually to watch movie with earliest ending time!

Text

CSES - Movie Festival (Interval Scheduling)

The optimal solution is actually to watch movie with earliest ending time!

CSES - Movie Festival (Interval Scheduling)

We will prove that this greedy method is correct!

CSES - Movie Festival (Interval Scheduling)

Claim. Sort the intervals by $r_i$ and watch the movies in order will give optimal answer

CSES - Movie Festival (Interval Scheduling)

Claim. Sort the intervals by $r_i$ and watch the movies in order will give optimal answer

Proof.

Let the movies watched by greedy algorithms be $\{a_1, a_2, \cdots, a_n\}$, and $\{b_1, b_2, \cdots, b_m\}$ be an optimal choice

in which all $a_i, b_i$ are indices, and $r_{a_i} \le r_{a_{i+1}}$, $r_{b_i} \le r_{b_{i+1}}$

CSES - Movie Festival (Interval Scheduling)

Claim. Sort the intervals by $r_i$ and watch the movies in order will give optimal answer

Proof.

Let the movies watched by greedy algorithms be $\{a_1, a_2, \cdots, a_n\}$, and $\{b_1, b_2, \cdots, b_m\}$ be an optimal choice

in which all $a_i, b_i$ are indices, and $r_{a_i} \le r_{a_{i+1}}$, $r_{b_i} \le r_{b_{i+1}}$

CSES - Movie Festival (Interval Scheduling)

Proof (Continued).

We claim that $r_{a_i} \le r_{b_i}$ for all $1 \le i \le \min(m,n)$

we will prove this by induction on $i$

Base Case: When $i = 1$, then the induction hypothesis holds by the design of greedy algorithm

Inductive Step: Suppose the hypothesis holds for all $k \le i$. Then since $r_{b_{i-1}} \le l_{b_i}$, we get $r_{a_{i-1}} \le l_{b_i}$. Therefore, greedy algorithm can also take interval $b_i$. Hence, $r_{a_i} \le r_{b_i}$

CSES - Movie Festival (Interval Scheduling)

Proof (Continued).

We proved that $r_{a_i} \le r_{b_i}$ for all $1 \le i \le \min(m,n)$

CSES - Movie Festival (Interval Scheduling)

Proof (Continued).

We proved that $r_{a_i} \le r_{b_i}$ for all $1 \le i \le \min(m,n)$

Now suppose $n < m$ (greedy is worse than optimal solution), then it means that the optimal soluton takes at least one interval more than the greedy solution.

CSES - Movie Festival (Interval Scheduling)

Proof (Continued).

We proved that $r_{a_i} \le r_{b_i}$ for all $1 \le i \le \min(m,n)$

Now suppose $n < m$ (greedy is worse than optimal solution), then it means that the optimal soluton takes at least one interval more than the greedy solution.

We have $r_{b_n} \le l_{b_{n+1}}$, and therefore $r_{a_n} \le l_{b_{n+1}}$. This means greedy can also take interval $b_{n+1}$, this is a contradiction.

How to solve Greedy Problems

Come up with a greedy thought that might work
You have to make sure it's correct
1. Prove it! (Try to do this when solving problems)
2. Prove by AC (It is fine to do this in contests)

Ways to prove Greedy Algorithm

Let $S$ be the answer from Greedy, and $S'$ be the an optimal solution
Suppose $S$ is not optimal, show contradiction

Exchange Argument

UMD CP Club Practice Contest 2 D

UMD CP Club Practice Contest 2 D

We claim that the queue sorted this way is optimal

// Let v[i] = {a_i, b_i}
sort(v.begin(),v.end(), [](auto a, auto b){
    return a.second * (a.second-2) + a.first * (a.first + 5) \
    < b.second * (b.second-2) + b.first * (b.first + 5);
});

UMD CP Club Practice Contest 2 D

In this type of greedy, try to think about what would happen when you swap two people

UMD CP Club Practice Contest 2 D

Suppose the sequence is now $\{c_1, c_2, \cdots, c_n\}$

Then the sum of impatience will be

\sum_{i=1}^n \sum_{j=i+1}^n \left (b_{c_i}(b_{c_i}-2) - a_{c_j}(a_{c_j}+5) \right )

UMD CP Club Practice Contest 2 D

What would happen if we swap $c_i, c_j$?

UMD CP Club Practice Contest 2 D

What would happen if we swap $c_i, c_j \ (i < j)$?

\Delta_{cost} = \left (-\sum_{i < k < j} b_{c_i}(b_{c_i}-2)\right) + \left (\sum_{i < k < j} b_{c_j}(b_{c_j}-2)\right) + \left (j-i+1) (a_{c_j}(a_{c_j}+5) - a_{c_i}(a_{c_i}+5)\right) \\ = (j-i+1)(a_{c_j}(a_{c_j}+5) - a_{c_i}(a_{c_i}+5) - b_{c_i}(b_{c_i}-2) + b_{c_j}(b_{c_j}-2))

So if there are two elements where

$a_{c_i}(a_{c_i}+5) + b_{c_i}(b_{c_i}-2) > a_{c_j}(a_{c_j}+5) + b_{c_j}(b_{c_j}-2) $

Swap them gives smaller costs!

UMD CP Club Practice Contest 2 D

This type of greedy is called Exchange Argument

We can prove that sorting (swaping) would give optimal solution

Undo Greedy

Codeforces 1526C2 - Potions

-4

-3

Initially your HP is $0$, you have to start from left

Each potion will add $a_i$ to your HP

Your HP need to be $\ge 0$ at all time

Find the maximum number of potions you can drink

Codeforces 1526C2 - Potions

-4

-3

Claim. It is always best to drink $a_i \ge 0$

Proof. Trivial

Codeforces 1526C2 - Potions

-4

-3

What about $a_i < 0$?

Codeforces 1526C2 - Potions

-4

-3

There are two cases for $a_i < 0$

$hp > 0$
$hp \le 0$

If it is first case, just drink it.

Should we drink $-3$ here?

The answer is Yes!

We undrink $-4$ and drink $-3$!

Codeforces 1526C2 - Potions

-4

-3

Red mean we do not drink

Green means we drink it

\text{health} = 2

Codeforces 1526C2 - Potions

-4

-3

Red mean we do not drink

Green means we drink it

\text{health} = 3

Codeforces 1526C2 - Potions

-4

-3

Red mean we do not drink

Green means we drink it

\text{health} = 0

Codeforces 1526C2 - Potions

This is the final code of this problem

int n;
cin >> n;

int sum = 0;
priority_queue<int,vector<int>,greater<>> pq;

for(int i = 0;i < n;i++){
    int x;
    cin >> x;
    pq.push(x);
    sum += x;
    while(sum < 0){
        sum -= pq.top();
        pq.pop();
    }
}

cout << pq.size() << '\n';

Greedy requires some experiences

So it is actually normal if you don't understand

Just practice more and you will get it!

Modular Arithmetic

In Competitive Programming Problems (Usually Math or DP)

Since the answer could be large, output the answer modulo $10^9+7$

Is a very common sentence that you will see in CP problems

We need modular arithmetic to prevent large numbers / overflow

In math, we denote

x \equiv y \pmod m

if $x \bmod m = y \bmod m$

and $0 \le (x \bmod m) < m$

Under modulo, these three operations will hold

(a \bmod m) + (b \bmod m) \equiv a + b \pmod m

(a \bmod m) - (b \bmod m) \equiv a - b \pmod m

(a \bmod m) \times (b \bmod m) \equiv a \times b \pmod m

This can be easily shown with Division Algorithm

However, how can we do division?

\frac{a \bmod m}{b \bmod m} \not \equiv \frac{a}{b} \pmod m

To do division under modulo, we need modular inverse!

Definition.

$b$ is the modular inverse of $a$ under modulo $m$ iff

$ab \equiv 1 \pmod m$

We denote $b$ as $a^{-1}$

We can do division under modulo where

ab^{-1}

means $a$ divided by $b$

There are three ways to find modular inverse in CP

Fermat's Little Theorem
A Recurrence that Works
Extended Euclidean Algorithm

We will only talk about Fermat's Little Theorem

and focus on the case when $m$ is a prime

Binary Exponentiation

Now, suppose we want to find the value of

$a^n$

How can you do this?

Binary Exponentiation

a^n = \underbrace{a \times a \times \cdots \times a}_{n \text{ times}}

This can be done in $O(n)$

$n \le 10^9$

Binary Exponentiation

Think about it, what would you do when you want to calculate

3^8

without a calculator

Binary Exponentiation

3^8 = 3^4 \times 3^4

Binary Exponentiation

3^8 = 3^4 \times 3^4 \\ 3^4 = 3^2 \times 3^2 \\ 3^2 = 3^1 \times 3^1

Binary Exponentiation

3^8 = 3^4 \times 3^4 \\ 3^4 = 3^2 \times 3^2 \\ 3^2 = 3^1 \times 3^1

Can you do same thing with arbitrary n?

Binary Exponentiation

a^n

Binary Exponentiation

a^n = a^{\lfloor \frac{a}{2} \rfloor} \times a^{\lfloor \frac{a}{2} \rfloor} \times a^{(n \bmod 2)}

We can now do this in

$T(n) = T(\frac{1}{2} n) + O(1) \implies O(\log n)$

Binary Exponentiation

int fastpow(int a, int n){
    if(n == 0) return 1;
    int tmp = fastpow(a, n/2);
    return tmp * tmp * (n % 2 ? a : 1);
}

Recursive binary exponentiation

Binary Exponentiation

There is also another way

Binary Exponentiation

a^{15} = a^{(1111)_2} = a^8 \times a^4 \times a^2 \times a

If you convert $n$ into binary

then you also only need to do $O(\log n)$!

Binary Exponentiation

int fastpow(int a, int n){
    int res = 1;
    while(n){
        if(n % 2 == 1) res *= a;
        a = a * a;
        n >>= 1;
    }
    return res;
}

Iterative binary exponentiation

Fermat's Little Theorem

Fermat's Little Theorem

From number theory, we know that if $p$ is prime then

a^p \equiv a \pmod p

By the Fermat's Little Theorem

Fermat's Little Theorem

a^p \equiv a \pmod p

multiplying $a^{-2}$ on both sides, we get

a^{p-2} \equiv a^{-1} \pmod p

which is the modular inverse!

Fermat's Little Theorem

int fastpow(int a, int n, int mod);

int inv(int a, int p){
    return fastpow(a, p-2, p);
}

code for modular inverse in $O(\log n)$

CSES - Exponentiation

Be careful!

In C++, $a \% b$ is defined as $a - b \lfloor \frac{a}{b} \rfloor$

This means it could possibly give negative results (e.g. -4 % 3 = -1)

You will have to convert it back to $[1,m]$

Small Practice (Use C++)

Evaluate the following values with C++

$(100000 \times 99999) \bmod 13$
$2^{100} \bmod 10^9+7$
$((−4) × 1234567890) \bmod 998244353$
$100$th fibonacci number $\bmod 10^9+7$

Small Practice (Use C++)

Evaluate the following values with C++

$(100000 \times 99999) \bmod 13$
$2^{100} \bmod 10^9+7$
$((−4) × 1234567890) \bmod 998244353$
$100$th fibonacci number $\bmod 10^9+7$

Answer:

12
976371285
52950205
782204094

Prefix Sum & Difference Array

CSES - Static Range Sum Queries

You are given an array $a_1, a_2, \cdots, a_n$. There will be $q$ queries. For each query $[l,r]$, output the sum of values in $[l,r]$.

\sum_{i=l}^r a_i

$1 \le n \le 10^6$
$1 \le q \le 10^6$

CSES - Static Range Sum Queries

Naive Solution

int sum = 0;
for(int i = l; i <= r; i++){
    sum += arr[i];
}
cout << sum << "\n";

This is too slow! $O(nq)$

CSES - Static Range Sum Queries

Let $pref[i] = \sum_{j = 1}^i a_j$, which is the sum from $[1,i]$

Then $[l,r]$ can be calculated by $[1,r]$ - $[1,l-1]$

CSES - Static Range Sum Queries

We can precompute $pref$ in $O(n)$, then each query can be answered in $O(1)$!

int pref[n+1] = {};
for(int i = 1; i <= n; i++){
    pref[i] = arr[i];
    pref[i] += pref[i-1];
}

int q;
cin >> q;
while(q--){
    int l, r;
    cin >> l >> r;
    cout << pref[r] - pref[l-1] << "\n";
}

CSES - Static Range Sum Queries

The array $pref[i]$ is called the prefix sum of $a$

When you need multiple queries of a range, finding prefix sum can do it in $O(1)$

HackerRank - Array Manipulation

You are given an array $a_1, a_2, \cdots, a_n$.

There will be $q$ operations. For each operation, add $v$ to $[l,r]$.

After all the operations, output the final array

$1 \le n \le 10^7$
$1 \le q \le 2 \times 10^5$

HackerRank - Array Manipulation

Think about what would not change when we add values to a range?

HackerRank - Array Manipulation

Think about what would not change when we add values to a range?

1, \ 2, \ 3, \ 4, \ 5, \ 6, \ 7, \ 8

HackerRank - Array Manipulation

Think about what would not change when we add values to a range?

1, \ 2, \ 3, \ 4, \ 5, \ 6, \ 7, \ 8

\Downarrow

1, \ 2, \ 4, \ 5, \ 6, \ 7, \ 8, \ 8

The difference between $a_3, a_4, a_5, a_6, a_7$ did not change!

HackerRank - Array Manipulation

Let's declare another array $b_i = a_i - a_{i-1}$, and we call $b_i$ the difference array of $a$

Then the prefix sum of $b$, $\sum_{k=1}^i b_i = a_1 + (a_2 - a_1) + \cdots + (a_i - a_{i-1}) = a_i$

Let's see what this can help us in this problem

HackerRank - Array Manipulation

a = 1, \ 2, \ 3, \ 4, \ 5, \ 6, \ 7, \ 8

b = 1, \ 1, \ 1, \ 1, \ 1, \ 1, \ 1, \ 1

\Downarrow

a = 1, \ 2, \ 4, \ 5, \ 6, \ 7, \ 8, \ 8

b = 1, \ 1, \ 2, \ 1, \ 1, \ 1, \ 1, \ 0

We add $1$ to $[3,7]$, then the result shows

$b_l := b_l + v, b_{r+1} := b_{r+1} - v$

HackerRank - Array Manipulation

Therefore, for each operation, we only need to modify two elements from $b$

This can be done in $O(1)$

To find $a$, just do prefix sum to $b$ after all operations!

HackerRank - Array Manipulation

int b[n+1];
for(int i = 1; i <= n; i++){
    b[i] = a[i] - a[i-1];
}
int q;
cin >> q;
while(q--){
    int l, r, v;
    cin >> l >> r >> v;
    b[l] += v, b[r+1] -= v;
}
for(int i = 1; i <= n; i++){
    b[i] += b[i-1];
}
//b is the original array after opeartions

O(n + q)

Actually, prefix sum and difference array are inverse operations

Actually, prefix sum and difference array are inverse operations

You can also find prefix xor, prefix product to do similar things

2D Prefix Sums

CSES - Forest Queries

You are given an $n \times n$ grid. Each cell is either empty or contains a tree.

There will be $q$ queries. For each query, output how many trees are there in the rectangle with top left corner $(a,b)$ and bottom right corner $(c,d)$.

$1 \le n \le 2000$
$1 \le q \le 10^6$

CSES - Forest Queries

We want to find the sum in the red rectangle

CSES - Forest Queries

\texttt{pref[x][y]}= \sum_{i=1}^x \sum_{j=1}^y a[i][j]

We will precompute the 2D Prefix Sum (the sum of rectangle from $(1,1)$ to $(x,y)$)

CSES - Forest Queries

\texttt{sum(a,b,c,d) = pref[c][d] - pref[c][b-1] - pref[a-1][d] + pref[a-1][b-1]}

CSES - Forest Queries

Therefore, each query is now $O(1)$ after precomputing the 2D Prefix Sum

\texttt{pref[i][j] = pref[i-1][j] + pref[i][j-1] - pref[i-1][j-1]}

To compute the prefix sum, we do

This can be done in $O(n^2)$

Can you also do 2D Difference Array?

Sweep Line Algorithm

What is Sweep Line?

What is Sweep Line?

We use an imaginary line to think about problems related to segments / rectangle / triangle / circle / ... on a 2D plane

Let's look at an example!

Length of Union of Segments

You are given $n$ segments, each covers the interval $[l_i, r_i]$. Find the total length of intervals these segments covers.

$1 \le n \le 10^6$
$1 \le l_i, r_i \le 10^9$

Length of Union of Segments

You are given $n$ segments, each covers the interval $[l_i, r_i]$. Find the total length of intervals these segments covers.

Length of Union of Segments

Think about this, let an imaginary line start from the left of all segments, and moves to the right by $1$ unit each second

Length of Union of Segments

Think about this, let an imaginary line start from the left of all segments, and moves to the right by $1$ unit each second

Length of Union of Segments

The only thing that actually matters are the left bound and right bound of the interval

Length of Union of Segments

Solution:

For each segment $[l,r]$, divide it into $\{l, 1\}$ which means the line passes $l$ and $\{r+1, -1\}$ means the lines leaves $r$
Keep track of how many segments the line passes through at this moment, and you only need to sum up how many seconds $a > 0$

Length of Union of Segments

map<int,int> mp;
for(int i = 0; i < n; i++){
    mp[l[i]]++;
    mp[r[i] + 1]--;
}

int cnt = 0, ans = 0, prev = 0;

for(auto [a,b] : mp){
    if(cnt > 0) ans += cnt * (a - prev);
    cnt += b;
}

cout << ans << "\n";

Code for Union of Segments

Monotonic Stack

CSES - Nearest Smaller Values

CSES - Nearest Smaller Values

Naive Solution

For each $i$, search $j$ from $i-1$ to $1$ and check if $a_j < a_i$

for(int i = 1; i <= n; i++){
    for(int j = i-1; j >= 1; j--){
        if(arr[j] < arr[i])
            ans[i] = j;
    }
}

O(n^2)

stack<int> st;
for(int i = 1;i <= n;i++){
    while(!st.empty() && arr[st.top()] >= arr[i]){
        st.pop();
    }
    if(!st.empty()) 
        ans[i] = st.top();
        st.push(i);
    }
 }

deque<int> mx;
vector<int> ans;
for(int i = 0;i < n;i++){
    while(!mx.empty() && i-mx.front() >= k)
        mx.pop_front();
    while(!mx.empty() && arr[mx.back()] <= arr[i])
        mx.pop_back();
    mx.push_back(i);
    if(i >= k-1) ans.push_back(arr[mx.front()]);
}

$$\min_{(l,r) \in I}\{\min(h_l,\cdots,h_r)(r-l+1)\}$$

// Assume that the heights are stored in h[]
int maxArea = 0;
for(int l = 1; l <= n; l++){
    int minH = h[l];
    for(int r = l; r <= n; r++){
        minH = min(minH ,h[r]);
        maxArea = max(maxArea, minH*(r-l+1));
    }
}

O(n^2)