COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
12.0 - Approximation Algorithms
Where are we?
12.0 - Approximation Algorithms
- P vs NP
- NP-hard
- NP-complete (the one that matters the most)
- Reduction
- polynomial time reduction
- we reduce an NP-complete problem into a problem to show that the problem is NP-hard (or NP-complete)
- A \(\le_p\) B
- A is an NP-complete problem
- B is the problem we want to show to be NP-hard
Where are we?
12.0 - Approximation Algorithms
- Remember that we are not discussing P vs NP because we want to prove that P = NP or that P \(\ne\) NP
- Our main goal is to get you to recognise hard (i.e. NP) problems so that you do not spend time trying to find an efficient (i.e. polynomial-time) solution for it
Where are we?
12.0 - Approximation Algorithms
- We use reduction to show that a problem is hard, by reducing an NP-complete problem to it
- so now at least you can tell your boss that the problem is hard
- but now what?
Where are we?
12.0 - Approximation Algorithms
- Generally, you have three options:
- Give up
- as in, use the exponential time algorithm to solve it (maybe the input size is small enough)
- use branch-and-bound and/or dynamic programming
- how often do you have the worst-case anyway?
- Consider only special cases
- e.g. 2-SAT vs 3-SAT, directed acyclic graph, Euler vs Hamiltonian cycle
- Use an approximation algorithm
- Give up
Approximation Algorithms
12.0 - Approximation Algorithms
- Suppose that you are working on an optimisation problem where each solution is a positive numerical value
- a lot of optimisation problems are either maximisation or minimisation problems
- travelling salesman: find the cheapest route
- knapsack: find the maximum value of items you can carry
- a lot of optimisation problems are either maximisation or minimisation problems
- An approximation algorithm is an algorithm that may produce a solution that is suboptimal
- can it produce the optimal solution?
- yes, sometimes
- can it produce the optimal solution?
Approximation Ratio
12.0 - Approximation Algorithms
- How do we know if an approximation algorithm is good or bad?
- For any input of size \(n\):
- let \(C\) be the solution produced by the approximation algorithm
- let \(C^*\) be the optimal solution
- The approximation ratio \(\rho(n)\) of an approximation algorithm is the ratio between \(C\) and \(C^*\)
- if our problem is a maximisation problem, then \(0 \le C \le C^*\), and
- \(C^*/C \le \rho(n)\)
- if our problem is a minimisation problem, then \(0 \le C^* \le C\), and
- \(C/C^* \le \rho(n)\)
- if our problem is a maximisation problem, then \(0 \le C \le C^*\), and
Approximation Ratio
12.0 - Approximation Algorithms
- For maximisation problems, \(C^*/C \le \rho(n)\)
- the optimal solution is going to be at most \(\rho(n)\) times the solution of the approximation algorithm
- For minimisation problems, \(C/C^* \le \rho(n)\)
- the solution to the approximation algorithm is going to be at most \(\rho(n)\) times the optimal solution
Approximation Ratio
12.0 - Approximation Algorithms
- You can think of \(\rho(n)\) as a performance guarantee, that is, we guarantee that the result produced by our approximation algorithm is not going to be worse than a factor of \(\rho(n)\) compared to the optimal solution
- We say that our approximation algorithm is a \(\rho(n)\)-approximation algorithm
- For example, an \(n\)-approximation algorithm means that the result of our approximation algorithm is not going to be worse than \(n\) times the optimal solution (for a minimisation problem)
- so if \(n =\) 100, and the optimal solution is 7, our answer is not going to be worse than 700
- that is actually pretty horrific!
- so if \(n =\) 100, and the optimal solution is 7, our answer is not going to be worse than 700
Approximation Ratio
12.0 - Approximation Algorithms
- In this unit, we will consider only approximation algorithms with a constant \(\rho(n)\) and one that runs in polynomial time
- e.g. a 2-approximation algorithm means that no matter what \(n\) is, our solution will be
- no more than twice the optimal solution (for minimisation problems) or
- no less than half the optimal solution (for maximisation problems)
- in this case, we can drop the \(n\), and say it is a \(\rho\)-approximation algorithm
- e.g. a 2-approximation algorithm means that no matter what \(n\) is, our solution will be
- What do you think a 1-approximation algorithm is?
Approximation Algorithms vs Heuristics vs Probabilistic
12.0 - Approximation Algorithms
- How is approximation algorithms different to heuristics and probabilistic algorithms
- heuristics: gut-feeling, intuition
- "I cannot prove that this works, but somehow it does most of the time"
- all three are similar, trade off accuracy for performance, but
- probabilistic algorithm: it is probabilistic, there is a random element in the algorithm
- heuristics: no performance guarantee
- heuristics: gut-feeling, intuition
Approximation Schemes
12.0 - Approximation Algorithms
- CLRS also mentions polynomial-time approximation scheme where in addition to the input to the problem, we also take a constant \(\epsilon > 0\)
- for any fixed \(\epsilon\), the scheme is a \((1+\epsilon)\)-approximation scheme that runs in polynomial time in the size of \(n\) (the size of the input)
- for example, the running time can be \(O(n^{2/\epsilon})\)
- so as you get more precise, the running time gets worse
- you can think of it as a customisable approximation algorithm, as in, we get to choose how good of an approximation we get
- You don't have worry about this
What do you need to know?
12.0 - Approximation Algorithms
- Approximation algorithm:
- runs in polynomial time
- has a performance guarantee (the approximation ratio \(\rho\))
- the topic is hard because every approximation algorithm is different, depending on the problem
- what I expect from you is to be able to understand how an approximation algorithm works, and then work out its approximation ratio
- showing the approximation ratio is the hard part of this topic
- the topic is easy because, well, the approximation algorithms we're going to see are simple
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Remember that the decision version is NP-complete and the optimisation version is NP-hard
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Remember that the decision version is NP-complete and the optimisation version is NP-hard
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Remember that the decision version is NP-complete and the optimisation version is NP-hard
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Remember that the decision version is NP-complete and the optimisation version is NP-hard
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Here is a 2-approximation algorithm, given a graph \(G\langle V,E \rangle\):
- let \(E^\prime = E\), \(C = \{\}\)
- while \(E^\prime\) is nonempty:
- pick an edge \((u,v)\) from \(E^\prime\) and add \(u\) and \(v\) to \(C\)
- remove any edge in \(E^\prime\) that is connected to either \(u\) or \(v\)
- return C
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
\(C = \{b,c,d,e,f,g\}\)
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
a
b
e
c
d
f
g
\(C = \{b,c,d,e,f,g\}\)
a
b
e
c
d
f
g
\(C^* = \{b,e,d\}\)
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- It is a very simple algorithm, but it comes with a performance guarantee, and this is the hard part of this topic
- We can show that this approximation algorithm is a 2-approximation algorithm, meaning that the solution it produces will not be worse than 2 times the optimal solution (since it is a minimisation problem)
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Proof:
- does it run in polynomial time?
- each iteration, we pick one edge, and then add the vertices to the set \(C\), and since we can add at most \(V\) vertices, this is \(O(V)\)
- each iteration, we have to remove the edges connected to \(u\) and \(v\), so at most this is \(O(E)\)
- complexity is \(O(E^2 + V\)
- or \(O(E\log E + V)\) if you use some sort of priority queue
- or \(O(E + V)\) if you use an adjacency matrix
- the point is, it is polynomial in complexity
- does it run in polynomial time?
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Proof:
- does it return a correct answer?
- yes, we remove an edge from consideration only if it is already covered by the vertices in \(C\), so at the termination of the algorithm, since \(E^\prime\) is empty, we have covered every single edge with the vertices in \(C\)
- does it return a correct answer?
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Proof:
- what is the approximation ratio?
- claim that it is a 2-approximation algorithm
- let \(A\) be the set of edges that we pick:
- if we want to cover the edges in this set, how many vertices do we need?
- what is the approximation ratio?
a
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Proof:
- what is the approximation ratio?
- claim that it is a 2-approximation algorithm
- let \(A\) be the set of edges that we pick:
- if we want to cover the edges in this set, how many vertices do we need?
- each vertex only occurs once, so a vertex cover for
the edges in \(A\) must have at least \(|A|\) vertices - \(|A|\) is the lower bound for the size of the minimum
vertex cover, that is
\(|C^*| \ge |A|\)
- what is the approximation ratio?
b
e
c
d
f
g
Example: Minimum Vertex Cover
12.0 - Approximation Algorithms
- Proof:
- so far we have \(|C^*| \ge |A|\)
- now what can you say about \(|C|\)?
- when we terminate the algorithm, how many vertices do we have in \(C\)?
- each edge in \(A\) corresponds to two vertices,
so \(C\) will have exactly \(2 * |A|\) vertices
- each edge in \(A\) corresponds to two vertices,
- when we terminate the algorithm, how many vertices do we have in \(C\)?
- put the two together
- \[\begin{aligned}|C| &= 2 * |A|\\ &\le 2 *|C^*| \end{aligned}\]
- so, at worst, we will have twice as many vertices than the
optimal answer
b
e
c
d
f
g
How To Prove It
12.0 - Approximation Algorithms
- The proof technique is the standard method to show that an algorithm is a \(\rho\)-approximation algorithm
- The idea is to tie in the lower bound for the optimal solution to the result of the approximation algorithm (for a minimisation problem)
- e.g.
- the optimal solution must use at least or at most \(k\) of something (assume the lowest/highest possible)
- \(C^* \ge k\) or \(C^* \le k\)
- the approximation algorithm solution uses exactly \(2*k\) or \(k/2\) of the same resource
- \(C = 2k \le 2 * C^*\) or \(C = k/2 \ge C^*/2 \rightarrow 2*C \ge C^* \)
- the optimal solution must use at least or at most \(k\) of something (assume the lowest/highest possible)
- e.g.
How To Prove It
12.0 - Approximation Algorithms
- Notice something important here:
- you don't need to know what the optimal solution is, just the lower bound for it (or the upper bound, for a minimisation problem)
Summary
12.0 - Approximation Algorithms
- \(\rho\)-approximation algorithm
- What is going to be assessed:
- you will be given an NP problem and an approximation algorithm for the problem
- you need to show that the approximation algorithm is a \(\rho\)-approximation algorithm using the steps described in the example
- We are going to do a bit more of this in the workshop, but really, that is all we're going to do in this topic, so the concept is easy, the execution is the harder part.