COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

11.1 - NP Completeness and Reduction

The Story So Far ...

11.1 - NP Completeness and Reduction

• We discussed different complexity classes, including P, SUBEXP, EXP, and of course, NP

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

• Problems in NP are verifiable in polynomial time, but we don't know how to solve them in polynomial time

The Story So Far ...

11.1 - NP Completeness and Reduction

\(\text{P}\)

\(\text{NP} =\)

\(\text{EXP}\)

• We discussed different complexity classes, including P, SUBEXP, EXP, and of course, NP

• Problems in NP are verifiable in polynomial time, but we don't know how to solve them in polynomial time

The Story So Far ...

11.1 - NP Completeness and Reduction

• Problems in NP are verifiable in polynomial time, but we don't know how to solve them in polynomial time

The Story So Far ...

11.1 - NP Completeness and Reduction

• TLDR:
  • P : easy to check, easy to solve
  • NP : easy to check, hard to solve
    • so maybe NP is also easy to solve, making P = NP
• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

The Story So Far ...

11.1 - NP Completeness and Reduction

• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

The Story So Far ...

11.1 - NP Completeness and Reduction

• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

The Story So Far ...

11.1 - NP Completeness and Reduction

• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

The Story So Far ...

11.1 - NP Completeness and Reduction

• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

\(\text{P}\)

\(\text{NP} =\)

\(\text{EXP}\)

The Story So Far ...

11.1 - NP Completeness and Reduction

• What is it that we want to do?
  • we want to know if NP is special or not, or a more extreme way of putting it, does NP actually exist?

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

The Story So Far ...

11.1 - NP Completeness and Reduction

• How do you prove that P = NP?
  • you have to prove that every single problem in NP is also in P, that is, you have to find a polynomial-time solution for all of them
  • yes, that is hard
• Fortunately, the notion of NP-completeness make this a bit easier 
  • but first we have to discuss NP-hard, so that's the plan:
    • NP-hard \(\rightarrow\) NP-complete \(\rightarrow\) Reduction

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• Optimisation problem: find the best possible solution among the set of all possible solutions 
  • find the weight of the longest path from A to B in \(G\)
  • find the Hamiltonian cycle in \(G\)
• Decision problem: the output to every instance of input is either a yes or a no
  • is the weight of the longest path from A to B in \(G\) more than \(k\)?
  • is there a Hamiltonian cycle in \(G\)?
• Which one do you think is harder?

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• Optimisation problem:
  • find the Euler cycle in a graph
  • Fleury's Algorithm or Hierholzer's Algorithm
    • not examinable, just putting it here in case you want to google it
• Decision problem:
  • is there an Euler cycle in the graph?
  • check if every vertex has an even degree (why?)
• In general, yes, the decision problem is easier

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• You can cast an optimisation problem into a decision problem
  • i.e. use the solution to a decision problem to answer the solution to the optimisation problem
• Example:
  • optimisation: what is the weight of the shortest path in the graph?
  • decision: is the weight of the shortest path in the graph less than \(k\)?
• Here's the trick, just keep asking:
  • is the weight of the shortest path less than 0?  
  • is the weight of the shortest path less than 1?
  • is the weight of the shortest path less than 2?
  • is the weight of the shortest path less than 3? 

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• If we know how to solve the decision problem of the shortest-path problem in polynomial time (relative to \(V\) and/or \(E\)), then can we solve the optimisation problem in polynomial time as well?
  • we start asking if the weight is 0, then move our way up until we hit a limit (say \(V*w_\text{max}\) where \(w_\text{max}\) is the largest weight of any edge)
  • this is still polynomial in complexity (we multiply by \(V\) and a constant)
• In other words, if we have a polynomial-time algorithm to solve the shortest-path decision problem, then we also have a polynomial-time algorithm to solve the shortest-path optimisation problem

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• What is the moral of the story?
  • can the decision problem be harder than the optimisation problem?
    • decision: yes or no
    • optimisation: find the actual answer that gives you yes or no
  • so optimisation problem is at least as hard as a decision problem
    • can the decision problem be as hard as the optimisation problem?

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• Hamiltonian cycle:
  • optimisation: find the Hamiltonian cycle
  • decision: is there a Hamiltonian cycle?
    • you have to find one anyway, so it is just as hard as the optimisation problem (or maybe there is a way?)

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• How is this relevant to our discussion about P vs NP?
  • you can use the decision problem to solve the optimisation problem (however, there is a big IF a bit later)
  • if I can solve the decision problem in polynomial time, then I can also solve the optimisation problem in polynomial time
    • assuming we can convert from the decision to optimisation in polynomial time (we can do so in the case of shortest and longest path problems)
  • if I cannot solve the decision problem in polynomial time, then that means I cannot solve the optimisation problem in polynomial time either

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• TLDR:

Decision in P

Optimisation in P

Decision not in P

Optimisation not in P

Decision not in P

Optimisation in P

Decision in P

Optimisation not in P

Decision Problem vs Optimisation Problem

11.1 - NP Completeness and Reduction

• In other words, we can just focus on decision problems, because it leads to the solution to the optimisation problem
  • and this is where we have to discuss that big IF I mentioned earlier

The Grey Area

11.1 - NP Completeness and Reduction

• What is an optimisation problem?
  • Example: The Longest Path Problem
    • What is the weight of the longest path from vertex A to vertex B
    • Find the set of edges that forms longest path from vertex A to vertex B
  • Some may argue that the second problem is a search problem, not an optimisation problem
• Why does this matter?
  • The decision version of the longest path problem cannot be used to find the actual set of edges that forms the longest path

The Grey Area

11.1 - NP Completeness and Reduction

• We seem to have two different definition of optimisation problem:
  • First definition: find the optimal value for the problem
  • Second definition: find the combination of elements that will give the optimal value for the problem
    • as I mentioned, we can argue that this is a search problem
• Nevertheless, whenever you encounter an NP problem, you will often hear it formulated using the second definition
  • e.g. Travelling Salesman: find the cheapest route from Town A that goes through every other town and then back to Town A
    • just the cost of the route or the route as well?
    • because if it's the route as well, then it's not NP

The Grey Area

11.1 - NP Completeness and Reduction

• CLRS uses the first definition
  • if the decision problem is hard, then both definitions are hard anyway, and so it doesn't matter if one definition is harder than the other
• However, if we are using the second definition, then if the decision problem is easy, it doesn't help us in finding the solution to the optimisation problem
  • longest path \(\rightarrow\) we only know the weight, not the actual path
  • does it matter?
    • to me, no, because I personally think P \(\ne\) NP, so I'm pretty sure the decision problem is not easy anyway
    • but I also think that solving the optimisation problem requires you to do the searching problem

The Grey Area

11.1 - NP Completeness and Reduction

• The problem is, we have to define the class NP-hard and NP-complete and you may encounter different definitions of NP-hard
  • does optimisation problem involves searching the actual combination that will give you the optimal solution
• For now I actually think it is easier if we use the second definition, that is, optimisation problem may require you to search the actual combination

NP-Hard

11.1 - NP Completeness and Reduction

• Remember that our end goal is to prove either P = NP or P \(\ne\) NP,
  • so we need to prove that every problem in NP can be done in polynomial time
  • or alternatively show that some problems in NP (but not in P) cannot be done in polynomial time

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{P}\)

\(\text{NP}=\)

\(\text{EXP}\)

• Remember that our end goal is to prove either P = NP or P \(\ne\) NP,
  • so we need to prove that every problem in NP can be done in polynomial time
  • or alternatively show that some problems in NP (but not in P) cannot be done in polynomial time

NP-Hard

11.1 - NP Completeness and Reduction

• One major ingredient in the attempt of a proof, is yet another complexity class NP-hard
  • you can ignore SUBEXP from now on, but that still leaves us with 4: EXP, P, NP, and NP-hard
  • NP-hard informally is the class of problems that are as hard as any problem in NP
    • doesn't really mean much ...

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

NP-Hard

11.1 - NP Completeness and Reduction

• Earlier you saw that we can use a decision problem algorithm to solve an optimisation problem (using the first definition)
• It should also be obvious that you can use the optimisation problem algorithm to solve the decision problem
• In both cases, we are converting one problem to another problem, and while the setting is the same, they're actually different problems!

NP-Hard

11.1 - NP Completeness and Reduction

• In the research on P vs NP, it was discovered that there is a class of problems such that every problem in NP can be converted to a problem in this class in polynomial time
  • for example, we can convert Hamiltonian cycle decision problem to Travelling Salesman decision problem

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

• Hamiltonian Cycle:
  • is there a cycle from one vertex back to itself that visits every other vertices once?
• Travelling Salesman:
  • is there a route from vertex A back to itself, visiting every other vertices once, that costs \(m\) or less?

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

1

1

1

1

1

1

1

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

1

1

1

1

1

1

1

2

2

2

2

(technically with travelling salesman problem we know the distance between each pair of cities)

Hamiltonian Cycle to Travelling Salesman

11.1 - NP Completeness and Reduction

1

1

1

1

1

1

1

2

2

2

2

• Is there a route from vertex A back to itself, visiting every other vertices once, that costs \(m\) or less?
  • let \(m = V\)

Reduction

11.1 - NP Completeness and Reduction

Reduction

11.1 - NP Completeness and Reduction

Reduction

11.1 - NP Completeness and Reduction

Reduction

11.1 - NP Completeness and Reduction

NP-Hard

11.1 - NP Completeness and Reduction

• Earlier we said NP-hard is the class of problems which are at least as hard as any problem in NP
• Formally, a problem H is NP-hard if every problem L in NP can be reduced in polynomial time to H
  • i.e. we can convert any problem in NP to any problem in NP-Hard
  • if you can use the solution in H to solve EVERY problem in NP
• So what do you think is harder, a problem that is NP, or a problem that is NP-hard?
  • cue in the sorting vs searching example in the previous slide
  • does it make more sense now when they say NP-hard problems are as hard as any problem in NP?

NP-Hard

11.1 - NP Completeness and Reduction

• Does this mean NP-hard problems are also NP?
  • no, we only said that you can reduce (convert) a problem in NP to a problem in NP-Hard
  • we may not be able to verify the solution for NP-hard problems in polynomial time, which is the requirement for being in NP
  • however, we can do this for SOME problems in NP-hard, so SOME NP-hard problems ARE in NP

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

\(\text{NP-hard}\)

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{P}\)

\(\text{NP} =\)

\(\text{EXP}\)

\(\text{NP-hard}\)

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{EXP}\)

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

\(\text{NP-hard}\)

NP-Hard

11.1 - NP Completeness and Reduction

\(\text{EXP}\)

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

\(\text{NP-hard}\)

???

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

• If a problem in NP and also in NP-Hard, then it is NP-complete
• In other words, the class NP-complete is the set of problems which are both NP and NP-hard
  • that is,
    • every problem in NP can be reduced down to a problem in NP-complete
    • every problem in NP-complete is in NP (the solution can be verified in polynomial time)
• NP-complete is a stronger condition than NP-hard
  • note: in the exam, I refer to some problems as NP-hard, but they may well be NP-complete
• NP-complete is always a decision problem (retcon: so are NP problems)

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

\(\text{EXP}\)

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

\(\text{NP-hard}\)

\(\text{NP-complete}\)

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

• Why are NP-complete problems so important?
• First of all, why is NP-hard not as important?
  • our focus is NP, we only care about problems that can be verified in polynomial time, and not all NP-hard problems can be verified in polynomial time
  • so, NP-hard problems are, well, HARD, as in for some of them we don't expect to be able to solve them in polynomial time, so fat chance trying to show that NP-hard = P
  • TLDR:
    • NP-hard is not NP (by definition), so can't use it to prove P = NP
    • good luck trying to show that P = NP-Hard

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

• However, in the case of NP-complete, we can still reduce every problem in NP to problems in NP-complete
  • Therefore, if we find just ONE problem in NP-complete that can be solved in polynomial time, we have proven that P = NP

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

\(\text{EXP}\)

\(\text{P}\)

\(\text{NP}\)

\(\text{EXP}\)

\(\text{NP-hard}\)

\(\text{NP-complete}\)

NP-Hard and NP-Complete

11.1 - NP Completeness and Reduction

\(\text{EXP}\)

\(\text{P}\)

\(\text{NP} = \)

\(\text{EXP}\)

\(\text{NP-hard}\)

\(\text{NP-complete} =\)

Back to the Grey Area (not examinable)

11.1 - NP Completeness and Reduction

• Remember that earlier we discussed two ways to define the optimisation problem, and it is easy to see that these problems are NP-hard (assuming that all problems in NP can be reduced to them)
• However, what happens if we only allow decision problems?
  • are there decision problems which are NP-hard but not NP-complete?
    • yes, but we are probably not going to come across them
    • one is the halting problem because of undecidability
    • and of course there problems in EXP that is not NP
  • we can discuss this more in the workshop if you find this confusing, but I also don't want to scare you off, since I do think these matters are hard and beyond the scope of this unit

What Are You Missing Out On

11.1 - NP Completeness and Reduction

• A proper study in complexity theory would require us to go through formal-language theory, that is, defining an alphabet \(\Sigma\) and a language \(L\) over \(\Sigma\)
  • you probably have seen \(\Sigma = \{0,1\}\) and \(L\) being the set of bits
• I have decided to skip these, but this is a big chunk of Chapter 34 of CLRS if you are interested in giving them a read
• The end result is that we will not be very rigorous when we are discussing complexity and reduction, but I want to focus on the more practical aspects of P vs NP, and less on the mathematics formality

Summary

11.1 - NP Completeness and Reduction

• At this point, you should understand the major ideas behind the P vs NP problem:
  • the classes P, NP, NP-hard, NP-complete
    • NP-complete and NP problems are always decision problems
  • decision problems and optimisation problems (and how we can go from one to the other)
  • basic idea of reduction