COMP3010: Algorithm Theory and Design
Daniel Sutantyo, Department of Computing, Macquarie University
3.0 Brute Force Methods
Prelude
3.0 - Brute Force Methods
- What have we done so far?
- Problems and Algorithms (types of problems, input, output)
- Complexity - \((O,\Theta,\Omega)\)-notations, worst/average/best-case
- Correctness - loop invariants, induction
Why brute force?
3.0 - Brute Force Methods
- It is still a useful technique
- if you don't know how to solve a problem efficiently (and you have to solve it soon), then you can always try brute-forcing it
- brute-forcing a problem may lead to a better algorithm
- Students are not normally asked to do something using brute force, so you didn't get that much practice
- Topics:
- search space (permutation and combinations)
- recursive backtracking
- branch and bound
Tractable vs Intractable Problems
3.0 - Brute Force Methods
- Tractable problems are problems that we can solve using a polynomial time algorithm
- remember that polynomial time algorithm means that it is \(O(n^k)\) for some constant \(k\) (you can bound it above with some polynomial function)
- These are 'good' algorithms, in fact, given a problem, we often say that an algorithm is 'bad' unless it's polynomial
- Tractable problems is probably not as interesting because improving the algorithm means going from 'good' to 'better', as opposed to going from 'bad' to 'good'
- and improving an already good algorithm is actually hard to do
Tractable vs Intractable Problems
3.0 - Brute Force Methods
- Intractable problems are problems for which there are no known polynomial-time algorithm to solve it
- "not polynomial" doesn't always mean exponential
- how about \(O(n!)\) or \(O(2^{2^n})\)
- if the algorithm cannot be bounded above by a polynomial, then we say that it is quasi-polynomial (or super-polynomial, or subexponential)
- remember \(O(n^{\log n})\)?
- how about \(O(n^{\log\log n})\)?
- "not polynomial" doesn't always mean exponential
- Remember that one of the motivations in studying algorithms is so that we can recognise intractable problems (so you don't waste time trying to find an efficient solution)
Tractable vs Intractable Problems
3.0 - Brute Force Methods
- Another way to think about intractable problems:
- what if the number of solutions is exponential?
- e.g. knapsack, give me all the possible combination where the total weight of the items is less than \(k\)
- e.g. travelling salesman, give me all the routes from \(A\) to \(B\) that costs less than \(k\)
- Generally though, we think about intractable problem as a hard problem with just a 'simple' solution (as per the previous slide)
Tractable vs Intractable Solution
3.0 - Brute Force Methods
- A little aside:
- I sometimes use the term "intractable solution" to mean an algorithm with worse-than-polynomial time complexity, as opposed to "tractable solution" which is polynomial-time.
- I make the distinction between "intractable solution" and "intractable problem" because you can have an intractable solution for a tractable problem
- i.e. the problem is easy, you just wrote a slow algorithm for it
- e.g. bogosort
Tractable vs Intractable Solution
3.0 - Brute Force Methods
intractable solution
for
an intractable problem
intractable solution
for
a tractable problem
Tractable vs Intractable Problems
3.0 - Brute Force Methods
- Just because a problem is hard, doesn't mean that it's hard to come up with a solution for it
- coming up with an efficient solution for a tractable problem can be difficult
- coming up with a solution for a tractable or intractable problem is generally not that difficult
- brute force!
- Are there problems that cannot be solved?
Tractable vs Intractable Problems
3.0 - Brute Force Methods
- Sometimes brute force is a good starting point in solving problems, because
- it is easy to write
- it lets you recognise patterns in the solution which can lead to a better algorithm
- sometimes it is the only solution
- this is also why exponential algorithm is considered 'bad', because brute-forcing something is exponential, so are really doing that much better?
COMP3010 - 3.0 - Brute Force Methods
By Daniel Sutantyo
COMP3010 - 3.0 - Brute Force Methods
General brute-forcing methods
