COMP3010 - 3.2 - Search Space

COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

3.2 - Search Space

Search Space

3.2 - Search Space

Does this term apply on for search algorithms?
- Another term: solution space
Definition : the set of all possible solutions
- so what about "impossible" solutions?
Examples:
- Search if a value exist in an array
  - the search space is every element of the array
  - [ 3 , 5 , 7 , 8 , 10 ]
- Knapsack, Travelling Salesman, SAT, etc: all possible combinations
  - some will fail the constraint

Example: Making Rods

3.2 - Search Space

We have a bunch of metallic rods with fixed lengths. Is it possible to produce a rod of certain length if we are allowed to join any two or more rods together, but we cannot cut any rod into two or more pieces.

Example: Making Rods

3.2 - Search Space

The input to the problem is the lengths of the metallic rods that we have and the length \(L\) of the rod that we need to make
Example:
- input = [ 10 , 12 , 5 , 7 , 11 ], required length = 25
- output = yes or no

Example: Making Rods

3.2 - Search Space

Input: The array \(B = \{b_1, b_2, \dots, b_n\}\) containing the lengths of the rods and an integer \(L \ge 0\)
Output: True if there is a subarray \(\{b_{i_0}, b_{i_1}, \dots, b_{i_k}\}, 1 \le i_j \le n\) such that

\[\sum_{j=0}^k b_{i_j} = L\]

Example: Making Rods

3.2 - Search Space

Sample input: [ 10 , 12 , 5 , 7 , 11 ]
- L = 25, there is no solution
- L = 22, there are multiple solutions, e.g. [ 10 , 12 ], [ 10 , 5 , 7 ]
The search space has \(2^5 = 32\) combinations, but how do we actually find them?

for (int i = 0; i < b.length; i++){
  for (int j = 0; j < b.length; j++){
    for (int k = 0; k < b.length; k++){
      for (int l = 0; l < b.length; l++){
         for (int m = 0; m < b.length; m++){
            ...
}}}}}

Problems and Subproblems

3.2 - Search Space

One of the most important thing you need to recognise when attempting to solve a problem is, can we make a smaller subproblem out of the original problem?
Recognising the subproblem will lead to us being able to construct a search space
We are going to look at this again next week, but for now, try to recognise the subproblems to the original problem

Problems and Subproblems

3.2 - Search Space

Given [10, 12, 5, 7, 11], can we make 25?
- reduce the problem to subproblems
- if I pick 10 now, then I have [ 12 , 5 , 7 , 11 ] to choose from, and have to make 15
- if I pick 12 now, then I have [ 10 , 5 , 7 , 11 ] to choose from, and have to make 13
- if I pick 5 now, then I have [ 10 , 12 , 7 , 11 ] to choose from, and have to make 20
- etc

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11] L = 25

[12,5,7,11] L = 15

[10,5,7,11] L = 13

[10,12,7,11] L = 20

[10,12,5,11] L = 18

[10,12,5,7] L = 14

Problems and Subproblems

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0) 
    return true;
  // if we don't have any more rods, then we can't make it
  if (b.length == 0)
    return false;
  
  // else keep on going with the remaining sum
  for (???) {
    // HALP
  }
}

Problems and Subproblems

3.2 - Search Space

Given [10, 12, 5, 7, 11], can we make 25?
- Option 1: pick one of the elements, then recurse
  - pick one of 10, 12, 5, 7, or 11
- Option 2: either pick the element at the current index, or don't, then recurse
  - pick 10 or don't pick it
  - pick 12 or don't pick it
  - pick 5 or don't pick it

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11] L = 25

[12,5,7,11] L = 15

[12,5,7,11] L = 25

don't pick 10

pick 10

[5,7,11] L = 3

[5,7,11] L = 15

[5,7,11] L = 25

don't pick 12

pick 12

don't pick 12

[7,11] L = -2

[11] L = -9

pick 5

pick 7

...

[7,11] L = 15

[11] L = 8

pick 7

don't pick 5

...

don't pick 7

...

[11] L = 15

[7,11] L = 15

pick 5

...

don't pick 7

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11] L = 25

don't pick 10

pick 10

[10,12,5,7,11] L = 25

[12,5,7,11] L = 15

[10,5,7,11] L = 13

[10,12,7,11] L = 20

[10,12,5,11] L = 18

[10,12,5,7] L = 14

[12,5,7,11] L = 15

[12,5,7,11] L = 25

Problems and Subproblems

3.2 - Search Space

Moral of the story: make sure you understand whether you need to generate permutations or combinations

Example: Word building

3.2 - Search Space

Another example: produce all possible combinations of three unique letters in the English alphabet
- e.g. abc, abd, abe, ..., ayz, bac, bad, ..., xyz

Example: Word building

3.2 - Search Space

"a"

"ab"

"a"

"b"

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

... ...

(and so on)

Example: Word building

3.2 - Search Space

"a"

"b"

"z"

"c"

"d"

... ...

"a"

"b"

"z"

"c"

"d"

... ...

"a"

"b"

"z"

"c"

"d"

... ...

Example: Word building

3.2 - Search Space

"a"

"b"

"z"

"c"

"d"

... ...

"a"

"b"

"y"

"c"

"d"

... ...

"b"

"c"

"z"

"d"

... ...

Tree Structure

3.2 - Search Space

Notice that the search space is actually a tree
- I often refer to this as the decision tree or search tree
- Is it always a binary tree?

Example: License Plates

3.2 - Search Space

Problem: Generate all possible license plates

"a" "b" "c" ..... "9" "0"

Search space: 36 * 36 * 36 * 36 * 36 * 36

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11] L = 25

[12,5,7,11] L = 15

[12,5,7,11] L = 25

don't pick 10

pick 10

[5,7,11] L = 3

[5,7,11] L = 15

[5,7,11] L = 25

don't pick 12

pick 12

don't pick 12

[7,11] L = -2

[11] L = -9

pick 5

pick 7

...

[7,11] L = 15

[11] L = 8

pick 7

don't pick 5

...

don't pick 7

...

[11] L = 15

[7,11] L = 15

pick 5

...

don't pick 7

Traversing the Search Space

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0){
    return true;
  }
  // if we get to the end of array but still can't make the sum, 
  // return false
  if (i >= b.length) {
    return false;
  }
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}

if (L == 0) or (i >= b.length), we are done since either we managed to make the sum earlier, or we've run out of rods
solve(b,index+1,L-b[index]): pick the current element, keep going
solve(b,index+1,L): don't pick the current element, keep going

Traversing the Search Space

3.2 - Search Space

If your search space is a tree, then how will we traverse through it (or more generally, how do you traverse the graph)?
You can use either DFS or BFS
What did we do here?

public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0){
    return true;
  }
  // if we get to the end of array but still can't make the sum, 
  // return false
  if (i >= b.length) {
    return false;
  }
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}

Traversing the Search Space

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  ...
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}

[10,12,5,7,11] L = 25

[12,5,7,11] L = 15

[12,5,7,11] L = 25

don't pick 10

pick 10

don't pick 12

pick 12

don't pick 12

[5,7,11] L = 3

[5,7,11] L = 15

[5,7,11] L = 25

Traversing the Search Space

3.2 - Search Space

"a"

"ab"

"a"

"b"

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

... ...

Traversing the Search Space

3.2 - Search Space

We use DFS almost all the time because this is the natural way to think when writing a recursive algorithm
- our task is often to produce all bitstrings of length n (remember our discussion in combinations)
- we build our solution incrementally
  - 0 -> 00 -> 000 -> 0000 -> 0001 -> etc
  - when we backtrack, we do so in a way that minimises the work we have to do
    - to build 01101 from 01100, it is easiest to backtrack to 0110

Traversing the Search Space

3.2 - Search Space

"a"

"ab"

"a"

"b"

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

... ...

backtrack

Summary

3.2 - Search Space

Constructing the search space
Problems and subproblems
Decision tree
Traversing using DFS