COMP3010: Algorithm Theory and Design

Daniel Sutantyo,  Department of Computing, Macquarie University

3.2 - Search Space

Search Space

3.2 - Search Space

  • Does this term apply on for search algorithms? 
    • Another term: solution space
  • Definition : the set of all possible solutions
    • so what about "impossible" solutions?
  • Examples:​
    • Search if a value exist in an array
      • the search space is every element of the array
      • [ 3 , 5 , 7 , 8 , 10 ]
    • Knapsack, Travelling Salesman, SAT, etc: all possible combinations
      • some will fail the constraint

Example: Making Rods

3.2 - Search Space

We have a bunch of metallic rods with fixed lengths. Is it possible to produce a rod of certain length if we are allowed to join any two or more rods together, but we cannot cut any rod into two or more pieces.

11

12

7

5

10

Example: Making Rods

3.2 - Search Space

  • The input to the problem is the lengths of the metallic rods that we have and the length \(L\) of the rod that we need to make
  • Example:
    • input = [ 10 , 12 , 5 , 7 , 11 ], required length = 25
    • output = yes or no

We have a bunch of metallic rods with fixed lengths. Is it possible to produce a rod of certain length if we are allowed to join any two or more rods together, but we cannot cut any rod into two or more pieces.

Example: Making Rods

3.2 - Search Space

We have a bunch of metallic rods with fixed lengths. Is it possible to produce a rod of certain length if we are allowed to join any two or more rods together, but we cannot cut any rod into two or more pieces.

  • Input: The array \(B = \{b_1, b_2, \dots, b_n\}\) containing the lengths of the rods and an integer \(L \ge 0\)
  • Output: True if there is a subarray \(\{b_{i_0}, b_{i_1}, \dots, b_{i_k}\}, 1 \le i_j \le n\) such that

\[\sum_{j=0}^k b_{i_j} = L\]

Example: Making Rods

3.2 - Search Space

  • Sample input: [ 10 , 12 , 5 , 7 , 11 ]
    • L = 25, there is no solution
    • L = 22, there are multiple solutions, e.g. [ 10 , 12 ],  [ 10 , 5 , 7 ]
  • The search space has \(2^5 = 32\) combinations, but how do we actually find them?
for (int i = 0; i < b.length; i++){
  for (int j = 0; j < b.length; j++){
    for (int k = 0; k < b.length; k++){
      for (int l = 0; l < b.length; l++){
         for (int m = 0; m < b.length; m++){
            ...
}}}}}

Problems and Subproblems

3.2 - Search Space

  • One of the most important thing you need to recognise when attempting to solve a problem is, can we make a smaller subproblem out of the original problem?
  • Recognising the subproblem will lead to us being able to construct a search space
  • We are going to look at this again next week, but for now, try to recognise the subproblems to the original problem

Problems and Subproblems

3.2 - Search Space

  • Given [10, 12, 5, 7, 11], can we make 25?
    • reduce the problem to subproblems
    • if I pick 10 now, then I have [ 12 , 5 , 7 , 11 ] to choose from, and have to make 15
    • if I pick 12 now, then I have [ 10 , 5 , 7 , 11 ] to choose from, and have to make 13
    • if I pick 5 now, then I have [ 10 , 12 , 7 , 11 ] to choose from, and have to make 20
    • etc

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11]   L = 25

[12,5,7,11]   L = 15

[10,5,7,11]   L = 13

[10,12,7,11]   L = 20

[10,12,5,11]   L = 18

[10,12,5,7]   L = 14

Problems and Subproblems

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0) 
    return true;
  // if we don't have any more rods, then we can't make it
  if (b.length == 0)
    return false;
  
  // else keep on going with the remaining sum
  for (???) {
    // HALP
  }
}

Problems and Subproblems

3.2 - Search Space

  • Given [10, 12, 5, 7, 11], can we make 25?
    • Option 1: pick one of the elements, then recurse
      • pick one of 10, 12, 5, 7, or 11
    • Option 2: either pick the element at the current index, or don't, then recurse
      • pick 10 or don't pick it
      • pick 12 or don't pick it
      • pick 5 or don't pick it

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11]   L = 25

[12,5,7,11]   L = 15

[12,5,7,11]   L = 25

don't pick 10

pick 10

[5,7,11]   L = 3

[5,7,11]   L = 15

[5,7,11]   L = 15

[5,7,11]   L = 25

don't pick 12

pick 12

pick 12

don't pick 12

[7,11]   L = -2

[11]   L = -9

pick 5

pick 7

...

[7,11]   L = 15

[11]   L = 8

pick 7

don't pick 5

don't pick 5

...

...

...

...

don't pick 7

...

[11]   L = 15

[7,11]   L = 15

pick 5

...

...

don't pick 7

10

12

5

7

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11]   L = 25

don't pick 10

pick 10

[10,12,5,7,11]   L = 25

[12,5,7,11]   L = 15

[10,5,7,11]   L = 13

[10,12,7,11]   L = 20

[10,12,5,11]   L = 18

[10,12,5,7]   L = 14

[12,5,7,11]   L = 15

[12,5,7,11]   L = 25

Problems and Subproblems

3.2 - Search Space

  • Moral of the story: make sure you understand whether you need to generate permutations or combinations

Example: Word building

3.2 - Search Space

  • Another example: produce all possible combinations of three unique letters in the English alphabet
    • e.g. abc, abd, abe, ..., ayz, bac, bad, ..., xyz 

Example: Word building

3.2 - Search Space

"a"

""

"ab"

"a"

"b"

""

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

""

""

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

""

... ...

... ...

(and so on)

Example: Word building

3.2 - Search Space

"a"

"b"

"z"

"c"

""

"d"

... ...

""

"a"

"b"

"z"

"c"

"d"

... ...

"a"

"b"

"z"

"c"

"d"

... ...

Example: Word building

3.2 - Search Space

"a"

"b"

"z"

"c"

""

"d"

... ...

""

"a"

"b"

"y"

"c"

"d"

... ...

"b"

"c"

"z"

"d"

... ...

Tree Structure

3.2 - Search Space

  • Notice that the search space is actually a tree
    • I often refer to this as the decision tree or search tree 
    • Is it always a binary tree?

Example: License Plates

3.2 - Search Space

  • Problem: Generate all possible license plates

"a"       "b"      "c"            .....        "9"    "0"

"a"       "b"      "c"            .....        "9"    "0"

  • Search space: 36 * 36 * 36 * 36 * 36 * 36

Problems and Subproblems

3.2 - Search Space

[10,12,5,7,11]   L = 25

[12,5,7,11]   L = 15

[12,5,7,11]   L = 25

don't pick 10

pick 10

[5,7,11]   L = 3

[5,7,11]   L = 15

[5,7,11]   L = 15

[5,7,11]   L = 25

don't pick 12

pick 12

pick 12

don't pick 12

[7,11]   L = -2

[11]   L = -9

pick 5

pick 7

...

[7,11]   L = 15

[11]   L = 8

pick 7

don't pick 5

don't pick 5

...

...

...

...

don't pick 7

...

[11]   L = 15

[7,11]   L = 15

pick 5

...

...

don't pick 7

10

12

5

7

Traversing the Search Space

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0){
    return true;
  }
  // if we get to the end of array but still can't make the sum, 
  // return false
  if (i >= b.length) {
    return false;
  }
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}
  • if (L == 0) or (i >= b.length), we are done since either we managed to make the sum earlier, or we've run out of rods
  • solve(b,index+1,L-b[index]): pick the current element, keep going
  • solve(b,index+1,L): don't pick the current element, keep going

Traversing the Search Space

3.2 - Search Space

  • If your search space is a tree, then how will we traverse through it (or more generally, how do you traverse the graph)?
  • You can use either DFS or BFS
  • What did we do here?
public static boolean solve(int[] b, int i, int L) {
  // if L == 0, we are done
  if (L == 0){
    return true;
  }
  // if we get to the end of array but still can't make the sum, 
  // return false
  if (i >= b.length) {
    return false;
  }
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}

Traversing the Search Space

3.2 - Search Space

public static boolean solve(int[] b, int i, int L) {
  ...
  // else keep on going with the remaining sum
  return solve(b,i+1,L-b[i]) || solve(b,i+1,L);
}

[10,12,5,7,11]   L = 25

[12,5,7,11]   L = 15

[12,5,7,11]   L = 25

don't pick 10

pick 10

don't pick 12

pick 12

pick 12

don't pick 12

[5,7,11]   L = 3

[5,7,11]   L = 15

[5,7,11]   L = 15

[5,7,11]   L = 25

Traversing the Search Space

3.2 - Search Space

"a"

""

"ab"

"a"

"b"

""

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

""

""

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

... ...

... ...

... ...

Traversing the Search Space

3.2 - Search Space

  • We use DFS almost all the time because this is the natural way to think when writing a recursive algorithm
    • our task is often to produce all bitstrings of length n (remember our discussion in combinations)
    • we build our solution incrementally
      • 0 -> 00 -> 000 -> 0000 -> 0001 -> etc
      • when we backtrack, we do so in a way that minimises the work we have to do
        • to build 01101 from 01100, it is easiest to backtrack to 0110

Traversing the Search Space

3.2 - Search Space

"a"

""

"ab"

"a"

"b"

""

"abc"

"ab"

"ac"

"a"

"bc"

"b"

"c"

a ?

b ?

c ?

""

""

d ?

"abd"

"ab"

"bcd"

"bc"

"d"

""

... ...

... ...

backtrack

Summary

3.2 - Search Space

  • Constructing the search space
  • Problems and subproblems
  • Decision tree
  • Traversing using DFS