COMP3010: Algorithm Theory and Design

Daniel Sutantyo,  Department of Computing, Macquarie University

2.3 Loop Invariant

Prelude

2.3 - Loop Invariant

  • Informally, a loop invariant is a statement about a loop that is true before and after each iteration (including the first and the last)
  • It assumed that you have used loop invariant before, since COMP225/2010 is a prerequisite for COMP3010
    • ​but I also understand that this is the one topic that most students find confusing (after years of tutoring COMP225/2010)
  • For this lecture, we will only talk about loop invariant on an informal basis
  • The aim is for you to better understand them, so that we can discuss it more formally in the next lecture

Definition

2.3 - Loop Invariant

  • Formally, a loop invariant must satisfy the following properties:
    • Initialisation: the loop invariant is true before the first iteration of the loop
    • Maintenance: if the loop invariant is true at the start of one iteration of the loop, then it is also true at the start of the next iteration
    • Termination: when the loop terminates, the loop invariant gives a useful property that can be used to show that the algorithm is correct

Informal Definition

2.3 - Loop Invariant

  • A statement that is true:
    • before the first iteration
    • at the end of any iteration, if it's true at the start of that iteration
    • at the very end of the loop
  • What do I mean by 'a statement'?
    • must be relevant to what the loop is trying to accomplish
    • most of the time, it's exactly what the loop is doing at every iteration

Informal Definition

2.3 - Loop Invariant

  • "Pikachu is an electric Pokémon" is a statement that is correct when you start your loop, at any point in the loop, and when the loop finishes
    • but it has no relevance to most loops
    • so whenever you write a loop invariant, think, am I writing a Pikachu?
  • Do you just write down what the loop is doing?
    • No, that's not a loop invariant
      • (but you just said so in the previous slide?)
        • (no, I didn't)
          • (yea, you did)
            • (no, go rewind)

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  while (i < n){       // loop guard
     ans = ans * x; 
     i = i + 1;   
  }
  return ans;
}
  • What is the code doing?
    • if you don't understand what the loop is doing, then you wouldn't be able to work out what the loop invariant is
    • conversely, if you know what the loop is doing, then you probably know what the loop invariant is

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  while (i < n){       // loop guard
     ans = ans * x; 
     i = i + 1;   
  }
  return ans;
}
  • What is the loop invariant?
    • the loop invariant is \(i < n\)
    • i is always between \(0\) and \(n\)
    • the loop computes \(x^n\)
    • ans is \(x^n\)

no, that's the loop guard

how is this relevant to what the loop is doing?

this is what the loop is doing

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  for(int i = 0; i < n; i++){
  	ans = ans * x;
  }
  return ans;
}
  • There really is only one line in the loop: Line 5
  • What is the loop doing?
    • Computes \(x^n\)
  • So, why is that not a loop invariant?
    • because it's not true at the start of the loop

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  for(int i = 0; i < n; i++){
  	ans = ans * x;
  }
  return ans;
}
  • A loop invariant is a statement that is relevant to what the loop is trying to accomplish
    • since it is a loop, we need several iterations before we get to our goal
    • each iteration brings you closer to the goal
    • so your statement can 'change' after each iteration, because it also 'progresses'

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  for(int i = 0; i < n; i++){
  	ans = ans * x;
  }
  return ans;
}
  • So, ans = \(x^n\) is not the loop invariant
  • The correct loop invariant is
    • ans = \(x^i\)
      • is this true at the start of the loop?
      • if this is true at the start of an iteration, is it true at the end of that iteration?

Example: Exponentiation

2.3 - Loop Invariant

// compute x^n 
int exponentiate(int x, int n){
  int i = 0, ans = 1;  // initialisation
  for(int i = 0; i < n; i++){
  	ans = ans * x;
  }
  return ans;
}
  • Before the first iteration
    • i = 0, ans = 1
    • so ans = \(x^i\)
  • At the start of iteration k
    • i = k, ans = \(x^k\)
    • so ans = \(x^i\)
  • At the end of iteration k
    • i = k+1, ans = \(x^{k+1}\)
    • so ans = \(x^i\)
  • Loop invariant: ans = \(x^i\)
    • is it relevant to what the loop is trying to accomplish?
    • does it kinda tell you the progress with your computation?
  • At the very end
    • i = n, ans = \(x^{n}\)
    • so ans = \(x^n\)

Example: Selection Sort

2.3 - Loop Invariant

  • What is the loop invariant?
    • which loop?
void selection_sort(int arr[]){
  for (int i = 0; i < n-1; i++){
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }
    swap(i,min_index);
  }
}

Example: Find Max

2.3 - Loop Invariant

  • what are we doing in this loop?
  • where do we put the result (of what we're doing)?
  • how about this?
    • min_index is the index of the smallest element in arr
  • how about this?
    • min_index is the index of the smallest element in arr[ i .. n-1 ]
  • how about this?
    • min_index is the index of the smallest element in arr[ i .. j-1 ]
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }

Example: Find Max

2.3 - Loop Invariant

  • min_index is the index of the smallest element in arr[ i .. j-1 ]
    • is this true before the loop starts?
      • j = i + 1, so min_index is the index of the smallest element in arr[ i .. i ]
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }

Example: Find Max

2.3 - Loop Invariant

  • min_index is the index of the smallest element in arr[ i .. j-1 ]
    • assume true at the start of the loop (j = k)
      • assume min_index is the index of the smallest element in arr [i .. k-1]
      • is it true at the end of the iteration?
      • in that iteration, we look at arr[k], and see if it's smaller than arr[min_index], and if it is, we set min_index = k
    • so at the end of that iteration, j = k+1
      • we have min_index is the index of the smallest element in arr [i .. k]
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }

Example: Find Max

2.3 - Loop Invariant

  • min_index is the index of the smallest element in arr[ i .. j-1 ]
    • at the very end, j = n
    • so the loop invariant reads
      • min_index is the index of the smallest element in arr [ i .. n-1 ]
      • and that is what we're looking for
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }

Example: Selection Sort

2.3 - Loop Invariant

  • What is the loop invariant?
    • again, we want to say something relevant, so we should have at least the words sort and array
    • how about "the array is sorted"
    • how about "the array is sorted up to index i"?
void selection_sort(int arr[]){
  for (int i = 0; i < n-1; i++){
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }
    swap(i,min_index);
  }
}

Example: Selection Sort

2.3 - Loop Invariant

void selection_sort(int arr[]){
  for (int i = 0; i < n-1; i++){
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }
    swap(i,min_index);
  }
}
  • Loop invariant: The array arr[0 .. i-1] is sorted
    • at beginning: The array arr[0 .. -1] is sorted
    • at start of iteration: The array arr[0 .. k-1] is sorted
      • at end of iteration: The array arr[0 .. k] is sorted
    • at the end: The array arr[0 .. n-2] is sorted

Example: Selection Sort

2.3 - Loop Invariant

void selection_sort(int arr[]){
  for (int i = 0; i < n-1; i++){
    int min_index = i;
    for (int j = i+1; j < n; j++){
      if (arr[j] < arr[min_index])
        min_index = j;
    }
    swap(i,min_index);
  }
}
  • what we have so far is pretty good, but it's still missing something, and we'll explore that in the next lecture

COMP3010 - 2.3 - Loop Invariants

By Daniel Sutantyo

COMP3010 - 2.3 - Loop Invariants

Revision of loop invariant topic from COMP2010/225

  • 148

More from Daniel Sutantyo