COMP3010: Algorithm Theory and Design

Daniel Sutantyo,  Department of Computing, Macquarie University

2.2 Induction and Recursive Functions

Mathematical Induction

2.2 - Induction and Recursive Functions

  • Quick review:
    • Prove a statement is true for all \(n \ge 1\)
      • or whatever really, as long as it's a sequence
    • Base case: 
      • show the statement is true for \(n = 1\) (or the start of the sequence)
    • Induction step:
      • induction hypothesis: assume that the statement you want to prove is true for \(n = k > 1\), then show that it is also true for \(n = k+1\)
  • Informally: if the statement is true for \(1\) AND if it is true for any \(k >1\) then it is also true for \(k+1\) THEN it must be true for all integers

Mathematical Induction

  • We use induction in computer science because it is analogous to loops, and so, also to recursions
    • loops commonly start at 0 or 1 
      • \(0 \rightarrow 1 \rightarrow 2 \rightarrow 3 \rightarrow \cdots\)
    • recursion always have base case
      • \(\dots \rightarrow 3 \rightarrow 2 \rightarrow 1 \rightarrow 0\)
    • in both cases, we have a starting point (base case) and we keep on doing the same thing again and again

2.2 - Induction and Recursive Functions

Mathematical Induction

  • For this lecture we are going to use induction to prove correctness of simple algorithms that use recursive functions
    • For algorithms that use a loop, we are going to use loop invariant instead, but both are based on the same principle (don't get them mixed up)
  • Don't forget that to prove that an algorithm is correct, you also need to show that it terminates
  • The most important takeaway is the logic behind the method
    • you may not remember how to do this years from now, but you should remember how it works

2.2 - Induction and Recursive Functions

2.2 - Induction and Recursive Functions

3

1

2

4

. . .

  • Fibonacci:

3

1

2

4

2

1

5

. . .

5

3

2

1

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)
  • Is the algorithm correct for the base case?
    • yes, we can see that factorial(1) = 1 as required
  • Will the algorithm terminate?
    • yes, at each recursive call, we reduce n by one, therefore we will eventually reach the base case

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)
  • Induction step:
    • Induction hypothesis: assume that the algorithm is correct for some value n = k > 1, that is, factorial(k) does return k!
    • Does the algorithm work for n = (k+1)?

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)
  • If n = k+1 > 1 then the algorithm will return
    • (k+1) * factorial(k) (Line 5) 
  • From our induction hypothesis, we assumed factorial(k) = k!, so this means that the algorithm will return (k+1)k! = (k+1)!
  • Therefore, by induction, factorial(n) returns n! for all n >= 1

2.2 - Induction and Recursive Functions

Factorial

  • In summary:
    • base case: we show that factorial(1) is correct
    • induction step:
      • we assume that factorial(k) is correct
      • we show that if factorial(k) is correct, then factorial(k+1) is also correct
        • and so factorial(n) is correct for all positive values of n

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • Let us now see a less trivial example
    • Given a polynomial of degree \(n\)


      write a recursive algorithm that will evaluate the polynomial
       
    • How would you write the algorithm?

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • Suppose you have the following polynomial:
     

    write a recursive algorithm that will evaluate the polynomial

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

(disclaimer: if you think that this is a slow algorithm, you are correct)

2.2 - Induction and Recursive Functions

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

\[\begin{aligned} i &= 0 &\rightarrow\ &c_0\\ i &= 1 &\rightarrow\ &c_1x\\ & \vdots & \vdots & \\ i &= n &\rightarrow\ &c_nx^n\end{aligned}\]

2.2 - Induction and Recursive Functions

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[\text{evaluate}(x,0) = c_0\]

\[\text{evaluate}(x,1) = c_1x + c_0\]

\[\text{evaluate}(x,n) = c_nx^n + c_{n-1}x^{n-1} + \cdots + c_1x + c_0\]

\[\vdots\]

\[\begin{aligned} i &= 0 &\rightarrow\ &c_0\\ i &= 1 &\rightarrow\ &c_1x\\ & \vdots & \vdots & \\ i &= n &\rightarrow\ &c_nx^n\end{aligned}\]

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

  • We need to prove that our algorithm is correct, i.e. for all \(i\)

     
  • The base case is when \(i = 0\), is the algorithm correct?

     
  • For the induction hypothesis, we assume our algorithm is true for \(i = k\), \(0 < k < n\)

     
  • For the induction step, we show that if our algorithm is correct for \(i = k\), then it is also true for \(i = k+1\)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

\[\text{evaluate}(x,0) = c_0\]

\[\text{evaluate}(x,i) = c_ix^i + c_{i-1}x^{i-1} + \cdots + c_1x + c_0\]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • For the induction hypothesis, we assume our algorithm is true for \(i = k\), \(0 < k < n\)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=3\)

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=4\)

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=2\)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • Is the algorithm correct for the base case?
    • yes, when \(i = 0\), this algorithm returns \(c_0\), thus \(\text{evaluate}(x,0) = c_0\) as required
  • Will the algorithm terminate?
    • yes, at each recursive call we reduce \(i\) by one, therefore it will eventually reach the base case
evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

  • Induction step
    • Induction hypothesis: assume the algorithm is correct for some value \(i = k,  0 < k < n\), that is 

2.2 - Induction and Recursive Functions

Polynomial Evaluation

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

  • Induction step
    • Induction hypothesis: assume the algorithm is correct for some value \(i = k,  0 < k < n\), that is 

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

  • we need to prove that

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • if \(i = (k+1)\), then \(t = x^{k+1}\)
  • from our induction hypothesis:
    ​          \(\text{evaluate}(x,k) = c_kx^k + \cdots + c_1x + 1\)
  • which means that on line 8 we compute
               \(c_{k+1} x^{k+1} + c_kx^k + \cdots + c_1x + 1\)
  • thus

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

  • In summary:
    • base case: we show that \(\text{evaluate}(x,0)\) is correct
    • induction step:
      • we assume \(\text{evaluate}(x,k)\) is correct
      • we show that \(\text{evaluate}(x,k+1)\) is also correct
        • so \(\text{evaluate}(x,n)\) is correct and will give us \(f(x)\)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)
  • can you suggest a simple improvement to speed up this algorithm?

COMP3010 - 2.2 - Correctness of Recursive Algorithms

By Daniel Sutantyo

COMP3010 - 2.2 - Correctness of Recursive Algorithms

Using induction to show correctness of recursive algorithms

  • 268

More from Daniel Sutantyo