COMP3010 - 2.2 - Correctness of Recursive Algorithms

COMP3010: Algorithm Theory and Design

Daniel Sutantyo, Department of Computing, Macquarie University

2.2 Induction and Recursive Functions

Mathematical Induction

2.2 - Induction and Recursive Functions

Quick review:
- Prove a statement is true for all \(n \ge 1\)
  - or whatever really, as long as it's a sequence
- Base case:
  - show the statement is true for \(n = 1\) (or the start of the sequence)
- Induction step:
  - induction hypothesis: assume that the statement you want to prove is true for \(n = k > 1\), then show that it is also true for \(n = k+1\)
Informally: if the statement is true for \(1\) AND if it is true for any \(k >1\) then it is also true for \(k+1\) THEN it must be true for all integers

Mathematical Induction

We use induction in computer science because it is analogous to loops, and so, also to recursions
- loops commonly start at 0 or 1
  - \(0 \rightarrow 1 \rightarrow 2 \rightarrow 3 \rightarrow \cdots\)
- recursion always have base case
  - \(\dots \rightarrow 3 \rightarrow 2 \rightarrow 1 \rightarrow 0\)
- in both cases, we have a starting point (base case) and we keep on doing the same thing again and again

2.2 - Induction and Recursive Functions

Mathematical Induction

For this lecture we are going to use induction to prove correctness of simple algorithms that use recursive functions
- For algorithms that use a loop, we are going to use loop invariant instead, but both are based on the same principle (don't get them mixed up)
Don't forget that to prove that an algorithm is correct, you also need to show that it terminates
The most important takeaway is the logic behind the method
- you may not remember how to do this years from now, but you should remember how it works

2.2 - Induction and Recursive Functions

. . .

Fibonacci:

. . .

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)

Is the algorithm correct for the base case?
- yes, we can see that factorial(1) = 1 as required
Will the algorithm terminate?
- yes, at each recursive call, we reduce n by one, therefore we will eventually reach the base case

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)

Induction step:
- Induction hypothesis: assume that the algorithm is correct for some value n = k > 1, that is, factorial(k) does return k!
- Does the algorithm work for n = (k+1)?

2.2 - Induction and Recursive Functions

Factorial

factorial(n):
if (n <= 1) 
  return 1
else 
  return n * factorial(n-1)

If n = k+1 > 1 then the algorithm will return
- (k+1) * factorial(k) (Line 5)
From our induction hypothesis, we assumed factorial(k) = k!, so this means that the algorithm will return (k+1)k! = (k+1)!
Therefore, by induction, factorial(n) returns n! for all n >= 1

2.2 - Induction and Recursive Functions

Factorial

In summary:
- base case: we show that factorial(1) is correct
- induction step:
  - we assume that factorial(k) is correct
  - we show that if factorial(k) is correct, then factorial(k+1) is also correct
    - and so factorial(n) is correct for all positive values of n

2.2 - Induction and Recursive Functions

Polynomial Evaluation

Let us now see a less trivial example
- Given a polynomial of degree \(n\)
  
  write a recursive algorithm that will evaluate the polynomial
- How would you write the algorithm?

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

Suppose you have the following polynomial:

write a recursive algorithm that will evaluate the polynomial

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

(disclaimer: if you think that this is a slow algorithm, you are correct)

2.2 - Induction and Recursive Functions

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

\[\begin{aligned} i &= 0 &\rightarrow\ &c_0\\ i &= 1 &\rightarrow\ &c_1x\\ & \vdots & \vdots & \\ i &= n &\rightarrow\ &c_nx^n\end{aligned}\]

2.2 - Induction and Recursive Functions

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[\text{evaluate}(x,0) = c_0\]

\[\text{evaluate}(x,1) = c_1x + c_0\]

\[\text{evaluate}(x,n) = c_nx^n + c_{n-1}x^{n-1} + \cdots + c_1x + c_0\]

\[\vdots\]

\[\begin{aligned} i &= 0 &\rightarrow\ &c_0\\ i &= 1 &\rightarrow\ &c_1x\\ & \vdots & \vdots & \\ i &= n &\rightarrow\ &c_nx^n\end{aligned}\]

\[ f(x) = c_nx^n + c_{n-1}x^{n-1} + \cdots +c_1x + c_0 \]

We need to prove that our algorithm is correct, i.e. for all \(i\)
The base case is when \(i = 0\), is the algorithm correct?
For the induction hypothesis, we assume our algorithm is true for \(i = k\), \(0 < k < n\)
For the induction step, we show that if our algorithm is correct for \(i = k\), then it is also true for \(i = k+1\)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

\[\text{evaluate}(x,0) = c_0\]

\[\text{evaluate}(x,i) = c_ix^i + c_{i-1}x^{i-1} + \cdots + c_1x + c_0\]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

For the induction hypothesis, we assume our algorithm is true for \(i = k\), \(0 < k < n\)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

2.2 - Induction and Recursive Functions

Polynomial Evaluation

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=3\)

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=4\)

\[ f(x) = 4x^5 + 3x^4 + 12x^3 + x^2 + 15x + 7 \]

\(i=2\)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

Is the algorithm correct for the base case?
- yes, when \(i = 0\), this algorithm returns \(c_0\), thus \(\text{evaluate}(x,0) = c_0\) as required
Will the algorithm terminate?
- yes, at each recursive call we reduce \(i\) by one, therefore it will eventually reach the base case

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

Induction step
- Induction hypothesis: assume the algorithm is correct for some value \(i = k, 0 < k < n\), that is

2.2 - Induction and Recursive Functions

Polynomial Evaluation

\[\text{evaluate}(x,k) = c_kx^k + c_{k-1}x^{k-1} + \cdots + c_1x + c_0\]

Induction step
- Induction hypothesis: assume the algorithm is correct for some value \(i = k, 0 < k < n\), that is

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

we need to prove that

2.2 - Induction and Recursive Functions

Polynomial Evaluation

if \(i = (k+1)\), then \(t = x^{k+1}\)
from our induction hypothesis:
\(\text{evaluate}(x,k) = c_kx^k + \cdots + c_1x + 1\)
which means that on line 8 we compute
\(c_{k+1} x^{k+1} + c_kx^k + \cdots + c_1x + 1\)
thus

\[\text{evaluate}(x,k+1) = c_{k+1}x^{k+1} + c_{k}x^{k} + \cdots + c_1x + c_0\]

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

In summary:
- base case: we show that \(\text{evaluate}(x,0)\) is correct
- induction step:
  - we assume \(\text{evaluate}(x,k)\) is correct
  - we show that \(\text{evaluate}(x,k+1)\) is also correct
    - so \(\text{evaluate}(x,n)\) is correct and will give us \(f(x)\)

2.2 - Induction and Recursive Functions

Polynomial Evaluation

evaluate(x,i):
 if (i == 0) 
   return c_0
 else 
   t = 1
   for j = 1 to i:
     t = t * x
   return c_i * t + evaluate(x,i-1)

can you suggest a simple improvement to speed up this algorithm?