Big O

The measurement of Algorithmic Complexity

Objectives

Define Algorithmic Complexity
Describe the relationship between "algorithmic complexity", Big O, and "runtime" of a program.
Compute the Big O of an algorithm

Caveat to the Objectives

After today, you will probably only have "enough to be dangerous"

Every day this week we're going to practice computing the Big O of something for the warmup.

Practice makes perfect :)

Primer Questions

What is an algorithm?

Are all programs algorithms?

Primer Questions

What is the "runtime" of a program?

Primer Questions

How many operations will occur when I run this program?

var arr = [1, 2, 3, 4, 5];
var sum = 0;

while(i < arr.length) {
    sum += arr.length;
    i++;    
}

Primer Questions

2 + 3*5 = 17

var arr = [1, 2, 3, 4, 5];
var sum = 0;

while(i < arr.length) {
    sum += arr.length;  
    i++;  
}

At its heart, computing Big O is about counting the number of operations a program will run.

Big O -- Medium Formality

"A theoretical measure of the execution of an algorithm. Usually the time or memory needed for the algorithm to complete given the problem size n.

Big O

"A theoretical measure of the execution of an algorithm. Usually the time or memory needed for the algorithm to complete given the problem size n.

Theoretical: We're dealing with the abstraction, not necessarily a specific implementation.

Big O

"A theoretical measure of the execution of an algorithm. Usually the time or memory needed for the algorithm to complete given the problem size n.

Time or Memory: We can measure two categories of complexity.

Time complexity is the number of operations a program must execute before completing

Big O

"A theoretical measure of the execution of an algorithm. Usually the time or memory needed for the algorithm to complete given the problem size n.

Time or Memory: We can measure two categories of complexity.

Memory complexity is the amount of data that must be stored before the algorithm completes.

Big O

"A theoretical measure of the execution of an algorithm. Usually the time or memory needed for the algorithm to complete given the problem size n.

Problem Size n: The algorithmic complexity is based on the size of the input.

Big O -- Informal

An estimate of the runtime or storage requirements of an algorithm

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

reading the notation:

"F of n is said to be Big Oh of g of n"

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

This is full of "math-ese", which makes it a bit intimidating. Lets break it down with an example.

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

This equation is a statement about a function called f. Lets call this function f:

function f(arr) {
    var sum = 0;
    
    while(i < arr.length) {
        sum += arr.length;  
        i++;  
    }
}

Big O -- Formal

f(n) = O(g(n))

function f(arr) {
    var sum = 0;
    
    while(i < arr.length) {
        sum += arr.length; 
        i++;   
    }
}

n represents the "size of the input".

f( [1, 2, 3, 4] );          // n === 4
f( [1, 2, 3, 4, 5] );       // n === 5
f( [1, 2, 3, 4, 6, 7, 8] ); // n === 8

Big O -- Formal

f(n) = O(g(n))

Because the Big O notation is relative to the input size the Big O is a measure of how the algorithm scales.

function f(arr) {
    var sum = 0;
    
    while(i < arr.length) {
        sum += arr.length;
        i++; 
    }
}

Big O -- Formal

f(n) = O(g(n))

An equation for the operations run by this algorithm:

Ops = 1 + 3*arr.length

Ops = 1 + 3n

f(n) = O(c + kn)

f(n) = O(n)

function f(arr) {
    var sum = 0;
    
    while(i < arr.length) {
        sum += arr.length;
        i++; 
    }
}

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

Ops = 1 + 3n

f(n) == O(c + kn) == O(n)

f(n) is said to be O(n)

g(n) is of magnitude n

As soon as n > 3, the size of n "dominates" the equation 1+3*n

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

This "domination" is the core of Big O.

For every algorithm, there comes a point where the size of the input becomes more important than any other factor.

Big O -- Formal

f(n) = O(g(n)) means there are positive constants c and k, such that 0 ≤ f(n) ≤ cg(n) for all n ≥ k. The values of c and k must be fixed for the function f and must not depend on n.

Recall:

Ops = 1 + 3n

so, it is true that 0 ≤ f(n) ≤ cg(n)

for all n ≥ 4

k == 4, c == 1

"Asymptotic Complexity"

The reason we measure algorithms in Big O is to determine how they behave as n gets very large.

In this way, Big O is kind of like integrals in calculus. We care about the big o as n approaches infinity.

"Asymptotic Complexity"

The reason we measure algorithms in Big O is to determine how they behave as n gets very large.

In this way, Big O is kind of like integrals in calculus. We care about the big o as n approaches infinity.

"Asymptotic Complexity"

Questions?

Take A Break!

In 30 minutes, we are going to "learn by doing" and examine several programs to determine their big o values

function print50nums() {
  for (var i = 0; i < 50; i++) {
    console.log(i);
  }
}

function print500000nums() {
  for (var i = 0; i < 500000; i++) {
    console.log(i);
  }
}

function addSomeNumbers(arr) {
  sum = 0;
  for (var i=0; i < Math.min(arr.length,1e7); i++) {
    sum += arr[i];
  }
  return sum;
}

function printAllSubstrings(string) {
    for(var i = 0; i < string.length; i++) {
        for(var j = i; j < string.length; j++) {
            console.log(string.substring(i, j));
        }
    }
}

Binary Search (recursive)

function binarySearch(arr, lowIndex, highIndex, searchVal) {
    var sliceLength = highIndex - lowIndex;
    var midIndex = lowIndex + Math.floor(sliceLength / 2);
    var midVal = arr[midIndex];

    if(midVal === searchVal){
        return midIndex;
    }
    else if(lowIndex === highIndex) {
        return -1;
    }
    else if(searchVal < midVal) {
        return binarySearch(arr, lowIndex, midIndex - 1, searchVal);
    }
    else if(searchVal > midVal) {
        return binarySearch(arr, midIndex + 1, highIndex, searchVal);
    }
    else {
        throw new Error("Incompatible data -- no comparison returned true");
    }

    throw new Error("Reached what should be unreachable code.");
}

Binary Search (iterative)

function binaryIndexOf(searchElement) { 
    var minIndex = 0;
    var maxIndex = this.length - 1;
    var currentIndex;
    var currentElement;
 
    while (minIndex <= maxIndex) {
        currentIndex = (minIndex + maxIndex) / 2 | 0;
        currentElement = this[currentIndex];
 
        if (currentElement < searchElement) {
            minIndex = currentIndex + 1;
        }
        else if (currentElement > searchElement) {
            maxIndex = currentIndex - 1;
        }
        else {
            return currentIndex;
        }
    }
 
    return -1;
}

Iterative Fibonacci sequence

function fibbonacciIterative(n) {
	if(n <= 1) {
		return 1;
	}

	var fibNumber;
	var numberOne = 1;
	var numberTwo = 1;

	for(var i = 1; i < n; i++) {
		fibNumber = numberOne + numberTwo;
		numberOne = numberTwo;
		numberTwo = fibNumber;
	}

	return fibNumber;

}

Recursive Fibonacci sequence

function fibbonacciRecursive(n) {
	if(n <= 1) {
		return 1;
	}

	return fibbonacciRecursive(n - 1) + fibbonacciRecursive(n - 2);
}

Made with Slides.com