COMP3010 - 1.2 - Complexity Analysis Introduction

Daniel Sutantyo, Department of Computing, Macquarie University

1.2 - Complexity Analysis

1.2 - Complexity Analysis

You don't measure the speed of an algorithm by just using a stopwatch
If we want to compare two algorithms fairly, then you should run both of them in the same environment
You definitely cannot say an algorithm is better just because you wrote it in a better language (e.g. Assembly vs Javascript)

... or can you?

... how about single-threaded vs multi-threaded algorithm?

1.2 - Complexity Analysis

Generally, to compare two algorithms fairly, we need to strip down our computer to its most basic components and simplify them

CPU

Memory

Input/Output

Let's assume that we have an abstract computer with a CPU, memory, and I/O. The speed doesn't matter, what matters is, what they can do.

1.2 - Complexity Analysis

Historically, we use a hypothetical simple computer called the Random Access Machine or RAM
Properties of RAM:
- we have primitive operations that take one time step each:
  \(+\) , \( -\) , \(*\) , \(\div\) , function calls, logical tests (\(<\), \(=\), \(>\), etc)
- unlimited memory (but each memory block has a size limit), memory access takes one time step
- instructions are executed one after another, i.e. no concurrency
- loops and subroutines are compositions of simple operations
https://en.wikipedia.org/wiki/Random-access_machine if you want to find out more

1.2 - Complexity Analysis

The Art of Computer Programming

Donald E. Knuth

The imaginary MIX computer

1.2 - Complexity Analysis

Basically, you have a computer that can only do primitive operations, then break down your algorithm to those operations and see how many operations they need to complete the computation
If you can think in terms of C or Assembly (COMP2100/202), then you should have a good idea.
If not, think of the average Java program that you wrote back in COMP1010/125
Problem: is it fair?
- single-threaded vs multi-threaded
Problem: is it realistic?
- multiplication vs addition

1.2 - Complexity Analysis

Is it perfect? No, but it allows us to have a simple model of our algorithm that we can analyse
The idea is, if in our simple model we see that Algorithm A is better than Algorithm B, then if we apply Algorithm A in the real world, it will probably still be better than Algorithm B

1.2 - Complexity Analysis

In practice, you don't have to worry too much about it. It just means that when you are analysing an algorithm, you have to strip away the more complex operations, e.g
- Sorting doesn't take just one time step
- TreeMap.find(key) doesn't take just one time step
- What about computing \(2^n\) ?
One more time: the running time or time complexity of an algorithm is the number of primitive operations that is executed by the algorithm

1.2 - Complexity Analysis

1.2 - Complexity Analysis

Time complexity (or just complexity) of an algorithm is usually represented with the notation \(T(n)\) as seen in the example
The good (?) news is that in this unit, we are not going to do something as rigorous as what you saw in the previous slide
In most cases, it is enough to identify the most significant operation to derive the complexity of the algorithm

\(T(n) \approx n^2\)

1.2 - Complexity Analysis

Analysing an algorithm can be hard, so we need to use a simplified model
To analyse an algorithm, break it down to its primitive operations and count the number of operation it takes to finish