Sorting Algorithms
Introduction
Text
Why Sort?
Searching for Reason
There are several reasons we might want to keep our list sorted
The primary reason is that it makes accessing elements more time-efficient
How?
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# How would we do this?
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# How would we do this?
# Given a random ordering, the list could be scanned in
# sequence of increasing indices until we find whatever number
# we're looking for.
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# How would we do this?
# Given a random ordering, the list could be scanned in
# sequence of increasing indices until we find whatever number
# we're looking for. If our number is in the list we would
# expect to check roughly half the elements before locating it
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# How would we do this?
# Given a random ordering, the list could be scanned in
# sequence of increasing indices until we find whatever number
# we're looking for. If our number is in the list we would
# expect to check roughly half the elements before locating it
# Assuming no pattern in the order of the numbers, there is
# no faster way to search for our number
# Say we have a list of n unique numbers, in no particular
# order, from which we want to find a single, specific number
ex_list = [2, 5, 1, 9, 3, 8, 7, 0, 6, 4]
# How would we do this?
# Given a random ordering, the list could be scanned in
# sequence of increasing indices until we find whatever number
# we're looking for. If our number is in the list we would
# expect to check roughly half the elements before locating it
# Assuming no pattern in the order of the numbers, there is
# no faster way to search for our number
# *Note this is ~n/2 checks, which is in O(n) time
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great!
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great! We can expect our new algorithm to run twice as fast
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great! We can expect our new algorithm to run twice as fast
# But n/4 is still in O(n) time..
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great! We can expect our new algorithm to run twice as fast
# But n/4 is still in O(n) time.. can we do better?
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great! We can expect our new algorithm to run twice as fast
# But n/4 is still in O(n) time.. can we do better?
# Well we know the numbers are ordered,
# Now let's say our list of n unique numbers is sorted in
# increasing order prior to us searching for our element
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# We could try scanning each index, one after another
# Again, that would require ~n/2 checks..
# How might we leverage the fact our list is sorted?
# We could try checking every other element, backtracking if
# we ever overshoot
# Great! We can expect our new algorithm to run twice as fast
# But n/4 is still in O(n) time.. can we do better?
# Well we know the numbers are ordered, so what if we tried
# using bigger jumps between checks?
# Let's try a jump-size of 50.
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2,
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n),
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n), meaning neither would
# be ideal for searching extremely large datasets where search-
# performance can bottleneck our system efficiency
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n), meaning neither would
# be ideal for searching extremely large datasets where search-
# performance can bottleneck our system efficiency
# Say we have a list of size n = 10^20.
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n), meaning neither would
# be ideal for searching extremely large datasets where search-
# performance can bottleneck our system efficiency
# Say we have a list of size n = 10^20. With our jump-size of
# 50 we would still expect ~(10^18 + 25) checks
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n), meaning neither would
# be ideal for searching extremely large datasets where search-
# performance can bottleneck our system efficiency
# Say we have a list of size n = 10^20. With our jump-size of
# 50 we would still expect ~(10^18 + 25) checks
# Even if each check took 1 nanosecond we would still take
# about 30 YEARS to complete our search
# Let's try a jump-size of 50. That would mean ~n/100 jumps
# and ~25 checks in backtracking to find our number
# An expected runtime like n/100 + 25 would be an improvement
# over n/2, but both are still in O(n), meaning neither would
# be ideal for searching extremely large datasets where search-
# performance can bottleneck our system efficiency
# Say we have a list of size n = 10^20. With our jump-size of
# 50 we would still expect ~(10^18 + 25) checks
# Even if each check took 1 nanosecond we would still take
# about 30 YEARS to complete our search
# We have to find a better way!
Searching as Reason
So we've seen that simply increasing our jump-size does not improve our big-O complexity
For lists that are too large, we cannot decrease the runtime enough
And for lists that are too small, our jump would overshoot the list and devolve into a simple sequential search in backtracking
We could set our jump-size in respect to the initial size of our list
in a way that could actually improve our algorithmic complexity to O(√n)
But we'll save that problem for you all :)
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# What if we cut the list in half?
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# What if we cut the list in half?
# Our list is sorted, so if we start in the middle,
# we can ignore the half we know our number isn't in
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# What if we cut the list in half?
# Our list is sorted, so if we start in the middle,
# we can ignore the half we know our number isn't in
# Let's say, for example, we're looking for the number 6
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# What if we cut the list in half?
# Our list is sorted, so if we start in the middle,
# we can ignore the half we know our number isn't in
# Let's say, for example, we're looking for the number 6
# We would start at element located at the fifth index (4)
# and determine which half to search from there
[0, 1, 2, 3 _4_ 5, 6, 7, 8, 9]
# Let's revisit our previous example of a sorted list
ex_list = [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
# What if we cut the list in half?
# Our list is sorted, so if we start in the middle,
# we can ignore the half we know our number isn't in
# Let's say, for example, we're looking for the number 6
# We would start at element located at the fifth index (4)
# and determine which half to search from there
[0, 1, 2, 3 _4_ 5, 6, 7, 8, 9]
# We choose the second half since 6 > 4
[ 5, 6, 7, 8, 9 ]
# So what now?
[ 5, 6, 7, 8, 9 ]
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# This time look at the smaller half since 6 < 7
[ 5, 6 ]
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# This time look at the smaller half since 6 < 7
[ 5, 6 ]
[_5_ 6 ] # And again
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# This time look at the smaller half since 6 < 7
[ 5, 6 ]
[_5_ 6 ] # And again
[6]
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# This time look at the smaller half since 6 < 7
[ 5, 6 ]
[_5_ 6 ] # And again
[6]
# By the time we narrow down the list to one index,
# either that index contains our value or our number
# is not in the list.
# So what now?
[ 5, 6, 7, 8, 9 ]
# Well this is just another list like our original
# Use the same technique on this one
[ 5, 6 _7_ 8, 9 ]
# This time look at the smaller half since 6 < 7
[ 5, 6 ]
[_5_ 6 ] # And again
[6]
# By the time we narrow down the list to one index,
# either that index contains our value or our number
# is not in the list. This is called a binary search!
# So how well does our algorithm perform?
# So how well does our algorithm perform?
# We can answer this by calculating how many times we expect
# to halve our list before narrowing it down to one element
# So how well does our algorithm perform?
# We can answer this by calculating how many times we expect
# to halve our list before narrowing it down to one element
# Since each time we halve our list we only check the middle
# element, the total number of checks is equal to the number
# of times we expect to cut our list in half
# So how well does our algorithm perform?
# We can answer this by calculating how many times we expect
# to halve our list before narrowing it down to one element
# Since each time we halve our list we only check the middle
# element, the total number of checks is equal to the number
# of times we expect to cut our list in half
# This can be solved with a simple algebraic equation
n(1/2)^x = 1
# So how well does our algorithm perform?
# We can answer this by calculating how many times we expect
# to halve our list before narrowing it down to one element
# Since each time we halve our list we only check the middle
# element, the total number of checks is equal to the number
# of times we expect to cut our list in half
# This can be solved with a simple algebraic equation
n(1/2)^x = 1
# where x is the number of iterations
n(1/2)^x = 1
n(1/2)^x = 1
n = 2^x
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20)
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20) # note this is in base-2
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20) # note this is in base-2
x = ~66
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20) # note this is in base-2
x = ~66
# This is much better than our previous performance!
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20) # note this is in base-2
x = ~66
# This is much better than our previous performance!
# We just reduced our expected runtime from 30 years to 66ns
n(1/2)^x = 1
n = 2^x
lg(n) = lg(2^x)
x = lg(n)
# So now we've established our search runs in O(lg(n))
# let's test it on our example list of size n = 10^20
x = lg(10^20) # note this is in base-2
x = ~66
# This is much better than our previous performance!
# We just reduced our expected runtime from 30 years to 66ns
# That's the importance of algorithmic complexity and
# the performance improvement that a sorted list can offer
Bubble Sort
Pseudocode
Now that we've established the importance of sorting let's look at some popular algorithms
Bubble Sort is one of the easiest to conceptualize
The idea is to look at each adjacent pair of elements in sequence and swap them if they're out of order
With a single sweep through our entire list we should expect the largest element to 'bubble' all the way to the end
where it belongs
If we sweep through once more we should now expect the last two indices in our list to be sorted
Once more and the last three will be sorted
then four
then five
and so on
With enough sweeps, we would eventually 'bubble' the entire list into correct order
# Below is condensed pseudocode for the algorithm described
# using input list 'A'
n := length(A)
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
end-if
end-for
end-for
# Below is condensed pseudocode for the algorithm described
# using input list 'A'
n := length(A)
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
end-if
end-for
end-for
# := is used for assigning values
# Below is condensed pseudocode for the algorithm described
# using input list 'A'
n := length(A)
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
end-if
end-for
end-for
# := is used for assigning values
# = is reserved for equality comparisons
# Below is condensed pseudocode for the algorithm described
# using input list 'A'
n := length(A)
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
end-if
end-for
end-for
# := is used for assigning values
# = is reserved for equality comparisons
# We order our indices from 1 to n to make this more
# conceptually intuitive.
# Below is condensed pseudocode for the algorithm described
# using input list 'A'
n := length(A)
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
end-if
end-for
end-for
# := is used for assigning values
# = is reserved for equality comparisons
# We order our indices from 1 to n to make this more
# conceptually intuitive. In python the indices would be
# ordered from 0 to n-1
# So what's going on here?
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So what's going on here?
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# Our outer loop selects each index, in order from n-1 to 1,
# and assigns that to i
# So what's going on here?
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# Our outer loop selects each index, in order from n-1 to 1,
# and assigns that to i
# Each time our outer loop selects a new index,
# our inner loop selects indices, in order from 1 to i,
# and assigns that to j
# So what's going on here?
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# Our outer loop selects each index, in order from n-1 to 1,
# and assigns that to i
# Each time our outer loop selects a new index,
# our inner loop selects indices, in order from 1 to i,
# and assigns that to j
# Inside our inner loop, we compare the element at index
# j to the element at index j+1 and swap them if they are out
# of order
Example Case
Now that we've described our algorithm in pseudocode,
let's try running it on an example list
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 6, 2, 4, 3, 1 ]
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 6, 2, 4, 3, 1 ]
# i = 4
[_6_ _2_ 4 3 1 ] # j = 1
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 6, 2, 4, 3, 1 ]
# i = 4
[_6_ _2_ 4 3 1 ] # j = 1
[ 2 6 4 3 1 ] # Swap!
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 6, 2, 4, 3, 1 ]
# i = 4
[_6_ _2_ 4 3 1 ] # j = 1
[ 2 6 4 3 1 ] # Swap!
[ 2 _6_ _4_ 3 1 ] # j =2
# Note - the end-statements and length assignment
# have been removed for brevity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 6, 2, 4, 3, 1 ]
# i = 4
[_6_ _2_ 4 3 1 ] # j = 1
[ 2 6 4 3 1 ] # Swap!
[ 2 _6_ _4_ 3 1 ] # j =2
[ 2 4 6 3 1 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 6, 3, 1 ]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 6, 3, 1 ]
# i = 4
[ 2 4 _6_ _3_ 1 ] # j = 3
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 6, 3, 1 ]
# i = 4
[ 2 4 _6_ _3_ 1 ] # j = 3
[ 2 4 3 6 1 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 6, 3, 1 ]
# i = 4
[ 2 4 _6_ _3_ 1 ] # j = 3
[ 2 4 3 6 1 ] # Swap!
[ 2 4 3 _6_ _1_] # j = 4
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 6, 3, 1 ]
# i = 4
[ 2 4 _6_ _3_ 1 ] # j = 3
[ 2 4 3 6 1 ] # Swap!
[ 2 4 3 _6_ _1_] # j = 4
[ 2 4 3 1 6 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
[ 2 4 3 1 6 ]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
[ 2 4 3 1 6 ]
[ 2 _4_ _3_ 1 6 ] # j = 2
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
[ 2 4 3 1 6 ]
[ 2 _4_ _3_ 1 6 ] # j = 2
[ 2 3 4 1 6 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
[ 2 4 3 1 6 ]
[ 2 _4_ _3_ 1 6 ] # j = 2
[ 2 3 4 1 6 ] # Swap!
[ 2 3 _4_ _1_ 6 ] # j = 3
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
A = [ 2, 4, 3, 1, 6 ]
# i = 3
[_2_ _4_ 3 1 6 ] # j = 1
[ 2 4 3 1 6 ]
[ 2 _4_ _3_ 1 6 ] # j = 2
[ 2 3 4 1 6 ] # Swap!
[ 2 3 _4_ _1_ 6 ] # j = 3
[ 2 3 1 4 6 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
[ 2 3 1 4 6 ]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
[ 2 3 1 4 6 ]
[ 2 _3_ _1_ 4 6 ] # j = 2
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
[ 2 3 1 4 6 ]
[ 2 _3_ _1_ 4 6 ] # j = 2
[ 2 1 3 4 6 ] # Swap!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
[ 2 3 1 4 6 ]
[ 2 _3_ _1_ 4 6 ] # j = 2
[ 2 1 3 4 6 ] # Swap!
# i = 1
[_2_ _1_ 3 4 6 ] # j = 1
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# i = 2
[_2_ _3_ 1 4 6 ] # j = 1
[ 2 3 1 4 6 ]
[ 2 _3_ _1_ 4 6 ] # j = 2
[ 2 1 3 4 6 ] # Swap!
# i = 1
[_2_ _1_ 3 4 6 ] # j = 1
[ 1 2 3 4 6 ] # Swap!
Implementation
So we now have a feel for how Bubble Sort works
Let's take a look at the code in Python!
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
# 0:n-2
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
# 0:n-2 0:n-3
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
# 0:n-2 0:n-3 0:n-4
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
# 0:n-2 0:n-3 0:n-4 ... 0:0
def bubbleSort( A ):
n = len(A)
for i in range(n-1, -1, -1):
for j in range(i):
if A[ j ] > A[ j+1 ]:
temp = A[ j ]
A[ j ] = A[ j+1 ]
A[ j+1 ] = temp
return A
# Note- the first loop gives us n-1:0
# the second loop gives us 0:i-1
# this will iterate over the correct indices
# 0:n-2 0:n-3 0:n-4 ... 0:0
# Try it out for yourself and see!
Comparison-Analysis
Now let's try analyzing our algorithm
We want to get a better feel for its runtime complexity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense -
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons and Swaps
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons and Swaps
# Let's choose Comparisons for now since it's easier:
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons and Swaps
# Let's choose Comparisons for now since it's easier:
# Outer for-loop from i := n-1 to 1
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons and Swaps
# Let's choose Comparisons for now since it's easier:
# Outer for-loop from i := n-1 to 1
# Inner for-loop from j := 1 to i
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# First we need to define our unit of complexity
# We have two options that make sense - Comparisons and Swaps
# Let's choose Comparisons for now since it's easier:
# Outer for-loop from i := n-1 to 1
# Inner for-loop from j := 1 to i
# 1 comparison per iteration of the inner loop
# So there we have it!
# So there we have it!
# Given input list of size n, our bubble sorting algorithm
# is guaranteed to make the same number of comparisons,
# regardless of order
# So there we have it!
# Given input list of size n, our bubble sorting algorithm
# is guaranteed to make the same number of comparisons,
# regardless of order
# Based on comparisons our algorithmic complexity is:
(1/2)(n^2 - n) = O(n^2)
# So there we have it!
# Given input list of size n, our bubble sorting algorithm
# is guaranteed to make the same number of comparisons,
# regardless of order
# Based on comparisons our algorithmic complexity is:
(1/2)(n^2 - n) = O(n^2)
# But what about Swaps?
# So there we have it!
# Given input list of size n, our bubble sorting algorithm
# is guaranteed to make the same number of comparisons,
# regardless of order
# Based on comparisons our algorithmic complexity is:
(1/2)(n^2 - n) = O(n^2)
# But what about Swaps? Let's try and see!
Swap-Analysis
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order.
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
# Say we passed in a sorted list as input:
A = [1, 2, 3, 4]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
# Say we passed in a sorted list as input:
A = [1, 2, 3, 4]
# Then we would have 0 swaps!
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
# Say we passed in a sorted list as input:
A = [1, 2, 3, 4]
# Then we would have 0 swaps!
# What if we passed in a reversed list?
B = [4, 3, 2, 1]
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
# Say we passed in a sorted list as input:
A = [1, 2, 3, 4]
# Then we would have 0 swaps!
# What if we passed in a reversed list?
B = [4, 3, 2, 1]
# Then we would 6 swaps -
for i := n-1 to 1 do
for j := 1 to i do
if A[j] > A[j+1] do
swap(A[j], A[j+1])
# So let's try the analysis again, using Swaps instead of
# Comparisons as our metric of complexity
# This time it depends on the list-order. Can you see why?
# Say we passed in a sorted list as input:
A = [1, 2, 3, 4]
# Then we would have 0 swaps!
# What if we passed in a reversed list?
B = [4, 3, 2, 1]
# Then we would 6 swaps - 1 for each comparison we make!
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity.
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# Worst-case complexity is the # swaps we would need to sort a
# least-ideal input of size n
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# Worst-case complexity is the # swaps we would need to sort a
# least-ideal input of size n
# B is an example of a least-ideal input
B = [ 4, 3, 2, 1 ]
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# Worst-case complexity is the # swaps we would need to sort a
# least-ideal input of size n
# B is an example of a least-ideal input
B = [ 4, 3, 2, 1 ]
# It requires the maximum number of swaps by our algorithm
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# Worst-case complexity is the # swaps we would need to sort a
# least-ideal input of size n
# B is an example of a least-ideal input
B = [ 4, 3, 2, 1 ]
# It requires the maximum number of swaps by our algorithm
# So our Best-case complexity is 0 = O(0)
# This is a great segue into the topic of Best Case and Worst
# Case analyses of complexity. Given our context:
# Best-case complexity is the # swaps we would need to sort an
# ideal input of size n (ideal meaning fewest comparisons)
# A is an example of an ideal input
A = [ 1, 2, 3, 4 ]
# It requires no swaps since it is already sorted
# Worst-case complexity is the # swaps we would need to sort a
# least-ideal input of size n
# B is an example of a least-ideal input
B = [ 4, 3, 2, 1 ]
# It requires the maximum number of swaps by our algorithm
# So our Best-case complexity is 0 = O(0)
# And our Worst-case complexity is (1/2)(n^2 - n) = O(n^2)
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that?
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur.
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur. The first requires 4n
# swaps, the second requires 2n swaps, and the third requires
# 3n swaps.
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur. The first requires 4n
# swaps, the second requires 2n swaps, and the third requires
# 3n swaps. Our Average-case complexity, then, would be:
# (1/3)(4n + 2n + 3n)
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur. The first requires 4n
# swaps, the second requires 2n swaps, and the third requires
# 3n swaps. Our Average-case complexity, then, would be:
# (1/3)(4n + 2n + 3n) = 3n
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur. The first requires 4n
# swaps, the second requires 2n swaps, and the third requires
# 3n swaps. Our Average-case complexity, then, would be:
# (1/3)(4n + 2n + 3n) = 3n
# But we have much more than just 3 possible inputs.
# Since Best-case complexity is often not a good measure of
# an algorithm's expected runtime, computer scientists will
# typically use Worst-case or Average-case to describe
# complexity
# Average-case.. what's that? Well, given our context:
# Average-case complexity is the # swaps we expect we need to
# sort our list across all possible inputs of size of n
# Say, for eample, there were only three possible inputs of
# size n, each equally likely to occur. The first requires 4n
# swaps, the second requires 2n swaps, and the third requires
# 3n swaps. Our Average-case complexity, then, would be:
# (1/3)(4n + 2n + 3n) = 3n
# But we have much more than just 3 possible inputs. So to
# calculate our avg-case we must generalize the possible inputs
# Say we have an input of random order:
Q = [$, %, @, &]
# Say we have an input of random order:
Q = [$, %, @, &]
# Given any pair of two elements from our list, if they are
# out of order relative to one another, they must be swapped
# at some point during our algorithm
# Say we have an input of random order:
Q = [$, %, @, &]
# Given any pair of two elements from our list, if they are
# out of order relative to one another, they must be swapped
# at some point during our algorithm
# e.g. B = [ 4, 3, 2, 1 ]
#
#
#
# Say we have an input of random order:
Q = [$, %, @, &]
# Given any pair of two elements from our list, if they are
# out of order relative to one another, they must be swapped
# at some point during our algorithm
# e.g. B = [ 4, 3, 2, 1 ]
# 4 and 2 are in incorrect order in respect to one
# another.
#
# Say we have an input of random order:
Q = [$, %, @, &]
# Given any pair of two elements from our list, if they are
# out of order relative to one another, they must be swapped
# at some point during our algorithm
# e.g. B = [ 4, 3, 2, 1 ]
# 4 and 2 are in incorrect order in respect to one
# another. At one and only one point during our algorithm
# 4 must swap with 2 so we end up with a sorted list
# Say we have an input of random order:
Q = [$, %, @, &]
# Given any pair of two elements from our list, if they are
# out of order relative to one another, they must be swapped
# at some point during our algorithm
# e.g. B = [ 4, 3, 2, 1 ]
# 4 and 2 are in incorrect order in respect to one
# another. At one and only one point during our algorithm
# 4 must swap with 2 so we end up with a sorted list
# Stop and think. How might we use this observation to derive
# an expected number of swaps for our algorithm?
# Say we have an input of random order:
Q = [$, %, @, &]
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# C(n,2) / 2
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# C(n,2) / 2
# = n!/(2! * (n-2)!) / 2
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# C(n,2) / 2
# = n!/(2! * (n-2)!) / 2
# = n(n-1)/4
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# C(n,2) / 2
# = n!/(2! * (n-2)!) / 2
# = n(n-1)/4 = O(n^2)
# Say we have an input of random order:
Q = [$, %, @, &]
# Well we know that given an arbitrary pair of elements from
# our unsorted list, the likelihood of the two being in order,
# in respect to one another, is exactly 1/2
# Great! So based on the last two observations we can conclude
# the number of swaps we can expect is equal to half the total
# number of pair-combinations we have
# count([ $%, $@, $&, %@, %&, @& ]) / 2 = 3
# How can express this in terms of our input size n?
# C(n,2) / 2
# = n!/(2! * (n-2)!) / 2
# = n(n-1)/4 = O(n^2)
# Review permutations and combinations if you're rusty on them!
Summary
That was quite a bit we just went over for our analysis!
So we looked at Bubble-Sort in terms of both Comparisons and Swaps
For each of these perspectives we can perform a Best-case, Worst-case, or Average-case analysis for complexity
# Bubble Sort Runtime Analysis Summary
# Bubble Sort Runtime Analysis Summary
# Comparisons
# Best Case Worst Case Average Case
# n(n-1)/2 n(n-1)/2 n(n-1)/2
# O(n^2) O(n^2) O(n^2)
# Bubble Sort Runtime Analysis Summary
# Comparisons
# Best Case Worst Case Average Case
# n(n-1)/2 n(n-1)/2 n(n-1)/2
# O(n^2) O(n^2) O(n^2)
# Swaps
# Best Case Worst Case Average Case
# 0 n(n-1)/2 n(n-1)/4
# O(0) O(n^2) O(n^2)
Selection Sort
Pseudocode
Selection Sort is another intuitive sorting algorithm
The idea is to sweep through a list and identify the smallest element
Once you get to the end of the list, swap that smallest element with the element at the front of the list
On the next sweep, do the same, except ignore the front of the list, which contains the elements sorted on previous sweeps
Repeat until the whole list is sorted
# Below is the pseudocode for Selection Sort with input list A
n := length(A)
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
end-if
end-for
A[indexOfMin] := A[i]
A[i] := minVal
end-for
# Below is the pseudocode for Selection Sort with input list A
n := length(A)
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
end-if
end-for
A[indexOfMin] := A[i]
A[i] := minVal
end-for
# Note that, once again, we label indices starting from 1
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# So what exactly is happening here?
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# So what exactly is happening here?
# Our outer loop selects each index, in order from 1 to n-1,
# and assigns that to i
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# So what exactly is happening here?
# Our outer loop selects each index, in order from 1 to n-1,
# and assigns that to i
# At the start of each iteration we set our minVal to ∞
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# So what exactly is happening here?
# Our outer loop selects each index, in order from 1 to n-1,
# and assigns that to i
# At the start of each iteration we set our minVal to ∞
# Our inner loop then selects indices, in order from i to n,
# and assigns that to j
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# As we iterate through our inner loop, we compare the element
# at index j and update our minVal and its corresponding index
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# As we iterate through our inner loop, we compare the element
# at index j and update our minVal and its corresponding index
# By the time we finish checking all indices j = i:n we should
# be able to identify the minimum value of the batch
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal := A[j]
indexOfMin := j
A[indexOfMin] := A[i]
A[i] := minVal
# As we iterate through our inner loop, we compare the element
# at index j and update our minVal and its corresponding index
# By the time we finish checking all indices j = i:n we should
# be able to identify the minimum value of the batch
# We swap this minimum element with the first element of the
# sublist we just swept through
Example Case
We've described our algorithm in pseudocode
Now let's try running it on an example-list
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
[ 6 _2_ 4 3 1 ] # j = 2, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
[ 6 _2_ 4 3 1 ] # j = 2, minVal = 2
[ 6 2 _4_ 3 1 ] # j = 3, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
[ 6 _2_ 4 3 1 ] # j = 2, minVal = 2
[ 6 2 _4_ 3 1 ] # j = 3, minVal = 2
[ 6 2 4 _3_ 1 ] # j = 4, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
[ 6 _2_ 4 3 1 ] # j = 2, minVal = 2
[ 6 2 _4_ 3 1 ] # j = 3, minVal = 2
[ 6 2 4 _3_ 1 ] # j = 4, minVal = 2
[ 6 2 4 3 _1_] # j = 5, minVal = 1
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j] # assume we save by reference
swap(A[i], minVal)
A = [ 6, 2, 4, 3, 1 ]
# i = 1
[_6_ 2 4 3 1 ] # j = 1, minVal = 6
[ 6 _2_ 4 3 1 ] # j = 2, minVal = 2
[ 6 2 _4_ 3 1 ] # j = 3, minVal = 2
[ 6 2 4 _3_ 1 ] # j = 4, minVal = 2
[ 6 2 4 3 _1_] # j = 5, minVal = 1
[ 1 2 4 3 6 ] # Swap!
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
# i = 2
[ 1 _2_ 4 3 6 ] # j = 2, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
# i = 2
[ 1 _2_ 4 3 6 ] # j = 2, minVal = 2
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
# i = 2
[ 1 _2_ 4 3 6 ] # j = 2, minVal = 2
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 2
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
# i = 2
[ 1 _2_ 4 3 6 ] # j = 2, minVal = 2
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 2
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 2
[ 1 2 4 3 _6_] # j = 5, minVal = 2
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
A = [ 1, 2, 4, 3, 6 ]
# i = 2
[ 1 _2_ 4 3 6 ] # j = 2, minVal = 2
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 2
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 2
[ 1 2 4 3 _6_] # j = 5, minVal = 2
[ 1 2 4 3 6 ] # Assume we swap 2 with itself here
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
[ 1 2 4 3 _6_] # j = 5, minVal = 3
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
[ 1 2 4 3 _6_] # j = 5, minVal = 3
[ 1 2 3 4 6 ] # Swap
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
[ 1 2 4 3 _6_] # j = 5, minVal = 3
[ 1 2 3 4 6 ] # Swap
# i = 4
[ 1 2 3 _4_ 6 ] # j = 4, minVal = 4
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
[ 1 2 4 3 _6_] # j = 5, minVal = 3
[ 1 2 3 4 6 ] # Swap
# i = 4
[ 1 2 3 _4_ 6 ] # j = 4, minVal = 4
[ 1 2 3 4 _6_] # j = 5, minVal = 4
for i := 1 to n-1 do
minVal := ∞
for j := i to n do
if A[j] < minVal do
minVal <-- A[j]
swap(A[i], minVal)
# i = 3
[ 1 2 _4_ 3 6 ] # j = 3, minVal = 4
[ 1 2 4 _3_ 6 ] # j = 4, minVal = 3
[ 1 2 4 3 _6_] # j = 5, minVal = 3
[ 1 2 3 4 6 ] # Swap
# i = 4
[ 1 2 3 _4_ 6 ] # j = 4, minVal = 4
[ 1 2 3 4 _6_] # j = 5, minVal = 4
[ 1 2 3 4 6 ] # Self-swap
Video Example
Quicksort
Explanation
A lesson on Quicksort can be found here
Walkthrough
References
Why Sort?
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi nec metus justo. Aliquam erat volutpat.
Bubble Sort
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi nec metus justo. Aliquam erat volutpat.
Selection Sort
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi nec metus justo. Aliquam erat volutpat.
Quicksort
Lorem ipsum dolor sit amet, consectetur adipiscing elit. Morbi nec metus justo. Aliquam erat volutpat.
Lec 3 - Sorting Algorithms
By Brian J Lee
Lec 3 - Sorting Algorithms
- 710