history of ideas
Algorithms
Neeldhara Misra
29 Jan 2022
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
notable Omissions*
*There will be some cameos!
What is an algorithm?
CS50 Opening lecture
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
Division algorithms were used by
ancient Babylonian mathematicians c. 2500 BC
&
Egyptian mathematicians c. 1550 BC
Greek mathematicians later used algorithms in 240 BC in the sieve of Eratosthenes for finding prime numbers, and the Euclidean algorithm for finding the greatest common divisor of two numbers.
Kuṭṭaka is an algorithm for finding integer solutions of linear Diophantine equations. The algorithm was originally invented by the Indian astronomer-mathematician Āryabhaṭa (476–550 CE) and is described very briefly in his Āryabhaṭīya.
Hamiltonian cycles and paths in the knight's graph of the chessboard, the knight's tour, had been studied in the 9th century in Indian mathematics by Rudrata, and around the same time in Islamic mathematics by al-Adli ar-Rumi.
The Fibonacci numbers were first described in Indian mathematics, as early as 200 BC in work by Pingala on enumerating possible patterns of Sanskrit poetry formed from syllables of two lengths.
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
The word algorithm is derived from the name of the 9th-century Persian mathematician
Muḥammad ibn Mūsā al-Khwārizm.
In late medieval Latin, algorismus, English algorism, the corruption of his name, simply meant the decimal number system.
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
The machine side of the story:
The machine side of the story:
Charles Babbage began to construct a small difference engine in c. 1819 and had completed it by 1822.
He announced his invention on 14 June 1822, in a paper to the Royal Astronomical Society, entitled:
Note on the application of machinery to the computation of astronomical and mathematical tables
In 1823, the British government gave Babbage £1700 to start work on the project. Although Babbage's design was feasible, the metalworking techniques of the era could not economically make parts in the precision and quantity required. Thus the implementation proved to be much more expensive and doubtful of success than the government's initial estimate.
By the time the government abandoned the project in 1842, Babbage had received and spent over £17,000 on development, which still fell short of achieving a working engine.
Ada Lovelace
(December 1815 – November 1852)
An English mathematician and writer, chiefly known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine.
She is she is often regarded as the first computer programmer.
Wolfram argues that there's nothing as sophisticated—or as clean—as Ada's computation of the Bernoulli numbers. Babbage certainly helped and commented on Ada's work, but she was definitely the driver of it.
Wolfram then suggests that Lovelace's main achievement was to distill from Babbage's correspondence "a clear exposition of the abstract operation of the machine—something which Babbage never did."
The machine side of the story:
Kelvin's harmonic analyser, 1878.
The machine was Invented by William Thomson, Lord Kelvin, (1824-1907), a pioneering Irish physicist.
The machine side of the story:
Automatic Computing Engine, designed by Alan Turing, built by Wilkinson and others
The Pilot ACE (Automatic Computing Engine) was one of the first computers built in the United Kingdom.
The machine side of the story:
A two-part Navy training film from 1953 that describes the inner workings of mechanical fire control computers. It covers seven mechanisms: shafts, gears, cams, differentials, component solvers, integrators, and multipliers, and does so in the well-executed fashion typical of the era. c.f: Retrotechtacular: Fire Control Computers in Navy Ships
The machine side of the story:
The machine side of the story:
C.f: a machine that does calculus
The machines will keep getting better.
The discipline of computing, however,
is not about the study of specific machines.
Computer science is no more about computers
than astronomy is about telescopes.
〞
Edsger W. Dijkstra
c.f. The Life and Times of Edsger W. Dijkstra
The machines will keep getting better.
Are there things that computers simply cannot do,
no matter how powerful?
The machines will keep getting better.
Are there things that computers simply cannot do,
no matter how powerful?
Theorem (Euclid, 300BC.)
It is possible to bisect an angle using ruler and compasses.
Theorem (Wantzel, 1837.)
It is impossible to trisect an angle using ruler and compasses.
Theorem (Euclid, 300BC.)
Mathematicians can prove that something is possible by doing it.
Theorem (Wantzel, 1837.)
It is impossible to trisect an angle using ruler and compasses.
Theorem (Euclid, 300BC.)
Mathematicians can prove that something is possible by doing it.
Theorem (Wantzel, 1837.)
They can also prove something is impossible by... *
1. Ask the user to type in a number. 2. Let X be that number. 3. If X = 7 then go to step 4. Otherwise, go to step 6. 4. Go to step 5. 5. Go to step 4. 6. Print "Hello" on the screen. 7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
When a computer program gets stuck in an infinite loop, it has crashed.
This makes it a bad computer program.
And let's call a computer program good if it never gets stuck in an infinite loop.
Turing: How do we check to see if a computer program is good?
Turing in 1936 turned this question into a pure mathematics question,
and then found the surprising solution.
Suppose you had a program that could check
if a program crashes on a given input or not.
It turns out that this assumption
leads to a contradiction!
Ask yourself: is this is a good program when it takes itself as input?
If yes... then no
If yes... then no
If no... then yes
If no... then yes
😬
Around 1989, a striking letter written in March 1956 from Kurt Gödel to John von Neumann
came to light.
It poses some problems about the complexity of algorithms; in particular, it asks a question that can be seen as the first formulation of the P =? NP question.
The Church-Turing thesis says that
any real-world computation can be translated into an equivalent computation involving a Turing machine.
Takeaways
Logicians and computer scientists had started formalising what computers can or cannot do even before the machines were developed.
Hilbert identified and popularized the idea of equating understanding with efficiently executable procedures without explicitly defining computation.
Turing shattered Hilbert's first dream.
In mathematics and computer science, the Entscheidungsproblem is a challenge posed by David Hilbert and Wilhelm Ackermann in 1928.
The problem asks for an algorithm that considers, as input, a statement and answers Yes or No according to whether the statement is universally valid, i.e., valid in every structure satisfying the axioms.
*Avi Wigderson, Mathematics & Computation
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
〞
Shortly after the wall fell in 1990, an American and a Russian who had both worked on development of weapons meet.
The American asks: When you developed the bomb, how were you able to perform such an enormous amount of computing with your weak computers?
The Russian responds: We used better algorithms.
[...] the labour required here is proportional to a power of the logarithm of the modulus, not to the modulus itself or its square root as in the indirect process, and hence see that
in the case of a large modulus the direct process will be much quicker than the indirect.
H. C. Pocklington
The determination of the exponent to which a number belongs, the practical solution of certain congruences, and the law of quadratic reciprocity
Proc. Cambridge Phil. Soc. 16 (1910), 1-5.
Alan Cobham.
The intrinsic computational difficulty of functions.
1964 International Congress for Logic, Methodology and Philosophy of Science.
Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.
Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.
Source: The Humble Programmer, Edsger W. Dijkstra
ACM Turing Lecture 1972 & EWD 340
〞
[...of a software crisis...]
Maximum matchings
Maximum Flows
&
Minimum Cuts
Screengrab from: CACM August 2014 Feature Video Efficient Maximum Flow Algorithms
Air power is an effective means of interdicting an enemy’s rail system, and such usage is a logical and important mission for this Arm.
...
The present paper describes the fundamentals of a method intended to help the specialist who is engaged in estimating railway capabilities, so that he might more readily accomplish this purpose and thus assist the commander and his staff with greater efficiency than is possible at present.
October 24, 1955
Transformers
Recall the
maximum matching
problem.
ThE
IPL Elimination
Problem
It turns out that IPL elimination
reduces to
the MaxFlow problem.
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
Limits
of
Computation
P: The class of problems for which we can efficiently find solutions.
NP: The class of problems for which we can efficiently check solutions.
P: The class of problems for which we can efficiently find solutions.
NP-complete: The class of the most badass problems
in the class NP.
[Cook-Levin]
P: The class of problems for which we can efficiently find solutions.
NP-complete: Solve one to solve them all.
(It is incredible that such problems even exist!)
[Cook-Levin]
If you are struggling with finding a solution for your problem X,
design a transformer from a NP-complete problem Y
to your problem X.
If you are struggling with finding a solution for your problem X,
design a transformer from a NP-complete problem Y
to your problem X.
Is every problem in NP?
Suppose you want to convince me that you have a winning strategy from a given chess position.
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
Coping with
np-hardness
The Worst-case Approach:
Great Expectations
The Worst-case Approach:
Great Expectations
The Worst-case
Approach:
unreasonable Expectations
We want our algorithms to be:
fast!
accurate!
all the time!
&
We want our algorithms to be:
fast!
accurate!
all the time!
&
exact
algorithms
We want our algorithms to be:
fast!
accurate!
all the time!
&
approximation
algorithms
We want our algorithms to be:
fast!
accurate!
all the time!
&
Randomized algorithms
Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.
Perhaps the most famous failure of worst-case analysis concerns
linear programming, the problem of optimizing a linear function
subject to linear constraints.
Dantzig proposed in the 1940s an algorithm for solving linear programs called the simplex method. The simplex method solves linear programs using greedy local search on the vertices of the solution set boundary, and variants of it remain in wide use to this day.
The enduring appeal of the simplex method stems from its consistently superb performance in practice.
Its running time typically scales modestly with the input size, and it routinely solves linear programs with millions of decision variables and constraints.
This robust empirical performance suggested that the simplex method might well solve every linear program in a polynomial amount of time.
Antiquity
Algorithmic thinking in
ancient math
&
warfare
Origins
Al-Khwārizmī
&
the decimal number system
Mind & machine
Entscheidungsproblem
David Hilbert
v/s
Church–Turing thesis
Hard Problems
[Cook & Levin]
A million dollars
for solving
Sudoku
P: The holy Grail
A digression
for Edmonds
lays the foundations
for algorithms
MIND V. MACHINE Redux
A computer once
beat me at chess,
but it was no match
for me at kickboxing.
Coming up...
Weapons
of
math
destruction
What can algorithms do for us?
What can algorithms do for us?
