*history **of **ideas*

### Algorithms

### Neeldhara Misra

### 29 Jan 2022

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

## notable Omissions*

**There will be some cameos!*

# What is an algorithm?

### CS50 Opening lecture

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

Division algorithms were used by

ancient Babylonian mathematicians c. 2500 BC

&

Egyptian mathematicians c. 1550 BC

Greek mathematicians later used algorithms in 240 BC in the sieve of Eratosthenes for finding prime numbers, and the Euclidean algorithm for finding the greatest common divisor of two numbers.

Kuṭṭaka is an algorithm for finding integer solutions of linear Diophantine equations. The algorithm was originally invented by the Indian astronomer-mathematician Āryabhaṭa (476–550 CE) and is described very briefly in his Āryabhaṭīya.

Hamiltonian cycles and paths in the knight's graph of the chessboard, the knight's tour, had been studied in the 9th century in Indian mathematics by Rudrata, and around the same time in Islamic mathematics by al-Adli ar-Rumi.

The Fibonacci numbers were first described in Indian mathematics, as early as 200 BC in work by Pingala on enumerating possible patterns of Sanskrit poetry formed from syllables of two lengths.

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

The word algorithm is derived from the name of the 9th-century Persian mathematician

Muḥammad ibn Mūsā al-Khwārizm.

In late medieval Latin, algorismus, English *algorism*, the corruption of his name, simply meant the decimal number system.

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

The machine side of the story:

The machine side of the story:

Charles Babbage began to construct a small difference engine in c. 1819 and had completed it by 1822.

He announced his invention on 14 June 1822, in a paper to the Royal Astronomical Society, entitled:

*Note on the application of machinery to the computation of astronomical and mathematical tables*

In 1823, the British government gave Babbage £1700 to start work on the project. Although Babbage's design was feasible, the metalworking techniques of the era could not economically make parts in the precision and quantity required. Thus the implementation proved to be much more expensive and doubtful of success than the government's initial estimate.

By the time the government abandoned the project in 1842, Babbage had received and spent over £17,000 on development, which still fell short of achieving a working engine.

## Ada Lovelace

(December 1815 – November 1852)

An English mathematician and writer, chiefly known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine.

She is she is often regarded as the first computer programmer.

Wolfram argues that *there's nothing as sophisticated—or as clean—as Ada's computation of the Bernoulli numbers. Babbage certainly helped and commented on Ada's work, but she was definitely the driver of it.*

Wolfram then suggests that Lovelace's main achievement was to distill from Babbage's correspondence "a clear exposition of the abstract operation of the machine—something which Babbage never did."

The machine side of the story:

Kelvin's harmonic analyser, 1878.

The machine was Invented by William Thomson, Lord Kelvin, (1824-1907), a pioneering Irish physicist.

The machine side of the story:

Automatic Computing Engine, designed by Alan Turing, built by Wilkinson and others

The Pilot ACE (Automatic Computing Engine) was one of the first computers built in the United Kingdom.

The machine side of the story:

A two-part Navy training film from 1953 that describes the inner workings of mechanical fire control computers. It covers seven mechanisms: shafts, gears, cams, differentials, component solvers, integrators, and multipliers, and does so in the well-executed fashion typical of the era. c.f: Retrotechtacular: Fire Control Computers in Navy Ships

The machine side of the story:

The machine side of the story:

C.f: a machine that does calculus

The machines will keep getting better.

The discipline of computing, however,

is *not *about the study of specific machines.

Computer science is no more about computers

than astronomy is about telescopes.

# 〞

Edsger W. Dijkstra

c.f. The Life and Times of Edsger W. Dijkstra

The machines will keep getting better.

Are there things that computers simply cannot do,

no matter how powerful?

The machines will keep getting better.

Are there things that computers simply cannot do,

no matter how powerful?

**Theorem (Euclid, 300BC.)**

It is possible to bisect an angle using ruler and compasses.

**Theorem (Wantzel, 1837.)**

It is impossible to trisect an angle using ruler and compasses.

**Theorem (Euclid, 300BC.)**

Mathematicians can prove that something is possible by doing it.

**Theorem (Wantzel, 1837.)**

It is impossible to trisect an angle using ruler and compasses.

**Theorem (Euclid, 300BC.)**

Mathematicians can prove that something is possible by doing it.

**Theorem (Wantzel, 1837.)**

They can also prove something is impossible by... *

1. Ask the user to type in a number. 2. Let X be that number. 3. If X = 7 then go to step 4. Otherwise, go to step 6. 4. Go to step 5. 5. Go to step 4. 6. Print "Hello" on the screen. 7. Stop.

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

```
1. Ask the user to type in a number.
2. Let X be that number.
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.
```

When a computer program gets stuck in an infinite loop, it has crashed.

This makes it a bad computer program.

And let's call a computer program good if it never gets stuck in an infinite loop.

Turing: *How do we check to see if a computer program is good?*

Turing in 1936 turned this question into a pure mathematics question,

and then found the surprising solution.

*Suppose you had a program that could check*

*if a program crashes on a given input or not.*

**It turns out that this assumption**

**leads to a contradiction!**

Ask yourself: is *this *is a good program when it takes *itself as input*?

If yes... then no

If yes... then no

If no... then yes

If no... then yes

😬

Around 1989, a striking letter written in March 1956 from Kurt Gödel to John von Neumann

came to light.

It poses some problems about the complexity of algorithms; in particular, it asks a question that can be seen as the first formulation of the P =? NP question.

The Church-Turing thesis says that

any real-world computation can be translated into an equivalent computation involving a Turing machine.

# Takeaways

Logicians and computer scientists had started formalising what computers can or cannot do even before the machines were developed.

Hilbert identified and popularized the idea of equating *understanding *with efficiently executable procedures without explicitly defining computation.

*Turing shattered Hilbert's first dream.*

In mathematics and computer science, the Entscheidungsproblem is a challenge posed by David Hilbert and Wilhelm Ackermann in 1928.

The problem asks for an algorithm that considers, as input, a statement and answers **Yes** or **No** according to whether the statement is universally valid, i.e., valid in every structure satisfying the axioms.

*Avi Wigderson, Mathematics & Computation

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

# 〞

Shortly after the wall fell in 1990, an American and a Russian who had both worked on development of weapons meet.

The American asks: *When you developed the bomb, how were you able to perform such an enormous amount of computing with your weak computers?*

The Russian responds: *We used better algorithms.*

[...] the labour required here is proportional to a power of the logarithm of the modulus, not to the modulus itself or its square root as in the indirect process, and hence see that

in the case of a large modulus the direct process will be much quicker than the indirect.

H. C. Pocklington

*The determination of the exponent to which a number belongs, the practical solution of certain congruences, and the law of quadratic reciprocity*

Proc. Cambridge Phil. Soc. 16 (1910), 1-5.

Alan Cobham.

*The intrinsic computational difficulty of functions.*

1964 International Congress for Logic, Methodology and Philosophy of Science.

Jack Edmonds.

*Paths, Trees and Flowers.*

Canadian J. Math. 17: 449467, 1965.

Jack Edmonds.

*Paths, Trees and Flowers.*

Canadian J. Math. 17: 449467, 1965.

Source: The Humble Programmer, Edsger W. Dijkstra

ACM Turing Lecture 1972 & EWD 340

# 〞

`[...of a software crisis...]`

### Maximum matchings

### Maximum Flows

### &

### Minimum Cuts

Screengrab from: CACM August 2014 Feature Video Efficient Maximum Flow Algorithms

Air power is an effective means of interdicting an enemy’s rail system, and such usage is a logical and important mission for this Arm.

...

The present paper describes the fundamentals of a method intended to help the specialist who is engaged in estimating railway capabilities, so that he might more readily accomplish this purpose and thus assist the commander and his staff with greater efficiency than is possible at present.

`October 24, 1955`

### Transformers

Recall the

*maximum matching *

problem.

### ThE

### IPL Elimination

### Problem

It turns out that IPL elimination

*reduces to*

the MaxFlow problem.

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

# Limits

# of

# Computation

P: The class of problems for which we can efficiently find solutions.

NP: The class of problems for which we can efficiently *check* solutions.

P: The class of problems for which we can efficiently find solutions.

NP-complete: The class of the most badass problems

in the class NP.

[Cook-Levin]

P: The class of problems for which we can efficiently find solutions.

NP-complete: Solve one to solve them all.

(It is *incredible *that such problems even exist!)

[Cook-Levin]

If you are struggling with finding a solution for your problem X,

design a transformer from a NP-complete problem Y

to your problem X.

If you are struggling with finding a solution for your problem X,

design a transformer from a NP-complete problem Y

to your problem X.

Is every problem in NP?

Suppose you want to convince me that you have a winning strategy from a given chess position.

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction

# Coping with

np-hardness

# The Worst-case Approach:

Great Expectations

# The Worst-case Approach:

Great Expectations

# The Worst-case

Approach:

unreasonable Expectations

We want our algorithms to be:

fast!

accurate!

all the time!

&

We want our algorithms to be:

fast!

accurate!

all the time!

&

### exact

### algorithms

We want our algorithms to be:

fast!

accurate!

all the time!

&

### approximation

### algorithms

We want our algorithms to be:

fast!

accurate!

all the time!

&

# Randomized algorithms

Jack Edmonds.

*Paths, Trees and Flowers.*

Canadian J. Math. 17: 449467, 1965.

Perhaps the most famous failure of worst-case analysis concerns

linear programming, the problem of optimizing a linear function

subject to linear constraints.

Dantzig proposed in the 1940s an algorithm for solving linear programs called the **simplex method**. The simplex method solves linear programs using greedy local search on the vertices of the solution set boundary, and variants of it remain in wide use to this day.

The enduring appeal of the simplex method stems from its *consistently superb performance in practice*.

Its running time typically scales modestly with the input size, and it routinely solves linear programs with millions of decision variables and constraints.

This robust empirical performance suggested that the simplex method might well solve every linear program in a polynomial amount of time.

## Antiquity

Algorithmic thinking in

ancient math

&

warfare

## Origins

Al-Khwārizmī

&

the decimal number system

## Mind & machine

*Entscheidungsproblem*

David Hilbert

v/s

Church–Turing thesis

## Hard Problems

[Cook & Levin]

A million dollars

for solving

Sudoku

## P: The holy Grail

A *digression *

for Edmonds

lays the foundations

for algorithms

## MIND V. MACHINE Redux

A computer once

beat me at chess,

but it was no match

for me at kickboxing.

## Coming up...

Weapons

of

math

destruction