Algorithms

Neeldhara Misra

29 Jan 2022

notable Omissions*

*There will be some cameos!

What is an algorithm?

CS50 Opening lecture

The word algorithm is derived from the name of the 9th-century Persian mathematician
Muḥammad ibn Mūsā al-Khwārizm.

In late medieval Latin, algorismus, English algorism, the corruption of his name, simply meant the decimal number system.

The machine side of the story:

Difference Engine, 1820's, Charles Babbage

The machine side of the story:

Difference Engine, 1820's, Charles Babbage

Ada Lovelace

(December 1815 – November 1852)
An English mathematician and writer, chiefly known for her work on Charles Babbage's proposed mechanical general-purpose computer, the Analytical Engine.

She is she is often regarded as the first computer programmer.

The machine side of the story:

Kelvin's harmonic analyser, 1878.
The machine was Invented by William Thomson, Lord Kelvin, (1824-1907), a pioneering Irish physicist.

The machine side of the story:

Automatic Computing Engine, designed by Alan Turing, built by Wilkinson and others
The Pilot ACE (Automatic Computing Engine) was one of the first computers built in the United Kingdom.

The machine side of the story:

A two-part Navy training film from 1953 that describes the inner workings of mechanical fire control computers. It covers seven mechanisms: shafts, gears, cams, differentials, component solvers, integrators, and multipliers, and does so in the well-executed fashion typical of the era. c.f: Retrotechtacular: Fire Control Computers in Navy Ships​

The machine side of the story:

Ars Technica

The machine side of the story:

C.f: a machine that does calculus

The machines will keep getting better.

The discipline of computing, however,
is not about the study of specific machines.

The machines will keep getting better.

Are there things that computers simply cannot do,

no matter how powerful?

The machines will keep getting better.

Are there things that computers simply cannot do,

no matter how powerful?

Theorem (Euclid, 300BC.)

It is possible to bisect an angle using ruler and compasses.

Theorem (Wantzel, 1837.)

It is impossible to trisect an angle using ruler and compasses.

Theorem (Euclid, 300BC.)

Mathematicians can prove that something is possible by doing it.

Theorem (Wantzel, 1837.)

It is impossible to trisect an angle using ruler and compasses.

Theorem (Euclid, 300BC.)

Mathematicians can prove that something is possible by doing it.

Theorem (Wantzel, 1837.)

They can also prove something is impossible by... *

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

1. Ask the user to type in a number.
2. Let X be that number. 
3. If X = 7 then go to step 4. Otherwise, go to step 6.
4. Go to step 5.
5. Go to step 4.
6. Print "Hello" on the screen.
7. Stop.

Ask yourself: is this is a good program when it takes itself as input?

If yes... then no

If yes... then no

If no... then yes

If no... then yes

😬

Around 1989, a striking letter written in March 1956 from Kurt Gödel to John von Neumann
came to light.

It poses some problems about the complexity of algorithms; in particular, it asks a question that can be seen as the first formulation of the P =? NP question.

The Church-Turing thesis says that
any real-world computation can be translated into an equivalent computation involving a Turing machine.

Takeaways

Logicians and computer scientists had started formalising what computers can or cannot do even before the machines were developed.

Hilbert identified and popularized the idea of equating understanding with efficiently executable procedures without explicitly defining computation.

Turing shattered Hilbert's first dream.

In mathematics and computer science, the Entscheidungsproblem is a challenge posed by David Hilbert and Wilhelm Ackermann in 1928.

The problem asks for an algorithm that considers, as input, a statement and answers Yes or No according to whether the statement is universally valid, i.e., valid in every structure satisfying the axioms.

*Avi Wigderson, Mathematics & Computation

〞

Shortly after the wall fell in 1990, an American and a Russian who had both worked on development of weapons meet.

The American asks: When you developed the bomb, how were you able to perform such an enormous amount of computing with your weak computers?

The Russian responds: We used better algorithms.

Source: Yefim Dinitz talk at Shimon Even's Party (2003); h/t: @manish_algomuse

[...] the labour required here is proportional to a power of the logarithm of the modulus, not to the modulus itself or its square root as in the indirect process, and hence see that

in the case of a large modulus the direct process will be much quicker than the indirect.

H. C. Pocklington
The determination of the exponent to which a number belongs, the practical solution of certain congruences, and the law of quadratic reciprocity
Proc. Cambridge Phil. Soc. 16 (1910), 1-5.

c.f: Lecture in the ACM Chennai series on Turing award winners by Prof. Meena Mahajan

Alan Cobham.
The intrinsic computational difficulty of functions.
1964 International Congress for Logic, Methodology and Philosophy of Science.

c.f: Lecture in the ACM Chennai series on Turing award winners by Prof. Meena Mahajan

Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.

c.f: Lecture in the ACM Chennai series on Turing award winners by Prof. Meena Mahajan

Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.

Source: The Humble Programmer, Edsger W. Dijkstra
ACM Turing Lecture 1972 & EWD 340

〞

Maximum matchings

Maximum Flows

&

Minimum Cuts

Air power is an effective means of interdicting an enemy’s rail system, and such usage is a logical and important mission for this Arm.

...

The present paper describes the fundamentals of a method intended to help the specialist who is engaged in estimating railway capabilities, so that he might more readily accomplish this purpose and thus assist the commander and his staff with greater efficiency than is possible at present.

October 24, 1955

Transformers

Recall the

maximum matching

problem.

ThE

IPL Elimination

Problem

It turns out that IPL elimination

reduces to

the MaxFlow problem.

Limits

of

Computation

P: The class of problems for which we can efficiently find solutions.

NP: The class of problems for which we can efficiently check solutions.

P: The class of problems for which we can efficiently find solutions.

NP-complete: The class of the most badass problems
in the class NP.

[Cook-Levin]

P: The class of problems for which we can efficiently find solutions.

NP-complete: Solve one to solve them all.
(It is incredible that such problems even exist!)

[Cook-Levin]

If you are struggling with finding a solution for your problem X, 

design a transformer from a NP-complete problem Y

to your problem X.

If you are struggling with finding a solution for your problem X, 

design a transformer from a NP-complete problem Y

to your problem X.

Is every problem in NP?

Suppose you want to convince me that you have a winning strategy from a given chess position.

Coping with
np-hardness

The Worst-case Approach:
Great Expectations

The Worst-case Approach:
Great Expectations

The Worst-case
Approach:
unreasonable Expectations

We want our algorithms to be:

We want our algorithms to be:

fast!

accurate!

all the time!

&

We want our algorithms to be:

fast!

accurate!

all the time!

&

We want our algorithms to be:

fast!

accurate!

all the time!

&

c.f: Lecture in the ACM Chennai series on Turing award winners by Prof. Meena Mahajan

Jack Edmonds.
Paths, Trees and Flowers.
Canadian J. Math. 17: 449467, 1965.

Perhaps the most famous failure of worst-case analysis concerns
linear programming, the problem of optimizing a linear function
subject to linear constraints.

Dantzig proposed in the 1940s an algorithm for solving linear programs called the simplex method. The simplex method solves linear programs using greedy local search on the vertices of the solution set boundary, and variants of it remain in wide use to this day.

The enduring appeal of the simplex method stems from its consistently superb performance in practice.

Its running time typically scales modestly with the input size, and it routinely solves linear programs with millions of decision variables and constraints.

This robust empirical performance suggested that the simplex method might well solve every linear program in a polynomial amount of time.

What can algorithms do for us?

What can algorithms do for us?

Thank you!