CS329 · Foundations of AI

Week 1. Introduction

large language models · stochastic parrots · attention is all you need · artificial general intelligence · killer robots · transformers · the singularity  · neural networks · backpropagation · few-shot learning · zero-shot learning · fairness · bias

Logistics

Course Website
https://www.neeldhara.courses/2024-CS329

Google Classroom Code: hstszfz

for announcements and discussions

Grading Policy

5 Quizzes. 15 points each, capped at 70
Group projects, evaluated by presentation. 10 points
Endsem. 20 points

The quizzes will be:

pen-and-paper · open book · open notes
no internet · no discussions

endsem format: TBA

Practice worksheets or mentimeter questions will not be evaluated.

Be it resolved, AI research and development poses an existential threat.

Be it resolved, AI research and development poses an existential threat.

Some three thousand years ago in China, the strategic board game Go was developed. Some believe warlords and generals based it on the stones they'd place on maps to determine their battle plans. Besides being the oldest continually played board game in human history, it's also one of the most complex.

In modern times, beating this game became known in the artificial intelligence community as the holy grail. Since the number of possible configurations on the board is larger than the number of atoms in the universe, it was believed computers didn't have the processing power needed to beat a skilled human player.

In move 37 of the second match, the machine was faced with a decision that would determine the way the rest of the game would be played. There were two apparent choices to be made. Choice A was the kind of move that would signal the computer was playing a game of offense. Choice B would signal it was playing a defensive game.

Instead, the computer decided to make a third move, a move no one steeped in the game had ever made in thousands of years of play. "Not a single human player would choose move 37," one commentator said. Most thought it was a mistake or simply a bad move.

The grandmaster playing against the machine was so taken aback, he stood up and walked out of the room. He eventually returned, not with his usual confident composure but visibly shaken and frustrated by the experience. In the end, AlphaGo won the game. And that never-been-seen-before move, experts said, was the one that turned the course of the game in favor of the Al. In the end, the computer won four out of five matches, and the grandmaster [Lee Sedol] permanently retired from competition.

The Creative Act: A Way of Being · Rick Rubin

What does it mean to be intelligent?

I think, therefore I am.

cogito ergo sum

René Descartes (1596-1650)

What does it mean to be intelligent?

I think, therefore I am.

cogito ergo sum

René Descartes (1596-1650)

Principle of dualism - that the mind exists separately from the body:
if a foot or arm or any other part of the body is cut off,
nothing has thereby taken away from the mind.

Descartes wondered if he could know for sure that others
who looked and behaved like him were not in fact automata.

This was a serious practical issue back in the 17th century!

What does it mean to be intelligent?

The Turk

What does it mean to be intelligent?

Maillardet's automaton, housed at The Franklin Institute in Philadelphia.

As you might guess:
the idea of machines that mimic human behaviour
goes way back!

In the Iliad by Homer, there is a description of the god of technology, Hephaestus, who has at his beck and call a group of machines. They are described as golden maidens who are able to help in building and constructing the shield of Achilles.

Ancient Indian texts have various descriptions of machines too.
Yantra Sarvasva, credited to Acharya Bharadvaj is all about machines.
The vedas and Indian epics have descriptions of machines too.

What does it mean to be intelligent?

1800's: all three of Jacques de Vaucanson's Automata.
The Flute Player, The Tambourine Player, and Digesting Duck

What does it mean to be intelligent?

What does it mean to be intelligent?

Are there imaginable digital computers which would do well in the imitation game?

Alan Turing (1912-1954)

The Imitation Game was a party game where two people would go off to different rooms in the house and other people at the party would interact with them by sending them questions to answer.

 The people who are hidden away in the other rooms would type back their answers and the other players would have to guess just based on these typewritten responses, who was who.

What does it mean to be intelligent?

from Artificial Intelligence: A Modern Approach

Machines

Humans

automata, robots, etc.

prosthetics, neuralink, etc.

Warren McCulloch and Walter Pitts (1943) were the first to suggest that neural activity is computational.

 They argued that neural computations explain cognition.

 The theory was proposed in its modern form by Hilary Putnam in 1967, and developed by his PhD student, philosopher, and cognitive scientist Jerry Fodor in the 1960s, 1970s, and 1980s.

 It was vigorously disputed in analytic philosophy in the 1990s due to work by Putnam himself, John Searle, and others.

Computational Theory of Mind

narrow/weak artificial intelligence
Limited to a single, narrowly defined task.
Most modern AI systems would be classified in this category.

artificial general intelligence (AGI)
A machine with the ability to apply at least human-level intelligence to any problem, rather than just one specific problem as with narrow AI.

artificial superintelligence (ASI)
A machine with a vastly superior intelligence to the average human being.

artificial consciousness
A machine that has consciousness, sentience and mind
(John Searle uses "strong AI" in this sense).

Types of AI

(warning: terminology informal and not necessarily widely agreed upon)

Books and Stories about Mind/Matter

J. Offray de la Mettrie · Machine Man (1747)

René Descartes · Treatise of Man (1633)

The Sandman (short story) · E. T. A. Hoffmann (1816)

R.U.R. (play) · Karel Čapek (1920)

Gödel, Escher, Bach · Douglas Hofstadter (1979)

Where Am I? (short story) · Daniel Dennett (1978)

Thomas Hobbes · Leviathan (1651)

What Is ChatGPT Doing... and Why Does It Work? · Stephen Wolfram (2023)

Philosophy

Confluences in AI

Mathematics

Economics

Neuroscience

Psychology

Computer engineering

Control theory and cybernetics

Philosophy

Confluences in AI

Mathematics

Economics

Neuroscience

Psychology

Computer engineering

Control theory and cybernetics