Game Programming 101
Tic-Tac-Toe
https://slides.com/artfuldev/gp101-tic-tac-toe/live
Sudarsan Balaji
@artfuldev
Game Programming 101
Tic-Tac-Toe
Game Programming
?
Game Programming
Game Programming
Software development of video games
Game Programming
Software development of video games
Game Programming
Software development of video games
Game Programming
Software development of video games
What qualifies as a game?
Game Programming
Software development of video games
Game Programming
Software development of video games
Game Programming
Software development of video games
Game Programming
Software development of video games
What all is involved?
Game Programming
Game Programming
Simulation
Game Programming
Simulation
Computer Graphics
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Audio Programming
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Audio Programming
Input
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Audio Programming
Input
Network Programming
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Audio Programming
Input
Network Programming
Database Programming
Game Programming
Simulation
Computer Graphics
Artificial Intelligence
Physics
Audio Programming
Input
Network Programming
Database Programming
Game Programming
Artificial Intelligence
Game Programming
Artificial Intelligence
Game Programming
Artificial Intelligence
Game Programming
Artificial Intelligence
Game Programming
Artificial Intelligence
Artificial Intelligence
Expert System
Game Programming
Artificial Intelligence
Expert System
Game Programming
Artificial Intelligence
Expert System
Game Programming
Game Engine
Artificial Intelligence
Expert System
Game Programming
Game Engine
Artificial Intelligence
Expert System
Game Programming
Game Engine
Artificial Intelligence
Expert System
Game Programming
Game Engine
Game Programming
Game Engine
Game Engine
Game Engine
A program that analyses and provides feedback on the best possible move in a particular state of a game
Game Engine
A program that analyses and provides feedback on the best possible move in a particular state of a game
Game Engine
Example: Chess engines
Game Engine
Example: Chess engines
We can build them easily.
Game Engine
Example: Chess engines
We can build them easily.
Game Engine
Let's start with TicTacToe
Game Engine
Let's start with TicTacToe
Tic-Tac-Toe
Tic-Tac-Toe
As an AI environment
D
S
O
A
K
E
D
Tic-Tac-Toe
As an AI environment
D
S
O
A
K
E
D
D
S
O
A
K
E
D
Deterministicness
D
S
O
A
K
E
D
Deterministicness
Staticness
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
Known (Expert)
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
Known (Expert)
Episodic
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
Known (Expert)
Episodic
Discrete
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
Known (Expert)
Episodic
Discrete
D
S
O
A
K
E
D
Deterministicness
Staticness
Observability
Agency
Knowledge
Episodicness
Discreteness
Deterministic
Static
Fully Observable
Single-Agency
Known (Expert)
Episodic
Discrete
Simulateable
Solving Tic-Tac-Toe
Solving Tic-Tac-Toe
As a Human
Solving Tic-Tac-Toe
As a Human
How do we think?
Solving Tic-Tac-Toe
As a Computer
Solving Tic-Tac-Toe
As a Computer
function getBestMove(grid, currentPlayer) {
// ...
// calculate best move
// ...
return bestMove;
}Solving Tic-Tac-Toe
As a Computer
Occam's Razor
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
"in explaining a thing no more assumptions should be made than are necessary"
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
"in explaining a thing no more assumptions should be made than are necessary"
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
the simplest solution is usually the best
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
the simplest solution is usually the best
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
the simplest solution is usually the best
Solving Tic-Tac-Toe
As a Computer
Occam's Razor
the simplest solution is usually the best
What is the simplest solution?
Solving Tic-Tac-Toe
As a Computer
What is the simplest solution?
Solving Tic-Tac-Toe
As a Computer
What is the simplest solution?
BRUTE FORCE
What is the simplest solution?
BRUTE FORCE
BRUTE FORCE
BRUTE FORCE
Okay, but...
BRUTE FORCE
Okay, but...
HOW?
Okay, but...
HOW?
Okay, but...
HOW?
"Time travel is the solution to all problems."
Okay, but...
HOW?
"Time travel is the solution to all problems."
-Unknown
Okay, but...
HOW?
"Time travel is the solution to all problems."
-Unknown
Okay, but...
HOW?
"Time travel is the solution to all problems."
-Unknown
Okay, but...
HOW?
"Time travel is the solution to all problems."
-Unknown
How?
Okay, but...
HOW?
Get all possible moves.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Pick the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Pick the best possible move.
Return the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Pick the best possible move.
Return the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Pick the best possible move.
Return the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
Pick the best possible move.
Return the best possible move.
HOW?
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
See the results of every move.
Return the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
See the results of every move.
Pick the move which results in least losses.
Return the best possible move.
Okay, but...
HOW?
Get all possible moves.
Play all possible moves until the end of the game.
See the results of every move.
Pick the move which results in least losses.
Return the best possible move.
Okay, but...
IN CODE?
Get all possible moves.
Play all possible moves until the end of the game.
See the results of every move.
Pick the move which results in least losses.
Return the best possible move.
Okay, but...
IN CODE?
Okay, but...
IN CODE?
Okay, but...
IN CODE?
Time to say hi...
Okay, but...
IN CODE?
Time to say hi...
...to
Game Trees
Game Trees
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
By the way,
we just stepped into Game Theory.
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Traversal
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Traversal
Evaluation
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Traversal
Evaluation
Let's look at one!
Game Trees
A directive graph whose nodes are positions in a game and whose edges are moves
Traversal
Evaluation
Let's look at one!
And next we have...
Minimax
Minimax
Minimax
A decision rule that minimizes possible loss for a worst case scenario.
Minimax
Depends on the maximin score.
Minimax
Depends on the maximin score.
Minimax
Depends on the maximin score.
The largest value that the player can be sure to get without knowing the actions of the other players
Minimax
Minimax
Let's watch it in action again, in more detail this time!
Minimax
Let's watch it in action again, in more detail this time!
Minimax
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Minimax
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
How?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}How to find?
How?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Is game over?
How?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Is game over?
How close is X to winning?
How?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Is game over?
How close is X to winning?
What are my next possible moves?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Is game over?
How close is X to winning?
What are my next possible moves?
function minimax(node, depth, maximizingPlayer) {
if (depth === 0 || isTerminalNode(node))
return evaluate(node);
if (maximizingPlayer)
return getChildren(node)
.reduce((best, child) =>
Math.max(best, minimax(child, depth - 1, false)),
-Infinity);
return getChildren(node)
.reduce((best, child) =>
Math.min(best, minimax(child, depth - 1, true)),
Infinity);
}Minimax
Minimax
Optimizations
Minimax
Optimizations
should we traverse the entire tree?
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
max(a,b) = -min(-a,-b);
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
max(a,b) = -min(-a,-b);
concurrency
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
max(a,b) = -min(-a,-b);
concurrency
simpler evaluation
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
max(a,b) = -min(-a,-b);
concurrency
simpler evaluation
Minimax
Optimizations
should we traverse the entire tree?
alpha-beta pruning
if(β ≤ α) break;
should we calculate min and max?
negamax
max(a,b) = -min(-a,-b);
concurrency
simpler evaluation
evaluation
evaluation
evaluation
evaluation
evaluation is the process of assigning a value to a node
evaluation
evaluation is the process of assigning a value to a node
it is the heart of any engine, optimal moves can only be found with a sound evaluation function
evaluation
for tic-tac-toe
evaluation
function evaluateCells(cells) {
const length = cells.length;
if(length === 0) return 0;
const initial = { undefined: 0, true: 0, false: 0 };
const counts = cells.reduce((counts, cell) => increment(counts, cell), initial);
if (counts.undefined === length) return 0;
if (counts.true === length) return Infinity;
if (counts.false === length) return -Infinity;
if (counts.false === 0) return Math.pow(2, counts.true);
if (counts.true === 0) return -Math.pow(2, counts.false);
return 0;
}
function evaluate(grid) {
return evaluateRows(grid) + evaluateColumns(grid) + evaluateDiagonals(grid);
}move generation
function getMoves(grid) {
return grid
.map((value, index) => value == undefined ? index : undefined)
.filter(value => value != undefined);
}move generation
function getMoves(grid) {
return grid
.map((value, index) => value == undefined ? index : undefined)
.filter(value => value != undefined);
}termination
function hasGameEnded(grid) {
return hasXWon(grid) || hasOWon(grid) || isFull(grid);
}Source!
Source!
Source!
Other interesting games
Other interesting games
Chess
Other interesting games
Chess
2048
Other interesting games
Chess
2048
Think of your own game to beat!
Other interesting games
Chess
2048
Think of your own game to beat!
Other interesting games
Chess
2048
Think of your own game to beat!
My favourite!
Other interesting games
Chess
2048
Think of your own game to beat!
My favourite!
Chess
Chess
- Iterative deepening
Chess
- Iterative deepening
- Killer moves
Chess
- Iterative deepening
- Killer moves
- Quiescence search
Chess
- Iterative deepening
- Killer moves
- Quiescence search
- Move ordering
Chess
- Iterative deepening
- Killer moves
- Quiescence search
- Move ordering
- Hashtables
Chess
- Iterative deepening
- Killer moves
- Quiescence search
- Move ordering
- Hashtables
- Transposition table
Chess
- Iterative deepening
- Killer moves
- Quiescence search
- Move ordering
- Hashtables
- Transposition table
2048
2048
- Expectimax
2048
- Expectimax
2048
- Expectimax
Experiments
- github.com/artfuldev/Kenny
- github.com/artfuldev/KennyClassIQ
- github.com/artfuldev/KenTacToe
- github.com/artfuldev/Sharp48
- github.com/artfuldev/CAESAR
- github.com/artfuldev/tictactoe-ai
Questions?
Game Progamming 101: Programming Tic Tac Toe
By artfuldev
