Game Programming 101
Tic-Tac-Toe

https://slides.com/artfuldev/gp101-tic-tac-toe/live

Sudarsan Balaji

@artfuldev

Game Programming 101
Tic-Tac-Toe

Game Programming
?

Game Programming

Game Programming

Software development of video games

Game Programming

Software development of video games

Game Programming

Software development of video games

Game Programming

Software development of video games

What qualifies as a game?

Game Programming

Software development of video games

Game Programming

Software development of video games

Game Programming

Software development of video games

Game Programming

Software development of video games

What all is involved?

Game Programming

Game Programming

Simulation

Game Programming

Simulation

Computer Graphics

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Audio Programming

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Audio Programming

Input

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Audio Programming

Input

Network Programming

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Audio Programming

Input

Network Programming

Database Programming

Game Programming

Simulation

Computer Graphics

Artificial Intelligence

Physics

Audio Programming

Input

Network Programming

Database Programming

Game Programming

Artificial Intelligence

Game Programming

Artificial Intelligence

Game Programming

Artificial Intelligence

Game Programming

Artificial Intelligence

Game Programming

Artificial Intelligence

Artificial Intelligence

Expert System

Game Programming

Artificial Intelligence

Expert System

Game Programming

Artificial Intelligence

Expert System

Game Programming

Game Engine

Artificial Intelligence

Expert System

Game Programming

Game Engine

Artificial Intelligence

Expert System

Game Programming

Game Engine

Artificial Intelligence

Expert System

Game Programming

Game Engine

Game Programming

Game Engine

Game Engine

Game Engine

A program that analyses and provides feedback on the best possible move in a particular state of a game

Game Engine

A program that analyses and provides feedback on the best possible move in a particular state of a game

Game Engine

Example: Chess engines

Game Engine

Example: Chess engines

We can build them easily.

Game Engine

Example: Chess engines

We can build them easily.

Game Engine

Let's start with TicTacToe

Game Engine

Let's start with TicTacToe

Tic-Tac-Toe

Tic-Tac-Toe

As an AI environment

D

S

O

A

K

E

D

Tic-Tac-Toe

As an AI environment

D

S

O

A

K

E

D

D

S

O

A

K

E

D

Deterministicness

D

S

O

A

K

E

D

Deterministicness

Staticness

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

Known (Expert)

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

Known (Expert)

Episodic

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

Known (Expert)

Episodic

Discrete

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

Known (Expert)

Episodic

Discrete

D

S

O

A

K

E

D

Deterministicness

Staticness

Observability

Agency

Knowledge

Episodicness

Discreteness

Deterministic

Static

Fully Observable

Single-Agency

Known (Expert)

Episodic

Discrete

Simulateable

Solving Tic-Tac-Toe

Solving Tic-Tac-Toe

As a Human

Solving Tic-Tac-Toe

As a Human

How do we think?

Solving Tic-Tac-Toe

As a Computer

Solving Tic-Tac-Toe

As a Computer

function getBestMove(grid, currentPlayer) {
    // ...
    // calculate best move
    // ...
    return bestMove;
}

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

"in explaining a thing no more assumptions should be made than are necessary"

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

"in explaining a thing no more assumptions should be made than are necessary"

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

the simplest solution is usually the best

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

the simplest solution is usually the best

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

the simplest solution is usually the best

Solving Tic-Tac-Toe

As a Computer

Occam's Razor

the simplest solution is usually the best

What is the simplest solution?

Solving Tic-Tac-Toe

As a Computer

What is the simplest solution?

Solving Tic-Tac-Toe

As a Computer

What is the simplest solution?

BRUTE FORCE

What is the simplest solution?

BRUTE FORCE

BRUTE FORCE

BRUTE FORCE

Okay, but...

BRUTE FORCE

Okay, but...

HOW?

Okay, but...

HOW?

Okay, but...

HOW?

"Time travel is the solution to all problems."

Okay, but...

HOW?

"Time travel is the solution to all problems."

-Unknown

Okay, but...

HOW?

"Time travel is the solution to all problems."

-Unknown

Okay, but...

HOW?

"Time travel is the solution to all problems."

-Unknown

Okay, but...

HOW?

"Time travel is the solution to all problems."

-Unknown

How?

Okay, but...

HOW?

Get all possible moves.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Pick the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Pick the best possible move.

Return the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Pick the best possible move.

Return the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Pick the best possible move.

Return the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

Pick the best possible move.

Return the best possible move.

HOW?

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

See the results of every move.

Return the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

See the results of every move.

Pick the move which results in least losses.

Return the best possible move.

Okay, but...

HOW?

Get all possible moves.

Play all possible moves until the end of the game.

See the results of every move.

Pick the move which results in least losses.

Return the best possible move.

Okay, but...

IN CODE?

Get all possible moves.

Play all possible moves until the end of the game.

See the results of every move.

Pick the move which results in least losses.

Return the best possible move.

Okay, but...

IN CODE?

Okay, but...

IN CODE?

Okay, but...

IN CODE?

Time to say hi...

Okay, but...

IN CODE?

Time to say hi...

...to

Game Trees

Game Trees

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

By the way,

we just stepped into Game Theory.

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Traversal

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Traversal

Evaluation

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Traversal

Evaluation

Let's look at one!

Game Trees

A directive graph whose nodes are positions in a game and whose edges are moves

Traversal

Evaluation

Let's look at one!

tinyurl.com/game-trees

And next we have...

Minimax

Minimax

Minimax

A decision rule that minimizes possible loss for a worst case scenario.

Minimax

Depends on the maximin score.

Minimax

Depends on the maximin score.

Minimax

Depends on the maximin score.

The largest value that the player can be sure to get without knowing the actions of the other players

Minimax

Minimax

Let's watch it in action again, in more detail this time!

Minimax

Let's watch it in action again, in more detail this time!

tinyurl.com/tictactoe-ai

Minimax

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Minimax

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

How?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

How to find?

How?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Is game over?

How?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Is game over?

How close is X to winning?

How?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Is game over?

How close is X to winning?

What are my next possible moves?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Is game over?

How close is X to winning?

What are my next possible moves?

function minimax(node, depth, maximizingPlayer) {
  if (depth === 0 || isTerminalNode(node))
    return evaluate(node);
  if (maximizingPlayer)
    return getChildren(node)
      .reduce((best, child) =>
        Math.max(best, minimax(child, depth - 1, false)),
      -Infinity);
  return getChildren(node)
    .reduce((best, child) =>
        Math.min(best, minimax(child, depth - 1, true)),
      Infinity);
}

Minimax

Minimax

Optimizations

Minimax

Optimizations

should we traverse the entire tree?

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

max(a,b) = -min(-a,-b);

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

max(a,b) = -min(-a,-b);

concurrency

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

max(a,b) = -min(-a,-b);

concurrency

simpler evaluation

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

max(a,b) = -min(-a,-b);

concurrency

simpler evaluation

Minimax

Optimizations

should we traverse the entire tree?
alpha-beta pruning

if(β ≤ α) break;

should we calculate min and max?
negamax

max(a,b) = -min(-a,-b);

concurrency

simpler evaluation

evaluation

evaluation

evaluation

evaluation

evaluation is the process of assigning a value to a node

evaluation

evaluation is the process of assigning a value to a node

it is the heart of any engine, optimal moves can only be found with a sound evaluation function

evaluation

for tic-tac-toe

evaluation

function evaluateCells(cells) {
  const length = cells.length;
  if(length === 0) return 0;
  const initial = { undefined: 0, true: 0, false: 0 };
  const counts = cells.reduce((counts, cell) => increment(counts, cell), initial);
  if (counts.undefined === length) return 0;
  if (counts.true === length) return Infinity;
  if (counts.false === length) return -Infinity;
  if (counts.false === 0) return Math.pow(2, counts.true);
  if (counts.true === 0) return -Math.pow(2, counts.false);
  return 0;
}

function evaluate(grid) {
  return evaluateRows(grid) + evaluateColumns(grid) + evaluateDiagonals(grid);
}

move generation

function getMoves(grid) {
  return grid
          .map((value, index) => value == undefined ? index : undefined)
          .filter(value => value != undefined);
}

move generation

function getMoves(grid) {
  return grid
          .map((value, index) => value == undefined ? index : undefined)
          .filter(value => value != undefined);
}

termination

function hasGameEnded(grid) {
  return hasXWon(grid) || hasOWon(grid) || isFull(grid);
}

Source!

Source!

github.com/artfuldev/tictactoe-ai

Source!

github.com/artfuldev/tictactoe-ai

artfuldev.github.io/tictactoe-ai

Other interesting games

Other interesting games

Chess

Other interesting games

Chess
2048

Other interesting games

Chess
2048

Think of your own game to beat!

Other interesting games

Chess
2048

Think of your own game to beat!

Other interesting games

Chess
2048

Think of your own game to beat!

My favourite!

Other interesting games

Chess
2048

Think of your own game to beat!

My favourite!

Chess

Chess

• Iterative deepening

Chess

• Iterative deepening
• Killer moves

Chess

• Iterative deepening
• Killer moves
• Quiescence search

Chess

• Iterative deepening
• Killer moves
• Quiescence search
• Move ordering

Chess

• Iterative deepening
• Killer moves
• Quiescence search
• Move ordering
• Hashtables

Chess

• Iterative deepening
• Killer moves
• Quiescence search
• Move ordering
• Hashtables
• Transposition table

Chess

• Iterative deepening
• Killer moves
• Quiescence search
• Move ordering
• Hashtables
• Transposition table

2048

2048

• Expectimax

2048

• Expectimax

2048

• Expectimax

github.com/artfuldev/Sharp48