# Game Theory II: Dynamic Games

Christopher Makler

Stanford University Department of Economics

Econ 51: Lecture 10

## Big Ideas

Today: extend notion of best-response (Nash) equilibrium to the class of dynamic games with complete & perfect information.

As we saw with Stackelberg vs. Cournot, the same actions can result in different equilibria if games are played sequentially vs. simultaneously.

As we saw with the model of collusion in cartels,
being involved in an ongoing relationship allows cooperation
of a sort that is impossible in a "one-shot" game.

## Big Ideas

Today: extend notion of best-response (Nash) equilibrium to the class of dynamic games with complete & perfect information.

"dynamic"

takes place over time;
not simultaneous/one-shot

"complete information"

all players know all relevant aspects of the game (especially payoffs)

"perfect information"

all players observe all moves

# Extensive Form

Nodes:

Branches:

Initial node: where the game begins

Decision nodes: where a player makes a choice; specifies player

Terminal nodes: where the game ends; specifies outcome

Individual actions taken by players; try to use unique names for the same action (e.g. "left") taken at different times in the game

Information sets:

Sets of decision nodes at which the decider and branches are the same, and the decider doesn't know for sure where they are.

A "tree" representation of a game.

Normal-Form vs. Extensive-Form Representations

The extensive-form representation
of a game specifies:

The normal-form representation
of a game specifies:

The strategies available to each player

The player's payoffs for each combination of strategies

The players in the game

When each player moves

The actions available to each player each time it's their move

The players in the game

The player's payoffs for each combination of actions

# Strategies and Strategy Spaces

A strategy is a  complete, contingent plan  of action for a player in a game.

This means that every player
must specify what action to take
at every decision node in the game tree!

A strategy space is the set of all strategies available to a player.

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

Consider the following game.

Player 1 starts.

She chooses X or Y.

Player 2 observes player 1's choice.

If player 1 chose X, player 2 chooses A or B.

If player 1 chose Y, player 2 chooses C or D.

After player 2 makes his choice, the payoffs to each player are realized.

"Play X."

"Play Y."

What is player 1's strategy space?

What is player 2's strategy space?

"Play A after X, and C after Y."

"Play A after X, and D after Y."

"Play B after X, and C after Y."

"Play B after X, and D after Y."

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

What is player 1's strategy space?

What is player 2's strategy space?

AC

BC

BD

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

BC

BD

We can create the normal-form representation of this game.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

BC

BD

We can create the normal-form representation of this game.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

What are the Nash equilibria of this normal-form game?

Do any of the Nash equilibria not make sense?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

BC

BD

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

What should player 2 do in each subgame?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

BC

BD

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

What should player 2 do in each subgame?

Anticipating how player 2 will react, therefore, what will player 1 choose?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

BC

BD

Think about the other Nash equilibrium.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

This is better for player 2.

It's kind of like player 2 is threatening to play D if player 1 chooses Y.

But this threat is not credible.

### Definition: Subgame Perfect Nash Equilibrium

In an extensive-form game of complete and perfect information,
a subgame in consists of a decision node and all subsequent nodes.

A Nash equilibrium is subgame perfect if the players' strategies
constitute a Nash equilibrium in every subgame.
(We call such an equilibrium a Subgame Perfect Nash Equilibrium, or SPNE.)

Informally: a SPNE doesn't involve any non-credible threats or promises.

## Another Example: Entry Deterrence

pollev.com/chrismakler

How many subgames does this game have?

What is player 1's

strategy space?

What is player 2's

strategy space?

## Backward Induction

The process of starting at the terminal nodes and working backwards to determine what each player will do.

You still have to specify complete, contingent plans of action (i.e. strategies)!

# Repeated Games

## Key Insight

• In a repeated game, you can sustain behavior which would not be a Nash equilibrium in a one-shot game.
• The mechanism you employ needs to be a credible threat or promise about future behavior.
• "Credible" = cannot involve a strategy which is not subgame perfect: when you get to that point in the game, it has to be an equilibrium from then on

## Infinitely Repeated Games

V(\pi_0,\pi_1,\pi_2, \cdots) = \pi_0 + {\pi_1 \over 1 + r} + {\pi_2 \over (1 + r)^2} + \cdots

Recall: when we looked at collusion,
we said a stream of payments had a present value of

This is a special case of a general formulation used in game theory called exponential discounting

V(\pi_0,\pi_1,\pi_2, \cdots) = \pi_0 + \delta \pi_1 + \delta^2 \pi_2 + \cdots

where $$\delta < 1$$ is called the "discount factor"

x + {x \over r}

Value of getting payoff $$x$$ forever, starting now:

Value of getting payoff $$z$$ forever, starting next period:

Value of getting payoff $$y$$ now and then payoff $$z$$ forever after:

{z \over r}
y + {z \over r}

(for $$\delta = {1 \over 1 + r}$$)

{x \over 1 - \delta}
{\delta z \over 1 - \delta}
y + {\delta z \over 1 - \delta}

(for general $$\delta$$)

### Strategy in an Infinitely Repeated Game

• Must specify what to do in each period,
as a function of all previously taken actions.
• We often group action histories to make strategies simpler:
• Grim trigger: "If anyone has ever played 'cheat'"
• Tit-for-tat: "If the other player played 'cheat' last period"
• Limited punishment: "If the other player played 'cheat' in any of the last 3 periods"

# Infinitely Repeated Prisoners' Dilemma

1

2

Cooperate

Defect

Cooperate

Defect

2

2

,

3

0

,

1

1

,

0

3

,

Suppose the following prisoner's dilemma is repeated indefinitely,
with future payoffs discounted at rate $$\delta < 1$$.

Consider the grim trigger strategy:
If you ever defect, I will defect for all eternity."
For what values of $$\delta$$ would that constitute a SPNE?

# Recall: Coordination

1

2

Top

Bottom

Left

Right

3

1

,

0

0

,

1

3

,

0

0

,

1

2

3

1

,

0

0

,

1

3

,

0

0

,

Top

Middle

Left

Center

Bottom

Right

0

0

,

0

1

,

0

0

,

5

0

,

4

4

,

What are the Nash Equilibria of this game?

Suppose this game is repeated twice,
with no discounting (so payoffs are
just the sum of the two periods).

Can (Bottom, Right) be played in the first period in a SPNE?

## Conclusions

• New refinement of Nash equilibrium: subgame perfection
• When you arrive at any node of a game, must be an equilibrium from then on
• Strategies must specify what each player does at every possible node in a game, even if that node is never reached
• In games that take place over time:
• some strategies which are Nash equilibria of the normal-form game are not subgame perfect (involve non-credible threats or promises)
• some strategies which are not Nash equilibria of a simultaneous game may be sustainable if the game is repeated

By Chris Makler

• 445