Recap of Dynamic Games

Christopher Makler

Stanford University Department of Economics

 

Econ 51: Bonus Review

(of complete and incomplete information)

Extensive Form

Nodes:

Branches:

Initial node: where the game begins

Decision nodes: where a player makes a choice; specifies player

Terminal nodes: where the game ends; specifies outcome

Individual actions taken by players; try to use unique names for the same action (e.g. "left") taken at different times in the game

Information sets:

Sets of decision nodes at which the decider and branches are the same, and the decider doesn't know for sure where they are.

A "tree" representation of a game.

Example: Gift-Giving

She chooses to give one of three gifts:
X, Y, or Z.

1

X

Y

Z

Player 1 makes the first move.

Initial node

Player 1's actions at her decision node

(and decision node)

Example: Gift-Giving

Twist: Gift X is unwrapped,
but Gifts Y and Z are wrapped.
(Player 1 knows what they are,
but player 2 does not.)

After each of player 1's moves,
player 2 has the move: she can either accept the gift or reject it.

2

Accept X

Reject X

2

1

X

Y

Z

We represent this by having an information set connecting
player 2's decision nodes
after player 1 chooses Y or Z.

2

2

Player 2's actions

Player 2's decision nodes

Information set

Accept Y

Reject Y

Accept Z

Reject Z

Also: player 2 cannot make her action contingent on Y or Z; her actions must be "accept wrapped" or "reject wrapped"

Accept Wrapped

Reject Wrapped

Accept Wrapped

Reject Wrapped

Example: Gift-Giving

After player 2 accepts or rejects the gift, the game ends (terminal nodes) and payoffs are realized.

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

Accept X

Reject X

Accept Wrapped

Reject Wrapped

Accept Wrapped

Reject Wrapped

Terminal Nodes

Player 1's payoffs

Player 2's payoffs

In this game, both players get a payoff of
0 if any gift is rejected,
1 if gift X is accepted, and
2 if gift Y is accepted.

 

If gift Z is accepted, player 1 gets a payoff of 3, but player 2 gets a payoff of –1.

Strategies and Strategy Spaces

A strategy is a  complete, contingent plan  of action for a player in a game.

This means that every player
must specify what action to take
at every decision node in the game tree!

A strategy space is the set of all strategies available to a player.

Strategies & Strategy Spaces

Player 1 has a single decision:
which gift to give (X, Y, or Z).

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

Accept X

Reject X

Accept Wrapped

Reject Wrapped

Accept Wrapped

Reject Wrapped

Player 2 might have to make one of two decisions: accept or reject gift X,
and accept or reject a wrapped gift.

Let's abbreviate these as A/R and A'/R'.

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

Then player 2's strategy space is

Therefore player 1's strategy space is

Strategy Profiles

A strategy profile \(s = (s_1,s_2)\) is a vector showing which strategy from their strategy space is chosen by each player.

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

s_1 = Z
s_2 = AR'

The outcome of this is that gift Z is given and rejected, and both players receive a payoff of 0.

\text{Example: }s = (Z, AR')

Note: the strategy profile specifies which action is taken at every decision node!

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

\(X\)

\(AA'\)

1

2

\(AR'\)

\(RA'\)

\(RR'\)

\(Y\)

\(Z\)

Normal Form Representation

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

\(X\)

\(AA'\)

1

2

\(AR'\)

\(RA'\)

\(RR'\)

\(Y\)

\(Z\)

0

0

,

Normal Form Representation

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

\(X\)

\(AA'\)

1

2

\(AR'\)

\(RA'\)

\(RR'\)

\(Y\)

\(Z\)

0

0

,

3

–1

,

Normal Form Representation

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

S_1 = \{X,Y,Z\}
S_2 = \{AA', AR', RA', RR'\}

A'

R'

A'

R'

\(X\)

\(AA'\)

1

2

\(AR'\)

\(RA'\)

\(RR'\)

\(Y\)

\(Z\)

1

1

,

1

1

,

0

0

,

0

0

,

2

2

,

0

0

,

2

2

,

0

0

,

3

–1

,

0

0

,

3

–1

,

0

0

,

Normal Form Representation

Extensive-Form Representation
of a Simultaneous Game

If 1 and 2 choose at the same time, we can represent this in the same way as if 
player 1 chooses first, and player 2 chooses second without observing player 1's choice.

Normal-Form vs. Extensive-Form Representations

The extensive-form representation
of a game specifies:

The normal-form representation
of a game specifies:

The strategies available to each player

The player's payoffs for each combination of strategies

The players in the game

When each player moves

The actions available to each player each time it's their move

The players in the game

The player's payoffs for each combination of actions

Backward induction is a method of determining the outcome(s) of a game by starting at the end and working backwards.

Finite games: start from terminal nodes

Infinite games: a bit more complicated

Centipede Game

Player 1 chooses to Quit or Play

  • If they quit, they get $200
    and player 2 gets $0
  • If they play, the game continues

Player 2 chooses to Quit or Play

  • If they quit, they get $200
    and player 2 gets $0
  • If they play, the game continues

Player 1 chooses to Take or Share

  • If they take, they get $500
    and player 2 gets $100
  • If they share, both get $300

Definition: Subgame

A subgame consists of a decision node and all nodes reachable after that node, provided that none of the nodes in the subgame is connected to a node outside the subgame via an information set.​ 

(basically, it's a separable game-within-a-game)

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

Consider the following game.

Player 1 starts.

She chooses X or Y.

Player 2 observes player 1's choice.

If player 1 chose X, player 2 chooses A or B.

If player 1 chose Y, player 2 chooses C or D.

After player 2 makes his choice, the payoffs to each player are realized.

"Play X."

"Play Y."

What is player 1's strategy space?

What is player 2's strategy space?

"Play A after X, and C after Y."

"Play A after X, and D after Y."

"Play B after X, and C after Y."

"Play B after X, and D after Y."

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

What is player 1's strategy space?

What is player 2's strategy space?

AC

AD

BC

BD

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

AD

BC

BD

We can create the normal-form representation of this game.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

AD

BC

BD

We can create the normal-form representation of this game.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

What are the Nash equilibria of this normal-form game?

Do any of the Nash equilibria not make sense?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

AD

BC

BD

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

Think about this: after player 1 makes her move, we are in one of two subgames.

What should player 2 do in each subgame?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

AD

BC

BD

Think about this: after player 1 makes her move, we are in one of two subgames.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

What should player 2 do in each subgame?

Anticipating how player 2 will react, therefore, what will player 1 choose?

1

2

X

Y

X

Y

A

B

3

2

1

0

2

0

1

3

C

D

1

2

2

AC

AD

BC

BD

Think about the other Nash equilibrium.

3

2

1

0

2

0

1

3

3

2

1

3

2

1

0

0

This is better for player 2.

It's kind of like player 2 is threatening to play D if player 1 chooses Y.

But this threat is not credible.

Definition: Subgame Perfect Nash Equilibrium

In an extensive-form game of complete and perfect information,
a subgame in consists of a decision node and all subsequent nodes.

A Nash equilibrium is subgame perfect if the players' strategies
constitute a Nash equilibrium in every subgame.
(We call such an equilibrium a Subgame Perfect Nash Equilibrium, or SPNE.)

Informally: a SPNE doesn't involve any non-credible threats or promises.

Application: Entry Deterrence

Incomplete Information

  • Nonstrategic player "Nature" introduces randomness (e.g. determines which game is being played, or a player's "type")
  • Not all the players see (all of) Nature's moves
    • Static game (last Thursday): can't infer hidden information from players' moves
    • Dynamic game (Tuesday and today): actions by the informed player result in updated probability distributions for uninformed player

Games of
Incomplete Information

A

B

X

Y

1

2

2

0

2

0

0

4

0

4

A

B

X

Y

1

2

0

4

2

0

2

0

0

4

Suppose one of these
two games is being played.

Both players know there is an equal probability of each game.

Only player 1 knows which game is being played right now.

What is player 1's strategy space?
Player 2's?

pollev.com/chrismakler

Nature

Heads

(1/2)

Tails

(1/2)

Both players know there is an equal probability of each game.

Only player 1 knows which game is being played right now.

We can model this "as if" there is a nonstrategic player called Nature who moves first, 

2

1

2

1

X

Y

2

0

2

0

0

4

0

4

X

Y

0

4

2

0

2

0

0

4

X

Y

X

Y

\(A^H\)

\(B^H\)

\(A^T\)

\(B^T\)

flips a coin, and picks which game is being played based on the coin flip.

Player 1 observes Nature's move, so they have to choose what to do if Nature flips Heads (\(A^H\) or \(B^H\)) and if Nature flips Tails (\(A^T\) or \(B^T\)).

Player 2 does not, so their information set spans the entire game: they are only choosing X or Y.

Nature

Heads

(1/2)

Tails

(1/2)

2

1

2

1

\(A^H\)

\(B^H\)

X

Y

1

2

2

0

2

0

0

4

0

4

\(A^T\)

\(B^T\)

X

Y

0

4

2

0

2

0

0

4

X

Y

X

Y

The Bayesian Normal Form representation of the game shows the expected payoffs for each of the strategies the players could play:

\(A^HA^T\)

\(A^HB^T\)

\(B^HA^T\)

\(B^HB^T\)

X

Y

1

2

1

0

3

1

0

2

0

1

3

2

2

0

2

4

Bayes Nash Equilibrium is the NE of this game. It maps private information onto (simultaneously taken) actions.

Consider this game:

  • Nature chooses U or D
  • Player 1 observes Nature's choice,
    and chooses U or D.
  • Player 2 can only observe if (U,U) was played; otherwise they're at an information set,
    and don't know which node of that set.
  • Denote their beliefs as \(\alpha,\beta,\gamma\).
  • What are their beliefs, for different P1 strategies?

Are you James Talarico?

  • Nature chooses whether player 1 is Talarico or Bot
  • Player 1 observes Nature's choice,
    and chooses Send text (S) or Don't (D).
  • You (Player 2) can only observe if you got a text.
  • Let \(q\) be the probability with which you believe the text came from Talarico, conditional on receiving a text.

Separating and Pooling Equilibria

Separating Equilibrium: Each type of informed player chooses differently,
thereby conveying information about their type to the uninformed player

Pooling Equilibrium: Each type of informed player chooses the same,
thereby leaving the uninformed player with their prior belief.

Steps for calculating perfect Bayesian equilibria: Guess and Check!

  1. Start with a strategy for player 1 (pooling or separating).
  2. If possible, calculate updated beliefs (q in the example) by using Bayes’ rule.
    In the event that Bayes’ rule cannot be used, you must arbitrarily select an updated belief; here you will generally have to check different potential values for the updated belief with the next steps of the procedure.
  3. Given the updated beliefs, calculate player 2’s optimal action.
  4. Check whether player 1’s strategy is a best response to player 2’s strategy.
    If so, you have found a PBE.

What's the PBE of this game?

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

A'

R'

A'

R'

Accept gift X (A).

Let's start with the uncomplicated part.

What would player 2 do with gift X?

Next: let's guess and check the two possible strategies for a wrapped gift.

Suppose player 2 accepted a wrapped gift (A').

What would player 1's best response be to AA'?

Give gift Z (wrapped).

What would player 2's beliefs be in that case?

They would know a wrapped gift was Z \((q = 0)\)

Is AA' their best response to Z and these beliefs?

❌

\(q\)

\(1 -q\)

What's the PBE of this game?

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

A'

R'

A'

R'

Accept gift X (A).

Let's start with the uncomplicated part.

What would player 2 do with gift X?

Next: let's guess and check the two possible strategies for a wrapped gift.

Suppose player 2 rejected a wrapped gift (R').

What would player 1's best response be to AR'?

Give gift X.

What would player 2's beliefs be in that case?

They could be anything, because it doesn't happen

\(q\)

\(1 -q\)

What's the PBE of this game?

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

A'

R'

A'

R'

What would player 2's beliefs be in that case?

They could be anything, because it doesn't happen

\(q\)

\(1 -q\)

Need to choose \(q\) to make the expected payoff to R' be greater than the expected payoff to A'.

2q + (-1)(1-q) < 0
2q < 1-q
3q < 1
q < \tfrac{1}{3}

What's the PBE of this game?

1

0

1

0

2

0

2

0

3

0

–1

0

2

2

1

X

Y

Z

,

,

,

,

,

,

A

R

A'

R'

A'

R'

What would player 2's beliefs be in that case?

They could be anything, because it doesn't happen

\(q\)

\(1 -q\)

Need to choose \(q\) to make the expected payoff to R' be greater than the expected payoff to A'.

q < \tfrac{1}{3}

Perfect Bayesian Equilibrium:

q < \tfrac{1}{3}
s_1 = X
s_2 = AR'