Finite Iterated Prisoner

1
Finite Iterated Prisoners Dilemma Revisited
Belief Change and End Game Effect

• Jiawei Li (Michael) Graham Kendall
• University of Nottingham

2
Outline

• Iterated prisoners dilemma (IPD)
• Probability vs. uncertainty
• A new model of IPD
• End game effect
• Conclusions

Jubilee Campus, University of Nottingham
3
Iterated Prisoners Dilemma (IPD)

• Prisoners Dilemma

Two suspects are arrested by the police. They are
separated and offered the same deal. If one
testifies (defects from the other) for the
prosecution against the other and the other
remains silent (cooperates with the other), the
betrayer goes free and the silent accomplice
receives the full 10-year sentence. If both
remain silent, both prisoners are sentenced to
only six months in jail for a minor charge. If
each betrays the other, each receives a five-year
sentence. Each prisoner must choose to betray the
other or to remain silent.
4
Iterated Prisoners Dilemma (IPD)

• Nash equilibrium (Defect, Defect)

5
Iterated Prisoners Dilemma (IPD)

• Finite IPD
• n-stages
• n is known
• End game effect
• Backward induction
• Nash equilibrium

6
Iterated Prisoners Dilemma (IPD)

• Finite IPD experiments

Each of 15 pairs of subjects plays 22-round IPD.
Average cooperation rates are 44.2 in known and
55.0 in unknown. (from Andreoni and Miller
(1993))
7
Iterated Prisoners Dilemma (IPD)

• Incomplete information (Kreps et al (1982))
• Tit for Tat (TFT) type or Always defect
  (AllD) type
• Assign probability ? to
• TFT type and 1-? to AllD type
• Mutual cooperation can be
• sequential equilibrium
• End game effect

8
Iterated Prisoners Dilemma (IPD)

• A new model
• Assumption that both players may be either AllD
  or TFT type
• Repeated game with uncertainty
• Changeable beliefs

9
Probability vs. uncertainty

• Probability
• Tossing a coin
• 50 -- 50
• Uncertainty
• Tossing a what?
• 50 -- 50?

10
Probability vs. uncertainty

• Probability
• Tossing a coin repeatedly
• 50 -- 50
• Uncertainty
• Tossing the dice repeatedly
• Expected probability may
• change according to the
• outcome of past playing.

11
Probability vs. uncertainty

• Repeated games with uncertainty
• Bayes theorem
• P(Head) 0.5
• p(Head1Head) 0.683
• p(Head1Tail) 0.317
• p(Head2Head) 0.888
• p(Head2Head1Tail) 0.565
• ......

12
A new model of IPD

• Let ROW and COL denote the players in a N-stage
  IPD game.

where R, S, T, and P denote, Reward for mutual
cooperation, Suckers payoff, Temptation to
defect, and Punishment for mutual defection, and
13
A new model of IPD

• We refer to COLs belief (Rows type) by
• that denote the probabilities that, if ROW
  is a TFT player, ROW chooses to cooperate at the
  1,,n stage. Similarly, we define ROWs belief
  about COLs type by
  that denotes the probabilities that COL chooses
  to cooperate at each stage if COL is a TFT
  player. di and ?i represent the beliefs of
  player COL and ROW respectively.

14
A new model of IPD

• Two Assumptions
• A1
• A2 If either player chooses to defect at stage
  i, there will be

15
A new model of IPD
16
A new model of IPD
17
A new model of IPD
18
A new model of IPD
(D, D) is always Nash equilibrium at stage i
but it is not necessarily the only Nash
equilibrium. When (2) is satisfied, both (C, C)
and (D, D) are Nash equilibrium. (2) is a
necessary and sufficient condition.
19
A new model of IPD
A sufficient condition for (C,C) to be Nash
equilibrium
This condition denotes a depth-one induction,
that is, if both players are likely to cooperate
at both the current stage and the next stage, it
is worth each player choosing to cooperate at the
current stage. For example, when T5, R3, P1,
and S0, the condition for (C,C) to be Nash
equilibrium is,
20
End game effect

• 
  ?
• Unexpected hanging paradox.
• A condemned prisoner
• Will be hanged on one weekday
• in the following week
• Will be a surprise

21
End game effect
22
End game effect
P1 1
P2 1
P3 1
23
End game effect
24
End game effect

• Process of belief change

P1 1/5 P2 1/5 P3 1/5 P4 1/5 P5 1/5
P1 0 P2 1/4 P3 1/4 P4 1/4 P5 1/4
P1 0 P2 0 P3 1/3 P4 1/3 P5 1/3
P1 0 ... P4 1/2 P5 1/2
P1 0 ... P5 1
25
End game effect

• Backward induction is not suitable for repeated
  games with uncertainty because uncertainty can
  be decreased during the process of games.
• End game effect has limited influence on the
  players strategies since it cannot be backward
  inducted.

26
End game effect

• Why does the rate of cooperation in finite IPD
  experiments decrease as the game goes toward the
  end?

27
Conclusions

• 1. We develop a new model for finite IPD that
  takes into consideration belief change.
• 2. Under the new model, the conditions of mutual
  cooperation are deduced. The result shows that,
  if the conditions are satisfied, both mutual
  cooperation and mutual defection are Nash
  equilibrium. Otherwise, mutual defection is the
  unique Nash equilibrium.

28
Conclusions

• 3. This model could also deal with indefinite IPD
  and infinite IPD.
• 4. The outcome of this model conforms to
  experimental results.

29
Conclusions

• 5. Backward induction is not suitable for
  repeated games with uncertainty when the beliefs
  of the players are changeable.
• 6. This model has the potential to apply to other
  repeated games of incomplete information.

30
Thank you.