The challenge of poker

1
The challenge of poker

• Artificial Intelligence 134 (2002) pp. 201-240
• Darse Billings, Aaron Davidson, Jonathan
  Shaeffer, Duane Szafron
• Presented by John A. Martiney

2
Why poker?

• Imperfect information
• Other players card not known
• Weak hands can win (bluffing)
• Early investigations in game theory
• John von Neumann
• father of the modern computer
• John Nash
• Letter of recommendation to Princeton read "This
  man is a genius
• Nash Equilibrium
• On-line Casinos

3
Limit Texas Holdem

• 4 betting rounds first two at 10 and second
  two at 20
• Pre-Flop each player gets two cards face down
• Flop three community cards face up
• Turn fourth community card face up
• River fifth community card face up
• Showdown show cards

4
Poki

• Written to play Limit Texas Holdem
• Previous version was called Loki
• Written in Java
• IRC-Dealer
• Tournament-Dealer
• TCP/IP-Dealer
• http//www.cs.ualberta.ca/games/poker

5
Pre-flop Betting

• There are 52 choose 2 1326 possible hands
• Betting can be handled using a simple
  formula-based expert system
• Create income rates (measured in milli-bets)
  for pairs off-line using roll-out simulations
• Use David Sklanskys opinion on pre-flop plays to
  validate the system
• (Page 12)

6
Post-flop Betting

• Must consider community cards and board state
• Hand-strength (HS) have a higher rank than
  opponent (47 choose 2 1081 for one opponent)
• For next two cards in the turn and river
• Positive Potential (PPot) currently behind but
  end up ahead
• Negative Potential (NPot) currently ahead but
  end up behind
• Do for all 45 choose 2 990 possible turn and
  river cards for each possible two-card opponent
  hand
• Fast algorithm within 5 of actual value 95 of
  the time
• Determine Effective Hand Strength (EHS)

7
Betting Strategy Modifiers

• Weight Tables
• Kept for each opponent
• Based on betting history
• Eliminates unlikely card pairs
• (Page 19)
• Probability Triples
• Ex. Fold 0.1/ Call 0.4/ Raise 0.5 1.0
• Select random number from 0-1 to decide action

8
Smarter Betting

• Use selective sampling
• Perform full information simulations on selected
  branches of the game tree
• Each simulation done for both a call and raise
  actions
• Results in Expected Values (EV) for call, raise,
  and fold betting actions
• Use opponent models and weight tables to perform
  the simulations

9
Opponent Modeling

• Statistics-based
• Can be done in real time
• Generic opponent modeling (GOM)
• Have a set of rules of action for certain stages
  of the game
• Betting frequencies in a variety of contexts
• Specific opponent modeling (SOM)
• Assume past behavior is indicative of future
  actions
• Neural network
• Cannot be done in real time (yet)
• Trained (off-line) using human player data
• Used to determine which elements of the game
  directly affect the fold/call/raise action
• (Page 26)

10
Performance

• Self-play
• Play against earlier versions
• Thousands of hands can be played
• Internet play
• Different levels of play
• Can only play at the amateur level
• Web applet performance of 0.13 small bets per
  hand

11
Conclusion

• Very good explanations of poker strategies
• Not enough technical detail
• Fun game to play online