MonteCarlo Tree Search in Settlers of Catan

1
Monte-Carlo Tree Search in Settlers of Catan

• István Szita, Guillaume Chaslot, Pieter Spronck

2
(No Transcript)
3
Board Games vs. Video Games
4
(No Transcript)
5
Modern Board Games vs. Others
6
Settlers of Catan

• Klaus Teuber
• Spiel des Jahres 1995
• Origins Awards 1996
• gt 11 million copies sold

7
(No Transcript)
8
(No Transcript)
9
Rule Changes

• Removal of imperfect information
• Contents of development cards
• Contents of stolen cards
• No trading of our agent with the opponents
• But opponents are allowed to trade

10
(No Transcript)
11
(No Transcript)
12
JSettlers Architecture
13
(No Transcript)
14
(No Transcript)
15
Effect of Starting Position Random Play
16
Effect of Starting Position MCTS 1000
17
Effect of Starting Position

• Random play advantage for first player
• MCTS play advantage for third and second player
• Thus starting position has an effect
• Therefore for all following experiments, player
  positions are randomized per game

18
Neighbour and Kingmaker Effect

• There is probably no neighbour effect in Settlers
  of Catan
• and if there is, randomized seating order is
  effective against it
• There might be a kingmaker effect in Settlers of
  Catan
• however, our bots are known to be impartial
• still, in future research this issue may play a
  role

19
Domain Knowledge in MC Simulation

• Basic action weight 1
• End turn 1
• Building a settlement or city 10000
• Building a road 10/10R
• Where R roads / (settlements cities)
• Playing a knight (soldier)
• If robber is blocking a resource 100
• Otherwise 1
• Playing a development card other than knight 10
• Uniform sampling gives better results!

20
Domain Knowledge in Tree Search

• Assign virtual wins to preferred actions
• Building a settlement 20 virtual wins
• Building a city 10 virtual wins
• Virtual wins are not propagated to parents!
• Otherwise, a weak action before one that gives
  virtual wins, might be rated far too high
• This improves playing strength considerably

21
Tests

• Against three JSettlers AIs
• Random AI
• MCTS with 1000 simulations per move
• MCTS with 10000 simulations per move
• Against a strong human player
• 30 games

22
Test 1

• JSettlers (top)vs.Random (bottom)

23
Test 2

• JSettlers (top)vs.MCTS 1000(bottom)

24
Test 3

• JSettlers (top)vs.MCTS 10000(bottom)

25
Test Against Humans

• SmartSettlers MCTS 10000 plays justifiable moves
• Play style is often human-like
• More challenging than JSettlers
• Still, humans tend to play stronger

26
Human Supremacy

• Wood clay strategy
• Focus on building settlements and roads
• Ore wheat strategy
• Focus on building cities and buying developments
• SmartSettlers prefers ore wheat strategy, and
  tends to build less settlements than a human
  would
• Wood clay requires that the player postpones
  building roads in favour of building settlements
• Search depth needed to discover this is pretty
  high
• Possible solutions more simulations, improved
  move selection heuristics

27
Conclusion

• MCTS is suitable for implementing AI for
  non-deterministic, multiplayer games such as
  Settlers of Catan
• Future work
• Implementing a version that complies with all
  rules
• Implementing trading
• Improving playing strength through domain
  knowledge, possibly automatically extracted

28
Contact details

• Pieter Spronck
• pieter_at_spronck.net
• http//www.spronck.net
• István Szita
• szityu_at_gmail.com