How computers play games with you

1
How computers play games with you

• CS161, Spring 03
• Nathan Sturtevant

2
Outline

• Historic Examples
• Classes of Games
• Algorithms
• Minimax
• ?-? pruning
• Other techniques
• Multi-Player Games

3
Successful Game Programs

• Checkers
• Chinook
• 1992 Tinsley won 40-game match, 4-2-34
• 1994 Tinsley withdrew due to health reasons
• 444 billion move end-game database
• Chess
• Kasparov is currently the best human
• 1997 Deep Blue won exhibition match 2-1-3
• 2003 Deep Junior played to a draw

4
Game Programs (continued)

• Othello (Reversi)
• 1997, Logistello beat Murakami 6-0 (264/120)
• Scrabble
• Maven
• 1998 played Adam Logan, won 9-5
• Came back from down 98 to win with MOUTHPART
• Awari (Mancala)
• Solved in 2002 - draw
• http//awari.cs.vu.nl/

5
Overview - Types of Games

• Single-Agent Search
• 1 player v. a difficult problem
• Defined by
• Start state
• Successor function
• Heuristic function
• Goal test

6
Overview - Types of Games

• Game Search (Adversary Search)
• Defined by
• Initial State
• Successor function
• Terminal Test
• Utility / payoff function
• Similar to heuristics in single agent problems

7
Chinese Checkers

• Based on European game Halma
• Americans called it Chinese Checkers
• 1 player game?
• 2 player game?
• Multi-player game?

8
Classes of Games

• Deterministic v. Non-deterministic
• Chess v. Backgammon
• Perfect Information v. Imperfect information
• Checkers v. Bridge
• Zero-sum (strictly competitive)
• Prisoners dilemna
• Non-zero sum

9
Classes of Games
Deterministic Chance
Perfect information
Imperfect information
10
Classes of Games
Deterministic Chance
Perfect information Chess, checkers, go, othello, chinese checkers
Imperfect information
11
Classes of Games
Deterministic Chance
Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk
Imperfect information
12
Classes of Games
Deterministic Chance
Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk
Imperfect information Stratego
13
Classes of Games
Deterministic Chance
Perfect information Chess, checkers, go, othello, chinese checkers Backgammon, monopoly, risk
Imperfect information Stratego Bridge, poker, scrabble, (real life)
14
How do we simulate games?

• Build a game tree
• Start state at the root
• All possible moves as children

15
Tic-Tac-Toe
16
How do we choose our move?

• Apply utility function at the leaves of the tree
• In tic-tac-toe, count how many rows and columns
  are occupied by each player and subtract

17
Tic-Tac-Toe
Utility 3
x 2r 2c 2d o 2r 2c 0d
x 2r 2c 2d o 2r 1c 1d
x 2r 3c 2d o 2r 1c 1d
x 3r 3c 2d o 2r 2c 0d
Utility 3
Utility 8
Utility 2
Utility 3
Utility 8
Utility 2
18
What is our algorithm?

• Apply utility function at the leaves of the tree
• In tic-tac-toe, count how many rows and columns
  are occupied by each player and subtract
• Back-up values in the tree
• This calculates the minimax value of a tree

19
Minimax
3
1 - ply
3
8
1 - ply
20
Minimax - Properties

• Complete?
• Yes - if tree is finite
• Optimal?
• Yes - against an optimal opponent
• Time Complexity?
• O(bd)
• Space Complexity?
• O(bd)

21
Minimax

• Assume our computer can expand 105 nodes/sec
• Assume we have 100 seconds to move
• 107 nodes/move
• Tic-tac-toe
• 9! 362880 (naïve) ways to play a game (b4)
• 39 19683 possible states (upper bound) on a
  board
• Chess
• b 35, d 100, must search 2154 nodes

22
Minimax - issues

• Evaluation function
• Where does it come from?
• Expert knowledge
• Chess material value
• Othello (reversi) positional strength
• Learned information
• Pre-computed tables
• Quiescence

23
quiescence
24
Minimax - issues

• Quiescence
• We dont see the consequences of our bad choices
• quiescence search
• Horizon problem
• We avoid dealing with a bad situation

25
Minimax

• In Chess
• b 35
• 107 nodes/move
• Can search 4-ply into tree (human novice)
• Good humans can search 8-ply
• Kasparov searches about 12-ply
• What to do?
• ?-? pruning

26
Minimax
3
1 - ply
3
8
1 - ply
27
?-? pruning

• ? lower bound on Maximizers score
• Start at -8
• ? upper bound on Minimizers score
• Start at 8

28
? -8 ? 8
? -8 ? 8
? -8 ? 8
1
?
?
-8
8
29
? -8 ? 8
? -8 ? 8
? 1 ? 8
1
2
-8
8
30
? -8 ? 8
? -8 ? 8
? 2 ? 8
2
?
?
-8
8
31
? -8 ? 8
? -8 ? 2
2
? -8 ? 2
3
2
2
?
?
-8
8
32
? -8 ? 8
2
? -8 ? 2
2
? 3 ? 2
3
2
2
?
?
-8
8
33
? 2 ? 8
2
2
3
5
2
1
2
3
?
?
-8
8
34
? 2 ? 8
2
2
3
5
2
6
1
2
3
?
?
-8
8
35
? 2 ? 8
2
? 2 ? 6
6
2
? 2 ? 6
3
7
2
6
1
2
3
5
6
7
?
?
-8
8
36
? 2 ? 8
2
6
? 2 ? 6
6
2
6
? 7 ? 6
3
7
2
6
1
2
3
5
6
7
?
?
-8
8
37
?-? pruning

• Complete?
• Yes - if tree is finite
• Optimal?
• Computes same value as minimax
• Time Complexity?
• Best case O(bd/2)
• Average case O(b3d/4)

38
?-? pruning

• Effectiveness depends on order of moves in tree
• In practice, we can usually get best-case
  performance
• Chess
• Before we could search 4-ply into tree
• Now we can search 8-ply into tree

39
Other Techniques

• Transposition tables
• Opening / Closing book

40
Transposition Tables

• Only using linear about of memory
• Search only takes a few kb of memory
• Most games arent trees but graphs

41
Transposition Tables
42
Transposition Tables

• A lot of duplicated effort
• Transposition tables hash game states into table
• Store saved minimax value in table
• Pre-compute store values
• Opening book
• Closing book

43
Multi-Player Games

• 2-Player game trees have a single minimax value
• Games with 2 players use a n-tuple of scores
• ie (3, 2, 5)
• The sum of values in every tuple should be
  constant

44
Maxn
(3 Players)
(7, 3, 0)
(7, 3, 0)
(0, 10, 0)
(1, 4, 5)
45
Can we prune maxn trees

• In minimax we bound the game tree value
• In maxn we bound based on sum of values
• All scores sum to 10
• If Player 1 gets 7 points
• Player 2-3 will get 3 points

46
Shallow Maxn Pruning
(3 Players)
(7, 3, 0)
(7, 3, 3)
(7, 3, 0)
(0, 10, 0)
(6, 4, 6)
47
Shallow Maxn Pruning

• Complete?
• Yes
• Optimal?
• Yes
• Time Complexity?
• Best-case bd/2
• Average-case bd
• Space Complexity?
• bd

48
Maxn Pruning

• Why is maxn weak in practice?
• Only compares 2 scores out of n players
• Relies on game evaluation properties, not
  ordering
• Last-Branch Pruning
• Speculative Pruning

49
Last-Branch/Speculative Pruning
(3 Players)
(3, 3, 4)
1
2
50
Last Branch/Spec. Pruning

• Best case O(bd(n-1)/n)
• As b gets large
• Dependent only on node ordering in tree
• http//www.cs.ucla.edu/nathanst/ for more info

51
Imperfect Information

• Most card games have imperfect information
• We can use monte-carlo simulation
• Create many consistent samples of possible
  opponent hands
• Solve using perfect-information methods
• Combine results together to make next move