Games, Theory and Application

1
Games, Theory and Application

• Jaap van den Herik and Jeroen Donkers
• Institute for Knowledge and Agent Technology,
  IKAT
• Department of Computer Science
• Universiteit Maastricht
• The Netherlands
• SOFSEM 2004 Prague, Hotel VZ Merin
• Sunday, January 25 8.30-10.00h

2
Contents

• Games in Artificial-Intelligence Research
• Game-tree search principles
• Using opponent models
• Opponent-Model search
• Gains and risks
• Implementing OM search complexities
• Past future of computer chess

3

• Games, such as Chess, Checkers, and Go
• are an attractive pastime and
• scientifically interesting

4
Chess

• Much research has been performed in Computer
  Chess
• Deep Blue (IBM) defeated the world champion
  Kasparov in 1997
• A Micro Computer better than the world champion?

5
World Champion Programs

• KAISSA 1974 Stockholm
• CHESS 1977 Toronto
• BELLE 1980 Linz
• CRAY BLITZ 1983 New York
• CRAY BLITZ 1986 Keulen
• DEEP THOUGHT 1989 Edmonton
• REBEL 1992 Madrid
• FRITZ 1995 Hong Kong
• SHREDDER 1999 Paderborn
• JUNIOR 2002 Maastricht
• SHREDDER 2003 Graz
• ? 2004 Ramat-Gan

6
International Draughts

• Buggy best draughts program
• Human better than computer, but the margin is
  small
• Challenge More knowledge in program

7
Go

• Computer Go programs are weak
• Problem recognition of patterns
• Top Go programs Go4, Many Faces of Go, GnuGo,
  and Handtalk

8
Awari

• A Mancala game (pebble-hole game)
• The game is a draw
• Solved by John Romein and Henri Bal (VU
  Amsterdam) 2002
• All positions (900 billion) with a cluster
  computer of 144 1GHz processors and 72Gb ram
  computed in 51 hour
• Proven that even the best computer programs still
  make many errors in their games

9
Scrabble

• Maven beats every human opponent
• Author Brian Sheppard
• Ph.D. (UM) Towards Perfect Play of Scrabble
  (02)
• Maven can play scrabble in any language

10
Overview
11
Computer Olympiad

• Initiative of David Levy (1989)
• Since 1989 there have been 8 olympiads 4x
  Londen, 3x Maastricht, 1x Graz
• Goal
• Finding the best computer program for each game
• Connecting programmers / researchers of different
  games
• Computers play each other in competition
• Demonstrations
• Man versus Machine
• Man Machine versus Man Machine

12
Computer versus Computer
13
Computer Olympiad

• Last time in Graz 2004
• Also World Championship Computer Chess and World
  Championship Backgammon
• 80 particpants in several categories
• Competitions in olympiads history
• Abalone, Awari, Amazons, Backgammon, Bao,
  Bridge, Checkers, Chess, Chinese Chess, Dots and
  Boxes, Draughts, Gipf, Go-Moku, 19x19 Go, 9x9 Go,
  Hex, Lines of Action, Poker, Renju, Roshambo,
  Scrabble, and Shogi

14
UM Programs on the Computer Olympiads
15
Computer Game-playing

• Can computers beat humans in board games like
  Chess, Checkers, Go, Bao?
• This is one of the first tasks of Artificial
  Intelligence (Shannon 1950)
• Successes obtained in Chess (Deep Blue),
  Checkers, Othello, Draughts, Backgammon,
  Scrabble...

16
Computer Game-Playing

• Sizes of game trees
• Nim-5 28 nodes
• Tic-Tac-Toe ? 105 nodes
• Checkers ? 1031 nodes
• Chess ? 10123 nodes
• Go ? 10360 nodes
• In practice it is intractable to find a solution
  with minimax so use heuristic search

17
Three Techniques

• Minimax Search
• a-ß Search
• Opponent Modelling

18
Game-playing

• Domain two-player zero-sum game with perfect
  information (Zermelo, 1913)
• Task find best response to opponents moves,
  provided that the opponent does the same
• Solution Minimax procedure (von Neumann, 1928)

19
Minimax Search
Nim-5
Players remove on turn 1, 2, or 3 matches. The
player who takes the last match wins the game.
20
Minimax Search
5
1
2
3
4
3
2
1
1
1
2
2
2
3
3
0
3
2
1
2
1
0
1
1
1
1
1
1
2
2
1
2
3
2
1
0
1
0
0
1
0
0
0
1
1
2
1
1
1
0
0
0
0
1
0
21
Minimax Search
1
1
2
3
MINIMAXING
1
1
1
1
1
1
2
2
2
3
3
1
1
1
1
1
1
1
1
1
1
1
1
1
2
2
1
2
3
1
1
1
1
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
1
22
Pruning

• You do not need the total solution to play
  (well) only the move at the root is needed
• Methods exist to find this move without a need
  for a total solution pruning
• Alpha-beta search (Knuth Moore 75)
• Proof-number search (Allis et al. 94), etc.
• Playing is solving a sequence of game trees

23
Heuristic Search
0.25
1
2
3
0.25
1
1
1
1
1
2
2
2
3
3
1
0.25
0.33
0.5
0.33
0.5
1
0.5

• Truncate the game tree (depth selection)
• Use a (static heuristic) evaluation function at
  the leaves to replace pay-offs
• Miminax on the reduced game tree

24
Heuristic Search

• The approach works very well in Chess, but...
• Is solving a sequence of reduced games the best
  way to win a game?
• Heuristic values are used instead of pay-offs
• Additional information in the tree is unused
• The opponent is not taken into account
• We aim at the last item opponents

25
Minimax
3
2
3
7
2
4
3
26
a-ß algorithm
3
3
ß-pruning
2
7
2
4
3
27
The strength of a-ß
More than thousand prunings
28
The importance of a-ß algorithm
3
3
ß-pruning
2
4
3
29

The Possibilities Of Chess

• THE NUMBER OF DIFFERENT, REACHABLE
• POSITIONS IN CHESS IS
• (CHINCHALKAR) 1046

30
A Clever Algorithm (a-ß)

• SAVES THE SQAURE ROOT OF THE NUMBER OF
  POSSIBILITIES, ?N, THIS IS MORE THAN
• 99,9999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99999999999999999999999999999999999999999999999999
  99

31
A Calculation

• NUMBER OF POSSIBILITIES 1046
• SAVINGS BY a-? ALGORITHM 1023
• 1000 PARALLEL PROCESSORS 103
• POSITIONS PER SECOND 109
• LEADS TO 1023-12 1011 SECONDS
• A CENTURY IS 109 SECONDS
• SOLVING CHESS 102 CENTURIES
• SO 100 CENTURIES OR 10,000 YEAR

32
Using opponents strategy (NIM-5)
1
1
2
3
1
1
1
1
1
2
2
2
3
3
1
1
1
1
1
1
1
1
1
1
1
2
2
1
2
3
1
1
1
1
1
1
1
1
1
1
1
1
2
1
1
1
1
1
1
1
Player 1 never takes 3
33
Using opponents strategy

• Well known Tic-Tac-Toe strategy
• R1 make 3-in-a-row if possible
• R2 prevent opponents 3-in-a-row
• if possible
• H1 occupy central square if possible
• H2 occupy corner square if possible

34
Opponent-Model search

• Iida, vd Herik, Uiterwijk, Herschberg (1993)
• Carmel and Markovitch (1993)
• Opponents evaluation function is known
  (unilateral the opponent uses minimax)
• This is the opponent model
• It is used to predict opponents moves
• Best response is determined, using the own
  evaluation function

35
OM Search

• Procedure
• At opponents nodes use minimax (alpha-beta) to
  predict the opponents move. Return the own value
  of the chosen move
• At own nodes Return (the move with) the highest
  value
• At leaves Return the own evaluation
• Implementation one-pass or probing

36
OM Search Equations
37
OM Search Example
v0 8 vop 6
7
v0 7 vop 6
v0 8 vop 6
7
6
V0 6 Vop 7
V0 8 Vop 6
V0 7 Vop 6
V0 8 Vop 9
6
8
7
8
38
OM Search Algorithm (probing)
39
Risks and Gains in OM search

• Gain difference between the predicted move and
  the minimax move
• Risk difference between the move we expect and
  the move the opponent really selects
• Prediction of the opponent is important, and
  depends on the quality of opponent model.

40
Additional risk

• Even if the prediction is correct, there are
  traps for OM search
• Let P be the set of positions that the opponent
  selects, v0 be our evaluation function and vop
  the opponents function.

41
Four types of error

• V0 overestimates a position in P (bad)
• V0 underestimates a position in P
• Vop underestimates a position that enters P (good
  for us)
• Vop overestimates a position in P

42
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
43
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
44
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
45
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
46
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
47
Vo our evaluation Vop opponents
evaluation Mmx minimax value Obt obtained value
48
Pruning in OM Search

• Pruning at MAX nodes is not possible in OM
  search, only pruning at MIN nodes can take place
  (Iida et al, 1993).
• Analogous to a-ß search, the pruning version is
  called ß-pruning OM search

49
Pruning Example
7 8
a
5 7
7 8
b
c
- ? 8
- ? 9
7 8
5 7
g
f
e
d
6 8
- 9
7 4
4 6
5 7
- 8
h
i
j
k
l
m
o
n
q
p
s
r
u
t
w
v
y
x
5 7
7 4
6 8
- 9
- 5
- 9
- 10
4 6
- 8
- 9
- 8
- 9
50
Two Implementations

• One-pass
• visit every node at most once
• back-up both your own and the opponents value
• Probing
• At MIN nodes use a-ß search to predict the
  opponents move
• back-up only one value

51
One-Pass ß-pruning OM search
52
Probing ß-pruning OM search
53
Best-case time complexity
54
Best-case time complexity
COM Subtree A
COM Subtree B
55
Best-case time complexity

• The time complexity for the one-pass version is
  equal to that of the theoretical best
• The time complexity for the probing version is
  different
• first detect the number of leaf evaluations
  needed
• Then find the number of probes needed

56
Best-case time complexity
In the best case, pruning in the probes is not
changed by ßv1 all probes take the same
number of evaluations as with a fully open
window.
57
Best-case time complexity
58
Time complexities compared
(Branching factors 4, 8, 12, 16, 20)
59
Average-case time complexity
(Measured on random game trees)
60
Probabilistic Opponent-model Search

• Donkers, vd Herik, Uiterwijk (2000)
• More elaborate opponent model
• Opponent uses a mixed strategy n different
  evaluation functions (opponent types) plus a
  probability distribution
• It is assumed to approximate the true opponents
  strategy
• Own evaluation function is one of the opponent
  types

61
PrOM Search

• Procedure
• At opponents nodes use minimax (alpha-beta) to
  predict the opponents move for all opponent
  types separately. Return the expected own value
  of the chosen moves
• At own nodes Return (the move with) the highest
  value
• At leafs Return the own evaluation
• Implementation one-pass or probing

62
PrOM Search Equations
63
PrOM Search Example
v0 7.4
?0 7 ?1 8
7
Pr(?0) 0.3 Pr(?1) 0.7
v0 0.3 x 6 0.7 x 8 7.4
v0 7
?0 6 ?1 8
?0 7 ?1 6
7
6
0.3
0.7
0.0
1.0
v0 8
v0 7
v0 8
v0 6
?0 6 ?1 10
?0 7 ?1 6
?0 8 ?1 7
?0 8 ?1 8
6
8
7
8
64
PrOM Search Algorithm
65
How did chess players envision the future of
non-human chess players?

• - Euwe
• - Donner
• - De Groot
• - Timman
• - Sosonko
• - Böhm
• Question Do you think that a computer will be
  able to play at grandmaster level?

66
Euwe (June 10, 1980)

• I dont believe so.
• I am almost convinced of that, too

67
Timman (June 22, 1980)

• If you would express it in ELO-points, than I
  believe that a computer will not be able to
  obtain more than ...... lets say
  ......2500 ELO-points.

68
Sosonko (August 26, 1980)

• I havent thought about that so much, but I
  believe that it will certainly not be harmful for
  the human chess community. It will become more
  interesting. I am convinced that computer chess
  will play a increasingly important role. It will
  be very interesting, I think

69
Donner (April 13, 1981)

• But the computer cannot play chess at all and
  will never be able to play the game, at least not
  the first two thousand years, (...)
• What it does now has nothing to do with chess.

70
De Groot (June 19, 1981)

• No, certainly not. As a matter of fact, I
  believe it will not even possible that it can
  reach a stable master level.
• I believe it is possible that the computer can
  reach a master level, but not that it is a stable
  master.

71
Contributions from Science

• COMPUTERS PLAY STRONGER THAN HUMANS.
• COMPUTERS CAN NOT SOLVE CHESS.
• COMPUTERS ENABLE AN ALTERNATIVE FORM OF GAME
  EXPERIENCE.

72
Conclusions

• 1. Chess is a frontrunner among the games
• 2. Kasparovs defeat has become a victory for
  brute force in combination with knowledge and
  opponent modelling

73
The Inevitable Final Conclusion

• TECHNOLOGY, AND IN PARTICULAR
• OPPONENT MODELLING,
• MAKES THE BEST BETTER,
• BUT DOES NOT LEAVE THE
• WEAKER PLAYERS WITH
• EMPTY HANDS
• SINCE THEIR LEVEL
• WILL INCREASE TOO.