Investigation of learning Behavioural Functions in games

1
Investigation of learning Behavioural Functions
in games

• Jonathan Hitchcock
• Supervisor George Wells

2
Background

• Artificial Intelligence
• Very useful to automate different behaviors
• Games are a good model to test AI
• An investigation into general game-intelligence
  will have implications for all AI
• This project is a survey of what has been done in
  these areas

3
Classification (1)

• Some games have very similar solutions
• A solution to one will work for others
• If a classification of these games can divide
  them into categories, then each category can be
  worked on
• Characteristics of games that are important to
  their solution need to be found

4
Classification (2)

• I found three important characteristics
• Number of options available at each move
• Size of search space for solution
• Necessity of context for game position
• Some characteristics seem important, but arent,
  for this project
• Number of players
• Perfect information
• Zero-sum

5
Graph of Characteristics
options
context
search space

• Note characteristics are not independent, and
  affect each other

6
Finite space, limited options, non-contextual
RoShamBo

• Rock Paper Scissors
• Very simple set of options
• Yet, huge tournaments held, with different
  strategies tried out, etc
• Some very clever methods thought up
• Statistical, pattern matching, strategy-matching

7
RoShamBo tournament results

• W L D
• 1. Iocaine Powder 30 0 11
• 2. Phasenbott 26 1 14
• 3. Simple Modeller 24 2 15
• 24. Random(optimal) 2 2 37
• 28. Multi-strategy 12 21 8
• 37. Beat Last Move 5 27 9
• 38. Good Ole Rock 3 27 11
• 40. Rotate R-P-S 1 26 14
• 41. R-P-S 20-20-60 1 27 13

8
Finite space, limited options, contextual
Tic-Tac-Toe

• More complex than non-contextual
• Addition of context increases the search-space
• Bean-playing, M.E.N.A.C.E
• Investigate all options, mark as good or bad
• After enough games, this will never lose
• Weighting may be necessary, rather than simple
  boolean marking, due to size of space

9
Infinite space, limited options, non-contextual
Backgammon

• Game is of indefinite length, options vary
• A good move can be worked out from simply
  examining the position
• Neural Nets have reached world-class level at
  backgammon, with no external training
• Excel at strategic and positional judgment
• Not good at technicalpositions (bearing in
  against an anchor) we solve by probabilities
• 1,500,000 games needed to train

10
Infinite space, limited options, contextual
Rubik Cube

• No indication that the solution is near
• Search is possible, but very expensive
• Search until solution no evaluation function
• Externally taught solutions do well enough, but
  are not extendible
• It seems a combination of searching and rules is
  necessary
• Search a little, and generate rules from results

11
Infinite space, unlimited options (1) Chess

• A very common problem
• Generally solved using space-searches, but
  these are not perfect
• Can use evaluation functions to shorten search
• Methods such as pattern-matching have been tried,
  but are not too successful
• Deep Blue uses very fast hardware, and highly
  optimized searches, with very game-specific
  evaluative functions

12
Infinite space, unlimited options (2) Robot Wars

• A fairly common concept, very general battle
  simulator
• Closest to real situation wide variety of
  actions, no specific goal
• Rule-based solutions often do well (Quake bots)
• Neural Nets and Genetic algorithms have also been
  successful
• Like Chess, no real solution other than searches

13
Findings

• There are some very good game-playing programs in
  existence
• Deep Blue and Neural-Net Backgammon programs have
  achieved world class status
• There are a number of very useful methods
• Optimized space-searches, and neural nets the
  most common, and apparently the most powerful

14
The Future

• A method which would work for all eight of my
  categories would be very useful
• Neural Nets seem to be the way forward in this
  respect
• Extend the categories Game Theory is a huge
  field, and there is much that I have not covered