Pattern Recognition

1
Pattern Recognition Limits of Machine Learning

• Game Programming
• Kit Ming Chan

2
About Our Authors

• Section 10.3
• Timo Kaukoranta, Jouni Smed, Harri Hakonen
• Department of I.T., University of Turku, Finland
• Smed computer games, optimization, and
  scheduling algorithms, doctoral degree from
  university of Turku lthttp//staff.cs.utu.fi/staff/
  jouni.smed/gt
• Hakonen Algorithms for Computer Games, String
  Algorithmic, Software Construction and Object
  Integrity lt http//staff.cs.utu.fi/staff/harri.hak
  onen/gt
• Section 10.5
• Neil Kirby
• Lucent Tech, Bell Labs, nak_at_lucent.com, MS in
  Comp Sci, Ohio State University.
• currently develops .NET solutions.
• built speech recognition software and teaching at
  the university level.
• In his spare time he designs multi-player,
  tactical combat computer games. He especially
  enjoys writing programs for computer opponents
  that play well without cheating.

3

• 10.5 Getting around the Limits of Machine Learning

4
Categories of Limitations

• Four Questions
• Is it cheap to see (recognize) the thing to learn
  from?
• Is it cheap to store the knowledge?
• Is it cheap to use the knowledge?
• Does learning make the game better?

5
Divisions of Problem Space

• Implicit vs. explicit learning
• Implicit means the game is self motivated to
  learn
• Explicit means the game is told to learn
• Learn in the field vs. in development
• Flexibility vs. risky quality assurance

6
Examples of Learning

• Simple question
• Implicit or explicit learning? Learning in field
  or development?
• Is it cheap to see/store/use knowledge and does
  it improve the game?
• How to deal with quality assurance?
• Black White
• Remember in the last class when the human player
  teaches his monster to throw stone at the
  village?

• Explicit and In field learning
• Cheap to see ( just copy the player), Cheap to
  store (proven by example), Cheap to use (game
  runs well on target machines)
• Improve the game (sure! Increase enjoyment of
  replaying)
• To avoid quality assurance problem, creatures are
  programmed with innate behaviors that constrain
  learning

7
Example of Learning

• Command Conquer Renegade
• A feature that is turned off in the shipped
  version is a code that copies player movement and
  add new paths to the pathfinder.

• Implicit and in field learning
• Cheap to see ( computationally easy)
• Cheap to store (free! Pathfinder already stores
  this type of info)
• Cheap to use (certainly! Just add some more
  paths to pathfinder)
• Improve the game (makes AI more intelligent)
• No quality assurance concern because it makes use
  of original, safe component (pathfinder) of the
  game

8
Example of Learning

• Re-volt (racing game)
• A genetic algorithm is used during development to
  tune parameters and reduce lap times
• Implicit in development learning
• Cheap to see
• ( during game has no cost, but during development
  it is costly to run GA)
• Cheap to store
• ( free! Just change the parameters)
• Cheap to use
• (during game has no cost, but during development
  it takes a long time)
• Improve the game
• (makes AI more intelligent)
• No quality assurance concern

9
Limits to Learning?

• What are some limits you can think of now?

• Most limitations fall into these categories
• Knowledge representation
• Recognizing the knowledge
• Storing the knowledge
• Using the knowledge
• The last 3 are simply the first 3 questions we
  have been talking about

• Is the knowledge cheap to see?
• Is the knowledge cheap to store?
• Is the knowledge cheap to use?

10
Knowledge Representation

• Consider music coding
• One channel of CD-quality music
• requires 88,200 bytes / sec regardless of content
• MIDI
• a few events per note,
• most music is roughly a few notes per second.
• Printed sheet music
• one event per note
• Limit 1 the higher the level of representation,
  the easier the knowledge to work with, but the
  more expensive to acquire

11
Seeing More Clearly

• Limit 2 AI must store information to deal with
  the past
• The present
• Easier to learn than past
• BW ties feedback to behavior temporally
• CC Renegade reduces mouse-and-keyboard input
  into paths, and pathfinder stores new area
  connectivity
• Re-Volt uses the lap timer to differentiate good
  parameters from suboptimal ones.
• The past
• Racing game, Backgammon are not history dependent
• For history dependent games, it is much more
  difficult to understand causal chain in history

12
Seeing More Clearly

• Limit 3 Instance based methods learn wrong
  things from data
• Neural nets, Markov models
• Need good training data that resembles testing
• Cannot differentiate similar (but different) data
  even if trained with it

13
Seeing More Clearly

• Examples of cases that are hard to learn in
  instanced-based learning
• 1. Neural Networks to train tank recognition

2. Pronunciation of One, Two, and Three in
German http//www.bbc.co.uk/languages/german/lj/la
nguage_notes/1_10.shtml
14
Storing new Knowledge

• Limit 4 AI cannot do sophisticated algorithm on
  the fly (thus we store knowledge)
• Avoid over-fitting (learning the wrong thing)
• Neural network, Markov
• Improvement learn from just the new datum
• Overfitting because of small new dataset!
• Neural network can compensate by temporal
  differences

15
Using new Knowledge

• Limit 5 If games are not designed to use new
  knowledge, new storage and capability will need
  to be added
• Games should design to exploit new knowledge in
  an integrated fashion
• CC again, where new data is integrated into
  pathfinder
• BW, new data determines behavior of monster

16
Conclusion

• Machine Learning present in
• In field (player may or may not be aware)
• In development
• Knowledge can be learned
• Explicitly or implicitly
• Limits of Machine Learning
• The cost to see / recognize knowledge
• The cost to store
• The cost to use
• Does it make the game better

17
Limits in Use of Pattern Recognition

• Limits of Machine Learning
• The cost to see / recognize knowledge
• Expensive!!
• The cost to store
• The cost to use ? Could be expensive
• Does it make the game better ? hopefully
• Usually in development
• Black and White is in field

18

• Unfortunately, in the development of AIs for CGs
  the scope of pattern recognition has not been
  widely realizedOur motivation is to do pattern
  recognition ourselves to discern where pattern
  recognition can be applied in CG Hakonen
• Many classic academic games such as Chess (Deep
  Blue), Backgammons, Go made use pattern
  recognition and achieved good result.
• Many games do use neural networks, but does not
  necessarily use pattern recognition.

19

• 10.3 Understanding Pattern Recognition Methods

20
Intro to Pattern Recognition 1

• Wikipedia
• Pattern recognition can be defined as "the act
  of taking in raw data and taking an action based
  on the category of the data" 1.
• image analysis, character recognition, speech
  analysis, man and machine diagnostics, person
  identification and industrial inspection.

21
Intro to Pattern Recognition 2

• Textbook Definition (in the context of CG)
• To abstract relevant information from the game
  world and, based on the retrieved information,
  construct concepts and deduce patterns for the
  use of higher level reasoning and decision-making
  systems.

22
Intro to Pattern Recognition 3
Abstract relevant info, compare to old info in
KB, deduce pattern
Choose from a list of possible actions available
in the current state, balance it to the requested
state
23
Questions

• 1. Why do we need a decision-making system? Why
  dont we just simply map patterns to some
  reaction? (Hint Do we know everything about the
  environment to make the decision?)

• The world is not deterministic..
• Built-in randomness
• Human players action

2. Where do you think PR can be applied for
computer games?
24
Intro to Pattern Recognition 4

• Some suggested uses of pattern recognition
• During game
• Enemy evaluation and prediction
• Coaching
• Group coordination
• Terrain analysis
• Learning
• During development

• Black and White
• Command and Conquer Renegade (attempted)
• Re-Volt

25

• Functional Approach.
• How is the use of PR differ with
• Levels of decision making ?
• Stance towards the player ?

26
Functional Level of Decision Making

• Suitability of pattern recognition depends on the
  level of decision making
• Strategic Level
• Tactical Level
• Operational Level

27
Functional Level of Decision Making
28
Functional - Stance

• Enemy
• Provide Challenge
• Demonstrate Intelligent (at least purposeful)
  behavior
• PR to aid computers decision making
• Prediction and production

29
Functional - Stance

• Ally,
• Augment user interface
• Hints and guide
• Aiding the human player
• End result should be visually accessible and
  consistent, but not necessarily complete

30
Functional - Stance

• Neutral
• Context dependent
• Commentory
• highlighting events and providing background
  highlighting events and background information
• soccer referee
• Judging rule violation

31
Functional Stance

• Modeled Language?
• A generator labels events and states with symbol
• Modeling recognizes the underlying dependencies
  between symbols
• Short term history sufficient

32
Functional Prediction

• Predict next symbol, calculate probability of the
  next action of opponent

33
Functional Production

• observe opponent and produce next symbol (memcpy)

34

• PR as these problems
• Optimization
• Adaptation
• Uncertainty (not going to talk about)
• Algorithm used in each problem area
• How the level of decision making decides what PR
  algorithm to use

35
Methodology - Optimization

• Optimization
• an objective function (to max or min)
• A set of variables
• A set of constraints
• Pattern recognition as an optimization problem is
  to have an objective function to rank the
  solution candidates

36
Methodology - Optimization
37
Methodology - Optimization

• Project 6 tweak AI to beat some opponents
• Marcs presentation the tweaking could be done
  automatically
• This is related to pattern recognition
• Trying to identify strengths and weaknesses of
  opponents

38
Methodology - Optimization

• Age of Empire Example
• Objective Balance civilizations and units
• A combat comparison simulator is used to test
  battles with different troop combinations
• The set of variables Attributes (armor, hit
  point, damage, range)
• The set of constraints the range of allowed
  values
• Objective Function min(difference of the number
  of victories of different civilizations in
  simulator battles)
• Attributes are changed to even out discrepancies

39
(No Transcript)
40
(No Transcript)
41
These slides are taken from Street
42
Methodology - Optimization

• How to find more optimal variable values?
• Heuristic functions
• Fastest and simplest
• Get stuck at local optimum before finding global
• Use multiple search traces instead of one

43
Methodology - Optimization

• How to find more optimal variable values and
  avoid local optimum?
• Genetic Algorithm
• Avoid local optimum. A population of candidate
  solutions, go thru stages of natural selection,
  objective function is to weed out the weak
  candidates
• Vulnerable to dependency of variables, CPU
  exhaustive

44
Methodology - Optimization

• Another way to find more optimal variable values
  and avoid local optimum?
• Swarm Algorithm
• Avoid local optimum (if min speed is set)
• faster than genetic algorithm
• next 3 slides on Particle Swarm Optimization

45
Particle Swarm Optimization 1

• PSO is very similar to genetic algorithm, but
  does not have genetic operators like crossover
  and mutation. Particles update themselves with
  the internal velocity.
• The idea is similar to bird flocks searching for
  food, dont know where the food is, but know how
  far from it
• pbest and gbest are stored in memory

Bird particle, Food optimal solution pbest
the best solution (fitness) a particle has
achieved so far. gbest the global best
solution of all particles
46
Particle Swarm Optimization 2

• v v0 c1 rand() (pbest - present)
  c2 rand() (gbest - present)
  -----------(a)
• present present0 v ------------------(b)
  

v the particle velocity present the
current particle (solution). rand () a random
number between (0,1). c1, c2 learning
factors. usually c1 c2 2.
http//uk.geocities.com/markcsinclair/pso.html
47
Optimization the Level of Decision Making

• Strategic Level
• Computationally hard to solve
• With relaxed constraint that the variables are
  not interdependent, then genetic algorithm
• Tactical Level
• Must be responsive
• Single trace, heuristic functions
• Operational Level
• Real-time,
• simple objective function (switch statement)

48
Methodology - Adaptation

• Adaptation
• The ability to make appropriate responses to
  changed or changing circumstances
• Try to model the originator of the modeled data

49
Methodology - Adaptation

• Adaptation vs. Optimization
• Adaptation
• looks for a function behind a solution,
• Optimization
• looks for a solution for a given function

50
Methodology - Adaptation

• When is adaptation useful? Why is it hard to use?

• Useful when the affecting factors or mechanisms
  behind the phenomena is unknown or dynamic
• 2. Hard to use because it requires sampling the
  search space to cover sufficiently. The more
  complex the cause is, the sparser the our sample
  gets (combinatorial explosion)

51
Methodology - Adaptation

• Neural Networks
• Can adapt to situation where we do not have
  background knowledge of dependencies
• Supervised (predefined categories for results) or
  unsupervised learning
• For more information, please read Chp11
• Hidden Markov Model
• System is conditionally independent of the past
  states
• each state has a probability distribution over
  the possible output tokens
• the challenge is to determine the hidden
  parameters from the observable parameters
• Can adapt to recurring structure

52
Adaptation and the level of decision making

• Strategic Level
• Supervised or unsupervised learning
• Can afford great computational demand
• Tactical Level
• Hidden Markov models
• More dynamic environment
• Credibility of results can be evaluated
• Operation Level
• Stochastic interpretation for input data, or
• Use a ready-adapted neural network

53
Conclusion

• Pattern Recognition
• PR is not well explored in gaming yet
• Biggest limit is computational complexity
• A lot of time is domain dependent
• Algorithms used depend highly on required
  responsiveness
• Functional Approach
• Level of decision making
• Stance
• Methodological Approach
• Optimization
• Adaptation
• Uncertainty ( not discussed )

54
References

• Hu, Xiaohui. Particle Swarm Optimization
  Tutorial, http//www.swarmintelligence.org/tutor
  ials.php
• Fraser, Neil. Neural Network Follies.
  http//neil.fraser.name/writing/tank/
• German steps, number 1-10. http//www.bbc.co.uk/la
  nguages/german/lj/language_notes/1_10.shtml
• Shutton, R. Learning to Predict by the Method
  of Temporal Differences, ftp//ftp.cs.umass.edu/p
  ub/anw/pub/sutton/sutton-88.ps.gz
• Kidd, Petersen, Street, How to Balance a
  Real-Time Strategy Game Lessons from the Age of
  Empires Series, http//www.gdconf.com/archives/2
  001/gstreetprintable3.ppt
• Timo Kaukoranta, Jouni Smed, Harri Hakonen.
  Role of Pattern Recognition in Computer Games.
  http//staff.cs.utu.fi/staff/jouni.smed/papers/PRi
  nCG.pdf