Player Modeling

1
Player Modeling

• Chris Janneck
• 11/15/04

2
Overview

• Player Modeling for Adaptive Games
• By Ryan Houlette
• About the author
• The point of modeling
• Regarding AI
• The simplest model
• A higher-level model
• More complex modeling
• Concerns with modeling and AI

3
Ryan Houlette

• Stottler Henke Associates
• http//www.shai.com
• Developer of AI systems
• Intelligent Tutoring, Scheduling, KM, Security,
  etc.
• SimBionic AI agent toolkit
• Lead Software Engineer
• Co-developer of BrainFrame system
• AI SDK for entertainment

4
SimBionic

• (Include AVI from www.simbionic.com/download.htm)

5
BrainFrame

• (Include Figures 1 and 2 from Putting AI in
  Entertainment)

6
The Point of Modeling

• Why model?
• To create a representation of some item or
  phenomena that can be (more) easily explored,
  tweaked, observed and tested
• Where do we model?
• Engineering, Design, Simulations, Manufacturing,
  Business, Mathematics
• And, yes, in AI
• Model-based reasoning, instead of Case-based
  reasoning

7
Modeling in AI

• So far, discussed modeling of computer AI
• FSMs, Behaviors and Knowledge Rep/Sharing, Maps
  and Movement,
• Today, discuss modeling of human player
• Why? Arent humans inferior to our AI?
• For entertainment, player must have fun
• Not fun ? not played ? bad rep ? game bombs
• Fun(g) Challenge(g) Usability(g)
  Replayability(g) SenseTickling(g)

8
Goals of Player Modeling

• Maximizing Challenge()
• Use player model to generate actions that exploit
  players weaknesses
• Maximizing Replayability()
• Since model is dynamic, AI never acts the same
  way twice always a different game
• Improving SenseTickling()
• Player thinks AI is good or smart if it
  adapts well to the player
• Tickling the brain-sense

9
The Simplest Model

• What is the simplest kind of model?
• Model (attribute, value), (attr2, val2)
• Model is a class of action/trait counters
• A trait value is modified when observed
• Value increase when action observed
• Value may decrease over time, or to maintain
  normalization
• Example traits
• UsesSmokeGrenades, AvoidsMovingThroughEnemyChok
  epoints, MeleeAttacks, AutomaticAttacks,
  RocketAttacks, Slipstreams,
  BreaksIntoTurns, AcceleratesThroughTurns,
  OverSpeedLimit, EatsPizza, etc

10
The Simplest Model Code

• (include code from 559, 560)

11
The Simplest Model Updating

• Recognize when update should be made
• Can be easy
• UsesLaser count number of times fired
• Or more difficult
• CanDoTrickyJumps if player successfully makes
  known difficult jumps on map
• CanDoWallJumps if player successfully jumps
  gt1x and lands on higher ground
• Instruct model to update
• Least Mean Squares training
• traitValue a observedValue (1-a)
  traitValue
• Normalize values
• 0 lt x lt 1
• 0 never happens, 1 all the time, 0.5 dont
  know

12
Evaluating Simple Models

• Benefits
• Easy to code and use
• Getting and updating values straightforward
• Good for statistics-tracking and display
• Shortcomings
• Difficult to model higher-level behavior
• AI choices will have to use many model traits and
  computations
• Ex Which side to defend? What units to
  produce?

13
A Higher-Level Model

• Hierarchical Modeling
• Higher-level node value combination of childrens
  values
• Abstract node combination of concrete traits

14
Hierarchical Player Model

• (include Figure 10.1.1 from 10.1)

15
Hierarchical Issues

• Similar to other hierarchical structures
• Like Troop-based AI tactics
• Propagation
• If values of concrete, leaf-nodes change, must
  propagate values up the tree
• Easier faster model state checking
• Can scan abstract nodes quickly instead of
  revisiting all of the concrete counters

16
More Complex Modeling

• Extract and extrude hierarchical model
• Can achieve wide range of modeling by tweaking
• What is abstracted?
• How higher-levels are computed from lower?
• Of what do leaf nodes keep track?
• How are leaf nodes designed?

17
Possible Tweaks

• Leaf nodes track state instead of counter
• Represent each leaf as a FSM
• Abstract nodes predict instead of report
• Represent higher-level nodes as N-grams
• Probabilistic directed graphs of frequencies of
  sequences
• A node to monitor force locations
• Leafs may return a TroopLocation class
  (containing (x,y)s of units) instead of a value

18
Modeling Issues

• Model complexity-time tradeoff
• More complicated more time spent
• Still need to reserve time for AI
  decision-making, rendering, and less-important
  things
• Decouple model from game good
• If model is data-driven (i.e. from an external
  file), can be modified without re-compiling
• If model is lightly-coupled to rest of AI, even
  better

19
Utilization of the Model

• AI queries model, uses result in decision-making
• Examples
• If determining which kind of weapon to use,
  check
• Players weapon proficiency/usage counts
• If determining what kind of attack to use
  (flanking, head-on, air-based, etc.), check
• Which attacks have been most often repelled
• Which attacks the Player uses
• Where are units located
• If determining how to allocate patrols, check
• If Player is more stealthy, put more in hidden
  paths, or detect better in shadow
• If Player is more Rambo, put more in open areas,
  with heavier artillery

20
Design of the Model

• Habits and Preferences semantics
• UsesSmokeGrenades means how often player uses
  this weapon
• Knowledge and Proficiency semantics
• UsesSmokeGrenades means how good the player
  feels about using this weapon
• There is a subtle, but important difference
• If UsesSmokeGrenades 1,
• What does that mean?
• How should the AI respond?

21
Other Player Model Uses

• Allow exporting / transferring player model
• If multiple games use same model (or similar
  nodes), a new game can more quickly adapt
• Allow multiplayer games to build more generic
  model, based on several players
• Allow player to view model
• Lets them see where they can/should improve
• Level completed statistics screen
• May reveal some intricacies about AI

22
References

• Introduction to Learning in Games
• http//www.lupinegames.com/articles/introlearn.htm