Lecture

1
Lecture 9Computer Game Design AI and Finite
States Machines

• Abdennour El Rhalibi

2
What is AI?

• AI is the control of every non-human entity in a
  game
• The other cars in a car game
• The opponents and monsters in a shooter
• Your units, your enemys units and your enemy in
  a RTS game
• AI controls and agent
• But, typically does not refer to passive things
  that just react to the player and never initiate
  action
• Thats physics or game logic
• For example, the blocks in tetris are not AI, nor
  is a flag blowing in the wind
• Its a somewhat arbitrary distinction

3
AI in the Game Loop

• AI is updated as part of the game loop, after
  user input, and before rendering
• There are issues here
• Which AI goes first? The state it sees will
  differ from the last AI to get updated
• Does the AI run on every frame? Typically, no
• The things that AI depends on may not change
  quickly
• The AI may be purely event driven
• Is the AI synchronized? Not necessarily
• Update different AI agents on different frames

4
AI and Animation

• AI determines what to do and the animation does
  it
• AI drives animation, deciding what action the
  animation system should be animating
• Scenario 1 The AI issues orders like move from
  A to B, and its up to the animation system to
  do the rest
• Scenario 2 The AI controls everything down to
  the animation clip to play
• Which scenario is best depends on the nature of
  the AI system and the nature of the animation
  system
• Is the animation system based on move trees
  (motion capture), or physics, or something else
• Does the AI look after collision avoidance? Does
  it do detailed planning?

5
AI Update Step

• The sensing phase determines the state of the
  world
• May be very simple - state changes all come by
  message
• Or complex - figure out what is visible, where
  your team is, etc
• The thinking phase decides what to do given the
  world
• The core of AI
• The acting phase tells the animation what to do
• Generally not interesting

AI Module
Sensing
Game Engine
Thinking
Acting
6
Event Driven AI

• Event driven AI does everything in response to
  events in the world
• Events sent by message (basically, a function
  gets called when a message arrives, just like a
  user interface)
• Example messages
• A certain amount of time has passed, so update
  yourself
• You have heard a sound
• Someone has entered your field of view
• Note that messages can completely replace
  sensing, but typically do not.
• Real system are a mix - something changes, so you
  do some sensing

7
AI Techniques in Games

• Basic problem Given the state of the world, what
  should I do?
• A wide range of solutions in games
• Finite state machines, Decision trees, Rule based
  systems, Neural networks, Fuzzy logic
• A wider range of solutions in the academic world
• Complex planning systems, logic programming,
  genetic algorithms, Bayes-nets
• Typically, too slow for games

8
Goals of Game AI

• Several goals
• Goal driven - the AI decides what it should do,
  and then figures out how to do it
• Reactive - the AI responds immediately to changes
  in the world
• Knowledge intensive - the AI knows a lot about
  the world and how it behaves, and embodies
  knowledge in its own behavior
• Characteristic - Embodies a believable,
  consistent character
• Fast and easy development
• Low CPU and memory usage
• These conflict in almost every way

9
Two Measures of Complexity

• Complexity of Execution
• How fast does it run as more knowledge is added?
• How much memory is required as more knowledge is
  added?
• Determines the run-time cost of the AI
• Complexity of Specification
• How hard is it to write the code?
• As more knowledge is added, how much more code
  needs to be added?
• Determines the development cost, and risk

10
Expressiveness

• What behaviors can easily be defined, or defined
  at all?
• Propositional logic
• Statements about specific objects in the world
  no variables
• Jim is in room7, Jim has the rocket launcher, the
  rocket launcher does splash damage
• Go to room8 if you are in room7 through door14
• Predicate Logic
• Allows general statement using variables
• All rooms have doors
• All splash damage weapons can be used around
  corners
• All rocket launchers do splash damage
• Go to a room connected to the current room

11
Levels of AI

• Empirical AI, hacking, procedural/scripted
  solutions
• Principled solutions based on AI algorithms (e.g.
  A for path planning)
• Real-time, state-of-the-art AI (e.g. heuristic
  search planning for agent behaviour)

12
Levels of AI (2)

• Game AI problems tend to be real-time AI
  problems
• Efficient algortihms
• Efficient representations
• gt 80 of Game AI problems have been described in
  Robotics or Agent-based systems

13
AI in Video Games / Genre

• Strategy games (C C, etc.)
• Adventure games and shoot em ups (opponent
  behaviour)
• Simulation (e.g. Theme Hospital, Populous,
  Civilization, The Sims, Republic The Revolution
  ...)
• Sports (e.g. football, car racing)

14
Uses of AI in Video Games

• Classical problems
• plan a path for an NPC within the game level
• define behaviour for an NPC (e.g. in Quake)
• define strategies and tactics in Wargames
• define collective behaviours for NPC
• Metal Gear Solid, Half Life, X-Squad
• all sports games (e.g. football)
• I/O for complex games (speech recognition, etc.)

15
AI in Video Games main techniques

• Behaviours planning, Finite-State-Machines
  (FSM), Rule-based systems (Quake)
• Path Planning search algorithms (e.g. A)
  (Half-Life)
• Strategy / Tactics Rule-based Systems (Age of
  Empires)
• Simulation multi-agent co-ordination, rule-based
  systems (Civilization)

16
Defining Behaviours (history)
state1 sit(guard1) if (the player
comes near) goto state2
state2 attack(guard1, player) if
(isDead(guard1)) stop
Interstate 76, Activision
17
Current Techniques FSTN
Team Buddies (SCEE)
18
The QUAKE FSA
19
Rule-Based Systems
IF enemy visible and my health is lt
very-low-health-value (20) OR his weapon is
much better than mine THEN propose retreat
(SOAR-Quake, courtesy of J. Laird)
20
Rule-Based Systems

• In Rule-Based Systems, data representation is
  declarative (i.e., distinct from the procedures
  that manipulate them)
• This allows flexible description of behaviours
• Examples IF-THEN rules (based on Production
  Systems)
• (if (no powerful weapons)
• (powerful opponent)
• (opponent near barrel)
• then (shoot barrel))

21
Non-Standard Behaviours
The Sims (Electronic Arts)
22
Search and Path Planning

• A significant number of games involve moving in
  buildings, battlefields, etc.
• Choosing the best path within a determined space
  is a classic AI problem
• Algorithms have been developed that construct
  the best path by exploring possible directions on
  the basis of individual cells properties

23
Search and Path Planning

• Path planning for NPC
• Main steps
• discretise the environment
• use a search algorithm (e.g. A)

24
Search Algorithms

• Real-time search algorithms (dynamic
  environments) RTA
• Two-player games (adversarial search) MiniMax

25
Multi Agents Systems

• Some games (simulation, etc.) are based on the
  interaction of elementary agents
• It is thus possible to obtain complex global
  behaviours from the aggregation of individual
  behaviours
• Multi agents systems are also a problem-solving
  technique (in games they are used because they
  are isomorphic to the situation)

26
Multi-Agent Co-ordination (1)

• Games with large numbers of NPC
• (Dropship, SCEE 400 entities)
• Co-ordination between agents and with human
  player (allied units)

27
Multi-Agent Co-ordination (2)
28
Virtual Actors (1)
AnanovaÔ
Kyoko DateÔ
Webbie TookayÔ
29
Virtual Actors (2)
Republic The Revolution
30
Virtual Actors (3)
ã University of Southern California, Institute
for Creative Technologies
31
Virtual Actors (4)
32
AI-based Animation (1)
33
AI-based Animation (2)
34
Natural Language Processing (1)

• In the future, it should be possible to control
  video games through natural language (e.g. speech
  recognition)
• Besides the accurate recognition of speech (see
  dictation systems) there is a need to actually
  understand what the sentence means in terms
  of game commands

35
Natural Language Processing (2)
36
Machine Learning (1)

• Machine learning is about the automatic creation
  of knowledge representations from experience
  (i.e. actual playing sessions)
• In the case of video games, artificial opponents
  could learn about the style of the human
  player. It is the machine that learns!

37
Machine Learning (2)
Virtua Fighter 4 (Sega)
Artificial Contender (Trusoft)
38
Finite State Machines (FSMs)

• A set of states that the agent can be in
• Connected by transitions that are triggered by a
  change in the world
• Normally represented as a directed graph, with
  the edges labeled with the transition event
• Ubiquitous in computer game AI
• You might have seen them, a long time ago, in
  formal language theory (or compilers)
• What type of languages can be represented by
  finite state machines?
• How might this impact a characters AI?
• How does it impact the size of the machine?

39
Quake Bot Example

• Types of behavior to capture
• Wander randomly if dont see or hear an enemy
• When see enemy, attack
• When hear an enemy, chase enemy
• When die, respawn
• When health is low and see an enemy, retreat
• Extensions
• When see power-ups during wandering, collect them

40
Example FSM

• States
• E enemy in sight
• S sound audible
• D dead
• Events
• E see an enemy
• S hear a sound
• D die
• Action performed
• On each transition
• On each update in some states (e.g. attack)

Spawn D
41
Example FSM Problem

• States
• E enemy in sight
• S sound audible
• D dead
• Events
• E see an enemy
• S hear a sound
• D die

Spawn D
Problem Cant go directly from attack to chase.
Why not?
42
Better Example FSM

• States
• E enemy in sight
• S sound audible
• D dead
• Events
• E see an enemy
• S hear a sound
• D die
• Extra state to recall whether or not heard a
  sound while attacking

Spawn D
43
Example FSM with Retreat
Attack-ES E,-D,S,-L
Retreat-S -E,-D,S,L
Attack-E E,-D,-S,-L
S

• States
• E enemy in sight
• S sound audible
• D dead
• L Low health
• Worst case Each extra state variable can add 2n
  extra states
• n number of existing states

L
-S
L
E
-L
-E
E
Retreat-ES E,-D,S,L
-L
E
Wander-L -E,-D,-S,L
-L
E
L
-S
-L
S
L
Retreat-E E,-D,-S,L
Wander -E,-D,-S,-L
-E
-E
E
D
D
Chase -E,-D,S,-L
D
D
Spawn D (-E,-S,-L)
S
44
Hierarchical FSMs

• What if there is no simple action for a state?
• Expand a state into its own FSM, which explains
  what to do if in that state
• Some events move you around the same level in the
  hierarchy, some move you up a level
• When entering a state, have to choose a state for
  its child in the hierarchy
• Set a default, and always go to that
• Or, random choice
• Depends on the nature of the behavior

45
Hierarchical FSM Example
Attack
Wander
E
E
Chase
Pick-up Powerup
S
S
Spawn
Start
Turn Right
D
E

• Note This is not a complete FSM
• All links between top level states still exist
• Need more states for wander

Go-through Door
46
Non-Deterministic HierarchicalFSM (Markov Model)

• Adds variety to actions
• Have multiple transitions for the same event
• Label each with a probability that it will be
  taken
• Randomly choose a transition at run-time
• Markov Model New state only depends on the
  previous state

Attack
Start
47
FSM Advantages

• Very fast one array access
• Expressive enough for simple behaviors or
  characters that are intended to be dumb
• Can be compiled into compact data structure
• Dynamic memory current state
• Static memory state diagram array
  implementation
• Can create tools so non-programmer can build
  behavior
• Non-deterministic FSM can make behavior
  unpredictable

48
FSM Disadvantages

• Number of states can grow very fast
• Exponentially with number of events s2e
• Number of arcs can grow even faster as2
• Propositional representation
• Difficult to put in pick up the better powerup,
  attack the closest enemy
• Expensive to count Wait until the third time I
  see enemy, then attack
• Need extra events First time seen, second time
  seen, and extra states to take care of counting

49
References

• Web references
• www.gamasutra.com/features/19970601/build_brains_i
  nto_games.htm
• csr.uvic.ca/mmania/machines/intro.htm
• www.erlang/se/documentation/doc-4.7.3/doc/design_p
  rinciples/fsm.html
• www.microconsultants.com/tips/fsm/fsmartcl.htm

50
General References

• AI
• Russell and Norvig Artificial Intelligence A
  Modern Approach, Prentice Hall, 1995
• Nilsson, Artificial Intelligence A New
  Synthesis, Morgan Kaufmann, 1998
• AI and Computer Games
• LaMothe Tricks of the Windows Game Programming
  Gurus, SAMS, 1999, Chapter 12, pp. 713-796
• Games Developers Conference (GDC, www.gdconf.org)
• AAAI Spring Symposium in AI and Computer Games
  (www.aaai.org)
• Game-ON Conference
• www.gameai.com
• www.gamedev.net