CMPSEM022 Advanced Games Programming

1
CMPSEM022 Advanced Games Programming

• Lecture 5 Applying Artificial Intelligence to
  Games

2
In this session

• The role of AI in games
• AI processing in the game engine loop
• Finite State Machines
• Implementation techniques
• Rule-based systems
• Neural networks
• Genetic algorithms
• Fuzzy logic
• Influence mapping
• Flocking

3
What is AI?

• The study of computational systems that exhibit
  intelligence
• Theories and computational models
• What is intelligence?
• What people do
• Leads to the Turing test
• Hard to separate out intelligent behaviour from
  other behaviour
• Behave rationally use available knowledge to
  maximise goal achievement
• Often leads to optimisation techniques
• A set of capabilities
• Problem solving, learning, planning, etc.

4
What is AI?

• AI is the control of every non player character
  (NPC) in a game
• The other cars in a car game
• The opponents and monsters in a first-person
  shooter
• Your units, your enemys units and your enemy in
  a RTS game
• But, typically does not refer to passive things
  that just react to the player and never initiate
  action
• Thats physics or game logic
• For example, the blocks in Tetris are not AI, nor
  is a flag blowing in the wind
• Its a somewhat arbitrary distinction

5
AI roles in games

• Opponents
• Teammates
• Support characters
• Autonomous characters
• Commentators
• Camera control
• Plot and story guides/directors

6
Goals of an AI action game opponent

• Provide a challenging opponent
• Not always as challenging as a human Quake
  monsters
• Not too challenging
• Should not be superhuman in accuracy, precision,
  sensing, etc.
• Should not be too predictable
• Through randomness
• Through multiple, fine-grained responses
• Through adaptation and learning

7
Goals of game AI

• Several goals
• Goal driven the AI decides what it should do,
  and then figures out how to do it
• Reactive the AI responds immediately to changes
  in the world
• Knowledge intensive the AI knows a lot about
  the world and how it behaves, and embodies
  knowledge in its own behaviour
• Characteristic embodies a believable,
  consistent character
• Fast and easy development
• Low CPU and memory usage
• These conflict in almost every way

8
AI and the game engine

• AI determines what to do and the game engine
  determines how to do it
• The AI drives the game engine, deciding what
  action the game engine should be displaying
• Example The AI issues orders like move from A
  to B, and its up to the game engine to do the
  rest
• AI usually goes through 3 distinct steps
• Sensing
• Thinking
• Acting

9
AI update processing

• The sensing phase determines the state of the
  world
• May be very simple state changes
• Or complex - figure out what is visible (LOS),
  where your team is, etc.
• The thinking phase decides what to do given the
  world state
• The core of AI
• The actual processing of the information
• The acting phase tells the game engine what to do
• Update positions, increase/decrease health,
  remove object, etc.

AI Module
Sensing
Game Engine
Thinking
Acting
10
AI techniques in games

• Basic problem Given the state of the world, what
  should I do?
• A wide range of solutions in games
• Finite State Machines (FSM), Decision trees, Rule
  based systems, Neural networks, Fuzzy logic
• A wider range of solutions in the academic world
• Complex planning systems, logic programming,
  genetic algorithms, etc.
• Typically, too slow for games

11
Simple behaviour

• Random motion
• Just roll the dice to pick when and which
  direction to move
• Simple pattern
• Follow invisible tracks Galaxians
• Tracking
• Pure Pursuit Move toward agents current
  position
• Head seeking missile
• Lead Pursuit Move to position in front of agent
• Collision Move toward where agent will be
• Weave Every N seconds move X degrees off
  opponents bearing
• Spiral
• Evasive opposite of any tracking

12
Pure pursuit
Missile tracks current position of aircraft at
each time that sensing routine is performed.
t3
t2
t1
13
Lead pursuit
Missile tracks current position of aircraft and
tries to head it off at the pass. Needs to
determine aircrafts current velocity and
direction.
t3
t2
t1
14
Collision
Missile tracks current and previous positions of
aircraft in order to compute a collision course.
t3 predicted
t2
t1
15
Two measures of complexity

• Complexity of execution
• How fast does it run as more knowledge is added?
• How much memory is required as more knowledge is
  added?
• Determines the run-time cost of the AI
• Complexity of specification
• How hard is it to write the code?
• As more knowledge is added, how much more code
  needs to be added?
• Determines the development cost, and risk

16
Expressiveness

• What behaviours can easily be defined, or defined
  at all?
• Propositional logic
• Statements about specific objects in the world
  no variables
• Jim is in room7, Jim has the rocket launcher, the
  rocket launcher does splash damage
• Go to room8 if you are in room7 through door14
• Predicate logic
• Allows general statement using variables
• All rooms have doors
• All splash damage weapons can be used around
  corners
• All rocket launchers do splash damage
• Go to a room connected to the current room

17
Finite State Machines (FSMs)

• A set of distinct states that the agent can be in
• Connected by transitions that are triggered by a
  change in the world
• Normally represented as a directed graph, with
  the edges labeled with the transition event
• Ubiquitous in computer game AI
• You might have seen FSM in formal language theory
  (or compiler design)

18
Quake example

• Types of behaviour to capture
• Wander randomly if dont see or hear an enemy
• When see enemy, attack
• When hear an enemy, chase enemy
• When die, respawn
• When health is low and see an enemy, retreat
• Extensions
• When see power-ups during wandering, collect them

19
Example FSM

• States
• E enemy in sight
• S sound audible
• D dead
• Events
• E see an enemy
• S hear a sound
• D die
• Action performed
• On each transition
• On each update in some states (e.g. attack)
• Problem Cant go directly from attack to chase.
  Why not?

Spawn D
NOT, i.e. E means not an enemy in sight
20
Better example FSM

• States
• E enemy in sight
• S sound audible
• D dead
• Events
• E see an enemy
• S hear a sound
• D die
• Extra state to recall whether or not heard a
  sound while attacking

Spawn D
21
Example FSM with retreat
Attack-ES E,D,S,L
Retreat-S E,D,S,L

• States
• E enemy in sight
• S sound audible
• D dead
• L Low health
• Worst case Each extra state variable can add 2n
  extra states
• n number of existing states

Attack-E E,D,S,L
S
L
S
L
E
L
E
E
Retreat-ES E,D,S,L
L
E
Wander-L E,D,S,L
L
E
L
S
L
S
L
Retreat-E E,D,S,L
Wander E,D,S,L
E
E
E
D
D
Chase E,D,S,L
D
D
Spawn D E,S,L
S
22
Hierarchical FSMs

• What if there is no simple action for a state?
• Expand a state into its own FSM, which explains
  what to do if in that state
• Some events move you around the same level in the
  hierarchy, some move you up a level
• When entering a state, have to choose a state for
  its child in the hierarchy
• Set a default, and always go to that
• Or, random choice
• Depends on the nature of the behaviour

23
Hierarchical FSM Example
Attack
Wander
E
Chase
Pick-up Powerup
E
S
S
Spawn
Start
Turn Right
D
E
Go-through Door

• Note This is not a complete FSM
• All links between top level states still exist
• Need more states for wander

24
Non-deterministic hierarchicalFSM (Markov Model)

• Adds variety to actions
• Have multiple transitions for the same event
• Label each with a probability that it will be
  taken
• Randomly choose a transition at run-time
• Markov Model New state only depends on the
  previous state

Attack
Start
25
FSM advantages

• Very fast one array access
• Expressive enough for simple behaviours or
  characters that are intended to be dumb
• Can be compiled into compact data structure
• Dynamic memory current state
• Static memory state diagram array
  implementation
• Can create tools so non-programmer can build
  behaviour
• Non-deterministic FSM can make behaviour
  unpredictable

26
FSM disadvantages

• Number of states can grow very fast
• Number of arcs can grow even faster
• Propositional representation
• Difficult to put in pick up the better powerup,
  attack the closest enemy
• Expensive to count Wait until the third time I
  see enemy, then attack
• Need extra events First time seen, second time
  seen, and extra states to take care of counting

27
Implementing FSMs in C/C
Use enum type to define states enum
QuakeBotState BOT_WANDERING, BOT_CHASING,
BOT_ATTACKING, BOT_RETREATING,
BOT_SPAWNING Player can only be in one state
at one time so we can store the state value in a
single variable.
struct QuakeBot int x_loc, y_loc, z_loc
QuakeBotState current_state int
health Here we have defined a struct to hold
all the necessary information about a given
QuakeBot. In C we could wrap this into a class
instead.
QuakeBot bot1 if (bot1.current_state
BOT_WANDERING) ememy_in_sight())
bot1.current_state BOT_ATTACKING attack()
28
Implementing FSMs in DarkBASIC
REM Define some states DEFINE wander,0 DEFINE
attack,1 REM Define a structure to hold
information about 'bots TYPE QuakeBot x_loc,
y_loc, z_loc health current_state ENDTYPE R
EM Declare a variable, bot1, of type
QuakeBot DIMTYPE QuakeBot,bot1 REM Modify a
value in the bot's structure bot1.current_state
wander REM Output a value depending on the
'bot's current state IF (bot1.current_state
wander) THEN PRINT "Wandering"
DarkEDIT specific commands
29
Rule-based systems

• Separate memories for long-term memory with
  rules, short-term memory of sensors and current
  interpretation
• Match rules against short-term memory, and pick a
  rule to fire
• Rules change short-term memory

Rule memory
Short-term memory
Sensors
Actions
30
Rules for behaviour

• If health is low, then want to run away
• If health is high, then want to chase
• If close to enemy, then attack
• If want to run away, and opponent bearing is x,
  then head in direction x 180
• If want to chase, and opponent bearing is x,
  then head in direction x
• Relatively easy to code
• Nested if statements or switchcase statement

31
Neural networks

• Computational models of networks of neural-like
  nodes
• Can be trained need not program in best
  response
• Present lots of examples of situations and
  responses
• Adjusts weights between nodes so it predicts
  responses
• Will learn, even generalise
• Weaknesses
• Takes lots of training, lots of examples
• Cant be easily used for planning
• Difficult to learn to manage multiple goals,
  perform sequential actions
• Hard to pre-program with known facts
• Hard to understand what they are doing and why
• Cant act on prior sensing only current
• Examples Colin McRae Rally

32
Neural net example
INPUT
OUTPUT
Clear?
Run
HIDDEN LAYER
Health?
Shoot
Distance to enemy?
Walk
Size of enemy?
Turn Left
Ammo?
Turn Right
33
Genetic Algorithms (GAs)

• Genetic algorithms are based on biological
  natural evolution - including reproduction,
  mutation and natural selection
• The architecture of systems that implement
  genetic algorithms (or GA) are more able to adapt
  to a wide range of problems
• A GA functions by generating a large set of
  possible solutions to a given problem
• It then evaluates each of those solutions, and
  decides on a "fitness level" ("survival of the
  fittest") for each solution set
• These solutions then breed new solutions
• The parent solutions that were more "fit" are
  more likely to reproduce, while those that were
  less "fit" are more unlikely to do so
• In essence, solutions are evolved over time

34
Fuzzy logic

• The answer to some questions is a Boolean value
  true or false
• However, in most cases there is also another
  element, uncertainty
• The major difference between fuzzy logic and
  Boolean (standard) logic is that possible values
  range from 0.0 to 1.0 (inclusive), not just 0 and
  1
• For example, you could say that the fuzzy truth
  value (FTV) of the statement "Graham is tall" is
  0.75 if Graham is 2 metres tall

35
Influence mapping

• An influence map is a spatial representation of
  an AI agents knowledge of the world
• Indicates where various objects are friend or
  foe, past events, etc.
• Influence maps can pick out areas of strategic
  control, weak spots, prime camping locations,
  interesting terrain points, etc.
• Superimpose a 2D square or hexagonal grid over
  the playing area
• Add influences
• Compute effectiveness

36
Step 1 Plot the positions of known objects (good
and bad) on the world map and assign them
weightings (ve being a good influence and ve
being a bad influence).
Step 2 Add influence scores for each cell in the
world map grid based on the surrounding objects.
In this example the influence diminishes the
further away from the source it gets. Repeat this
for all significant objects in the world map.
Step 3 Compute the effective influence by
combining the influence maps for all objects
(summation). The influence map now highlights the
boundary between safety and harm.
37
Flocking

• Flocking refers to the behaviour of certain types
  of animal when moving or hunting
• Birds flying, cattle or sheep herding, fish
  schools
• A swarm of hundreds of birds at a time can speed
  up in one direction, suddenly, in unison,
  decrease their velocity, and turn to follow a
  different route
• No matter how complicated their acrobatic path,
  the birds always stay close together but never
  collide
• Flocking birds obey three basic rules
• Attempt to maintain a minimum distance from all
  other birds and objects on the screen
• Attempt to match speed with other birds in the
  neighbourhood
• Attempt to move to the centre of the birds in the
  neighbourhood

Flocking example 1
Flocking example 2
38
Why isnt AI perfect?

• We generally dont have realistic models of
  sensing
• There is not enough processing time to plan ahead
• Space of possible actions too large to search
  efficiently (too high a branching factor)
• Single evaluation function or predefined
  sub-goals makes them predictable
• Only have one way of doing things
• Too hard to encode all the relevant knowledge
• Too hard to get them to learn

39
Summary

• AI adds a very important aspect to any game
• It can encourage the player to think and provide
  stimulation or maintain interest
• Can be difficult to program advanced AI
• Can be time consuming to work out logic
• Finite State Machines provide an easy model to
  understand and implement although not as powerful
  as other techniques