Designing AI character behaviour

1
Designing AI(character behaviour)

• Game Design
• Vishnu Kotrajaras, PhD

2
Challenge the players

• Shooting overwhelm by opponents
• AI can be easy, the numbers will provide the
  challenge anyway
• First-person-shooter
• player can run out of ammo and cant see in the
  dark (enemies do not have this disadvantage)
• Also, there are lots of enemies
• Therefore, each of them does not have to be
  smart.
• Player will still be challenged at this
  difficulty

3
(No Transcript)
4
Challenge the players(2)

• RTS AI has to do everything the player does, and
  has to seem smart
• We can help it a bit by
• AI can see the entire map
• AI has a larger number of starting unit
• AI has a bigger pool, or easier way to get
  resources

5
Challenge the players(3)

• Turn-based strategy game
• Most difficult to make AI
• Chess!
• Mostly, the AI cheated a bit.

6
(No Transcript)
7
Do not do stupid things

• Do not just fall off a clifferr..thats Lemmings
• If the enemy is supposed to be human, make them
  clever
• If you cant make them clever, just make them
  look like robots!
• Different acceptable levels for different games
• Metal Gear Solid, enemies are clever but their
  eyesight are extremely short
• But if they have the same viewing range as you
  the game will be too difficult

8
(No Transcript)
9
Be unpredictable

• Players want AI to try to defeat them in ways
  players never think of
• Of course, games can surprise players with story,
  bosses, next level.
• Ai adds replayability
• Uses random number and probability (weight) for
  choosing possible actions
• This is fuzzy decision making
• Some choice of actions may seem stupid, but real
  human does this things when panicked
• The player will be the one that thinks the AI is
  clever
• Unpredictability must not break other AI goals

10
Assist Storytelling

• For example, NPC in a particular town runs away
  from strangers
• This makes the player wants to investigate
• AI of team mates helps telling the story of each
  character
• Action, speech
• Dynamic storytelling
• Players can influence the NPCs mood
• Action somewhere else in the world may affect his
  attitude towards the player

11
Create a living world

• Add life that do not directly interact with
  players
• Ambient life
• Birds
• Cats
• Dogs

12
Should the AI not have any advantage?

• AI may not be challenging anymore, because
  naturally, man is more clever
• Programmers will spend too much time on AI
• Remember, the goal is not the use of AI
  techniques, but the challenge the player can get
• Players love to beat tons of enemies, it feels
  great. Therefore few clever enemies may not be
  good enough

13
Too real?

• AI that runs away when wounded
• If the AI is faster, or navigate better, the
  player will never outrun it
• Now it is a chase game, forever chase
• AI that always cover weak points when players
  shoot
• No way to win!

14
AI and levels

• Must be developed together
• AI that works on every level is hard
• Problem is, you may have 2 teams, one working on
  AI and the other on levels
• Must work together
• Create simple AI that works in some situation
• Then level designer create a level based on this
  simple AI
• Keep looking at each others work and
  communicate.
• Both must adjust their work
• Levels may have to sacrifice some coolness in
  order for AI to work well
• AI may not be able to swim, so the level must not
  have river
• Must prevent NPC getting stuck at corner

15
Good enough?

• Teammates AI
• Dont fail the player
• Especially, dont fail when the player need it
  the most

16
scripting

• Use dynamic AI, together with pre-made script
• Half-life
• Use scripted path (rule-based behaviour)
• The AI do things depending on situation
  (reacting)
• Tester is the one who work out what the AI should
  do

17
(No Transcript)
18
(No Transcript)
19
Dynamic scripting

• Maintain several rule bases, one for each
  opponent type.
• Each time a new opponent is encountered, rules
  are extracted from the rule base of that creature
  type.
• The probability that one rule is selected depends
  on a weight value that is associated with each
  rule.
• Weights of all rules are updated after a
  completion of an encounter.
• the order which the rules are placed depends on
  application.

20
Advantage of dynamic scripting

• Fast
• Extraction of prepared rules from rule base.
• Update weight only once per encounter.
• Still intelligent
• Every rule is from the prepared rule bases.
• Robust
• Rules are not removed immediately when punished.

21
Example of dynamic scripting

• Opposing party encounter (AI and player)
• Each party has 2 fighters and 2 wizards of the
  same level.
• Weapons are static.
• Can select 2 types of potion.
• Wizards know 7 spells.

22
The dynamic scripting language

• (Optional conditional statement) and an action.
• Conditional statement
• One or more conditions.
• Combined with logical AND or OR.
• Condition examples
• Distance separating characters.
• Characters health.
• Spell effects currently on characters.
• Actions (5 actions in this experiment)
• Attack an enemy.
• Drink potion.
• Casting spell.
• Moving.
• Passing the turn.

23
The dynamic scripting language(2)

• Things can be specific
• Cast spell magic missile at closest enemy
  wizard.
• Or general
• Cast any offensive spell at random enemy.
• Or somewhere in between
• Cats the strongest damaging spell available at
  the weakest enemy.

24

• Rules are executed in sequential order (condition
  of that rule must also satisfy.).
• If at the end of all chosen rules, no action is
  selected, a pass turn is chosen as a default
  action.
• Chosen rules are selected according to their
  corresponding weight.
• Chosen rules are ordered according to their
  priority. If they have equal priority, then they
  are ordered by their weight.

25
Total rules in Baldurs gate experiment

• Fighter
• 20 rules in total.
• The program choose 5 rules to form a script.
• Wizards
• 50 rules in total.
• 10 are chosen to form our sequence rule script.

26
The weight update function

• Party fitness
• Range 0..1
• 0 if party has lost the fight.
• 0.5 half the average remaining health of all
  party members.

27
Health of n at the end
Health of n at the start of the fight
28

• Character fitness
• Range 0..1
• Based on
• Avg. remaining health of the party.
• Avg. damage done to the opposing party.
• Remaining health of the character (if character
  dies, we use the time of death instead).
• Party fitness.

29
Time of death
Large reward to party victory. Smaller reward to
the individuals survival. Even smaller reward
for friends survival and the damage they did to
the enemy.
30
The actual weight update function

• Translated from character fitness.
• Only the rules executed were rewarded or
  penalized.

Max. penalty
Break even point
Max. reward
Max. weight value
31
In the trials

• MP30
• MR100
• MW 2000
• b0
• All weights are originally 100

32

• Static scripts to fight against
• Must get changed (to reflect players change in
  strategy).
• Offensive
• Fighter attacks nearest enemy.
• Wizards use most powerful spell to attack weakest
  enemy.
• Disabling
• Fighter first drink potion that prevent status
  effects, then attack the nearest enemy.
• Wizards use all spells that can disable enemies
  for a few rounds.
• Cursing
• Fighter attacks nearest enemy.
• Wizards try all spells that harm enemy.
• Defensive
• Fighter drinks potion that reduce damage first,
  then attacks nearest enemy.
• Wizards use all defensive spells.

33
Composite tactics

• Random party tactics
• Select 1 from 4 tactics at each encounter.
• Random character tactics
• Consecutive party tactics
• Continue using the same party strategy if still
  win.
• Most resemble human tactic.

34
Test run

• Basic tactics
• 21 tests, each test we run until we find
• Average turning point (the first time that
  dynamic scripting win for 10 consecutive
  encounters).
• Absolute turning point (first encounter which,
  following this encounter, consecutive loses of
  dynamic script is never more than consecutive
  win).

35
Avg is much higher than the median due to rare
occurrences of high turning points (early
divergence makes unlearn hard).
Easy to defeat
Hardest to defeat, but still learn to defeat it.
36

• http//www.cs.unimaas.nl/p.spronck/

37
(No Transcript)
38
Current NPC

• Do things only in the area that a player is
  active.
• Static agent
• wait for player to interact with it.
• Or play through a scripted sequence.
• Proactive Persistent agents (proposed in the
  paper)
• Have needs, beliefs, desires of their own.
• Can initiate action on their own.
• They are modeled regardless of a players
  location in the game.

39
PPA requirements

• Believable
• Configurable
• Scalable
• Only 1 or 2 is enough. The rest can be old
  static.
• Able to switch mode

40
Current agent architectures

• Reactive (like if-then-else), mostly implemented
  by FSM.
• advantages
• Very fast.
• Simple to program and understand.
• Completely under control.
• Require little support from infrastructure.
• Disadvantages
• Must try to code every possible situation that it
  will encounter.
• Stupid.

41

• Deliberative
• Build model of he world.
• Form plans to achieve goals.
• Prolog-like
• But
• Maybe very slow to decide things.
• Require constant maintenance of a knowledge base.
• Cannot be used in fast changing game.

42

• Hybrid agent
• Use reactive for time-critical behavior, such as
  avoiding collision.
• Long term planning can use deliberative system.

43
PPA requirement in detail

• Autonomy
• Social ability (human-like)
• Reactivity
• Pro-activeness
• Persistence
• Scalability
• Extendability
• Believability
• configuability

44
Proposed architecture

• System composed of
• Behaviour system
• Social system
• Goal based planner
• Schedule

45
What agent experiences in the world
Ordinary behaviour
Human-like communication relationship between
characters
A task for an agent at particular time
46
Sweetsers PhD. Thesis

• Intelligent AI adaptable to game environment in
  a strategy game.
• Each cell on the map has low level properties
  (Physics)
• Heat capacity, ignition point and burning rate.
• Wetness and fluid flow.
• Pressure flow.
• Explosion on a map will now cause natural fire.
  River realistically blocks out the fire.

47

• Object has
• Low level properties just like a cell. (Object is
  treated as a cell, but has only 1 adjacent cell,
  that is, the cell it is in.)
• High level properties
• Structures- is_open, has_volume
• So an object can contain water, pressure, etc.
  realistically.
• Agent
• Has low and high level properties like an object,
  but can move.
• Calculate comfort value of its current cell and
  its surrounding cell.
• lt0.1 -Comfortable
• 0.1-0.3 -Slowly move to a more comfortable cell.
• 0.3-0.6 distress and run from the cell.
• gt0.6 panic and run

48
Neighborhood size 1
Neighborhood size 2
49
Agent experiments

• Exp1 neighborhood size
• 1 can avoid immediate danger, but get stuck in a
  larger hazard. Or it runs into another danger.
• 3 picking good destination. More agents move to
  the same destination.
• But run through hazards.

50

• Exp2 optimising agent navigation
• First select a goal, using neighborhood size 3.
• Then evaluate cells in its immediate
  neighborhood, using comfort value and distance
  from the goal.
• Comfort 50, distance 50- appear intelligent
• Comfort 25, distance 75- still run through
  hazard
• Comfort 75, distance 25- appear stupid, moving
  randomly

51

• Exp3 combing comfort and desire
• Propagate the desire out from the goal position,
  with a factor of 0.7

4.9
7
7
7
10
4.9
7
7
4.9
7
7
7
4.9
52

• Next, an agent calculate comfort values for
  neighborhood size 3.
• Agent select the best cell to move to, based on
  desirability and comfort.
• 50 desire, 50 comfort
• 75 desire, 25 comfort
• 25 desire, 75 comfort
• Agent then chooses a cell to move to in its
  immediate neighborhood (comfort 50, closeness
  50)
• Result
• 50 desire, 50 comfort
• best, but only about half the agents found the
  goal, as they opted for comfort over goal.
• 75 desire, 25 comfort
• Appeared less intelligent, moving randomly. Only
  about half the agents found the goals, why????
• 25 desire, 75 comfort
• More cycles to converge. No significant
  difference between the number of agents that
  found the goal, compared to previous results.
• Increasing propagation constant to 0.8 helps
  improve agents.
• Agents can wait for hazards to pass before moving
  further.
• But the agents were still not successfully able
  to find the goal.
• Why????????

53
Answer by Sweetser

• It's not that the agents don't "survive", but
  rather that they stop in a cell and do not reach
  the goal.
• This means that they cannot see a better cell
  near them to move towards and thus stay in their
  current location.
• The reason why more agents don't reach the goal
  is due to the fact that they are considering both
  the dangers of the environment and trying to
  reach the goal.
• There are also lots of dangers for the agents to
  face on the map.
• So more agents would reach the goal with a higher
  desire weighting or if the map were less
  dangerous.
• I think it would be interesting to include some
  real-time adaptation for the agents and certainly
  basing their decision on their attributes would
  makesense.

54

• Last exp multiple goals
• Agents were much better in finding goals.
• But maybe because there are more goals within the
  same physical area.
• Or the desirability of many goals accumulate,
  allowing influence to propagate further.
• But did not appear as intelligent or realistic.
• What has not been done from this thesis.
• Agent can run away, but it does not use
  environment any more than that, such as burning
  forests or using water to put out fire. (There
  are no moving objects in Sweetsers thesis.)
• Maybe adding objects that can be moved between
  cells (by agents) is another step.
• Agent sees only cell, not objects in the cell.
• It will be better if we can have agents
  neutralizing hazards.