Realism in Computer Generated Soldiers

1
Realism in Computer Generated Soldiers

• Dr. Douglas Reece
• Soar Technology, Inc.

2
Outline

• Context
• Virtual simulation for infantry
• Distributed simulation
• Realism Factors
• Situation awareness
• Sound
• Vision
• Short term memory
• Persistent mental models
• Long term memory
• Inferences
• Terrain awareness
• Collisions terrain

3
Context

• Virtual simulation for military training,
  research
• Distributed Interactive Simulation
• Actions, animations limited for human and
  computer controlled avatars
• User defined scenarios
• Any mission
• Interoperate with any forces
• Free play
• (Now large databases (10s of km, 10ks of
  buildings))
• Individual soldiers in teams, squads
• High intensity warfare
• Computer controlled threats
• and friendlies
• No conversation with threats
• You dont see too much of the threats

4
Believability

• Many aspects of characters are iconic or absent
• Visual rendering
• Voice communication
• Gestures
• Want behavior that supports belief that an
  intelligent threat is playing the entity
• Avoid obviously stupid behavior that destroys
  suspension of disbelief
• Some of this is more/less subjective

5
Team Target Engagement Simulator and CCH

• USMC project, 1994 96

AUTOMATED COMPUTER CONTROLLED HOSTILES (CCH)
6
Distributed Warrior Network and DISAF

• US Army STRICOM project, 1996 99

VIC Bravo
VIC Alpha
VIC Charlie
Simulation Network
VIC Foxtrot (TTES)
Dismounted Infantry Semi Automated Forces (DISAF)
VIC Virtual Individual Combatant
7
Game Observations

• Mostly tactical shooters, including Full
  Spectrum Warrior

8
Situation Awareness

• A bot looks unrealistically stupid when
• it isnt aware of something it obviously should
  be, or
• it forgets something critical

9
Sound Detection

• Detect events that should be detected
• Gunfire
• Explosions
• Vehicles running
• Footsteps
• Important phenomenato model
• Sound level at listener
• f(Source level, attenuation)
• Direction information?
• Masking

10
Effects of Sound Detection

• Hearing any sound from an entity detects entity
• Detection of gunfire and vehicle sound may
  identify force (even if not visible)
• Detection provides some location update (even if
  not visible)

11
Vision Detection

• Detection behavior to model
• Visual searchlooking around
• Detection with peripheral vision
• Looking at bogey
• Lighting contrast limitations to model
• Entering dark room from bright light
• Target in shadow surrounded by light
• Blinding from weapon and munition flash
  (especially at night)

12
Short Term Memory

• Remember what just happened
• I just detected a threat now Im turned away...
• I just detected threat now it moved out of sight

13
Old Simulation Approach

• Remember threats that leave line of sight for
  several seconds
• But dont allow shot unless clear line of fire
• OK for threat that passes quickly behind obstacle
• Simple to implement, but
• Too committed for threats that dont reappear
• Forgets completely after timeout

14
Better Memory Model

• Explicit state for unseen threats

Location Known
Location Unknown
Visible
?
?
15
More Extensive Mental Models

• Instead of just detection state, extend to bigger
  mental model
• E.g. frame representing situation, with
  observations filled in
• Can use inferences, e.g. Bayes net, to fill in
  other parts of frame
• Dont just remember inferences or certainty
  levels remember observations

16
Long Term Memory

• What does an event mean? How does it affect
  behavior?
• Depends on longer term situation
• E.g. single gunshot from threat...what should the
  change in behavior be?
• Civilian, non-battlefield context...
• Battlefield, in friendly secured area...
• Battlefield, no shots in past hour...
• Battlefield, no shots in past 5 minutes...
• Battlefield, ongoing firefight...
• Example Guard detects intruder threat
• Unusual event
• Guard stays alerted for long time
• Guard force performs searches, alerts backup,
  etc. until threat found

17
Inferences

• Agents should make obvious inferences
• My buddy got shot gt theres an enemy
• My buddy is aiming his weapon gt enemy in that
  direction
• My buddy is looking in this direction gt may be a
  threat there to look at

18
Terrain Awareness

• Real soldiers use terrain as if their lives
  depended on it (it does)
• Body posture
• Selection of positions
• Movement
• They dont expose themselves to shoot
• Examples follow from USMC urban combat field
  manual

19
Terrain Use Examples
20
Terrain Use Examples
21
Terrain Use Examples
22
Terrain Use Examples
23
Terrain Use Examples
24
Terrain Use Examples
25
Terrain Use Examples
26
Counterexample
27
Challenges in Using Terrain

• Easier to simulate behavior mechanics without
  using terrain
• E.g. fire and maneuvermove in fixed geometry
• May take majority of effort of behavior
• Computationally expensive
• Requires simulating details
• Posture, body limb kinematics
• Intervisibility to body parts
• Fine motion control in world to position eyes,
  weapon, body

28
Example Terrain Analysis for Movement

• Movement using cover
• Controls posturestanding, prone
• Controls speedruns when exposed
• Uses A search over 1 meter grid
• Exposure cost is non-linear
• Quick exposures OK
• Longer exposures expensive

29
Movement Using Cover and Concealment
30
Terrain Collision/Penetration

• DWN user evaluation unrealistic because of body
  parts penetrating walls
• Weapons
• Avatar feet, when soldier dies or goes prone
• Can fix with sophisticated articulated model, rag
  doll physics and kinematic constraints
• But obstacles affect behavior
• Lift weapon instead of dragging it on wall
• Turn away from wall instead of falling into it

31
Body Rotation to Avoid Falling Into Wall

• Soldier wants to go prone
• Behavior wants entity to fall prone
• Test for collision first
• If would collide, body refuses command from
  behavior
• Soldier entity killed (must fall)
• Test for collision
• If would collide, try rotating parallel to wall
• If still collides (corner), try moving away from
  wall

32
Conclusions

• Situation awareness, including history, is key
  for realistic characters in military domain
• Physiological models (vision, hearing)
• Physical models (terrain collision)