Autonomous AI

1
Autonomous AI

• Steve Hookway
• 9/13/04

2
Outline

• Constraining Autonomous Behavior (3.1)
• Simple Techniques for Coordinated Behavior (3.2)
• Synthetic Adversaries for Urban Combat Training
  (Laird et al.)

3
No One Lives Forever 2

• UNITY agent Cate Archer fights the H.A.R.M.
  Worldwide crime syndicate

• Use a combination of stealth and brute force to
  work through each level

4
No One Lives Forever 2

• Excellent reviews for both game-play and AI
• Games Domain (5/5) "NOLF 2 is extremely hard to
  fault. Its presentation is top-notch, its levels
  are varied and interesting, its AI is convincing,
  and the whole game drips with love and attention
  to detail. In fact, it's hard to remember a
  single-player FPS we enjoyed this much since Deus
  Ex ... Any first-person shooter fan should
  consider this an essential purchase."
• HomeLan Fed (94) "one of the best single player
  first person shooter games ever made. With solid
  AI, great looking levels and graphics, fun
  weapons and gadgets, excellent voice acting, and
  a healthy dose of humor, A Spy In H.A.R.Ms way
  is much better than the original and thats
  saying something"

5
Ideal vs Actual Behavior

• Ideally completely autonomous AI
• Goal based system
• Tagged objects
• Realistically human behaviors need to be added
• Total autonomy does not necessarily lead to
  believable behavior

6
Human Characteristics

• Ownership
• Dependency
• Responsibility
• Relevance
• Priority
• State of Consciousness
• Expected State
• Presence of Others

7
Ownership

• Without constraint all objects of interest are up
  for grabs
• Implement ownership by grouping objects into sets
• Each character has his own set of objects
• Is this Object mine to use?

8
Dependency

• The use of one object depends on the use of
  another object
• Washing hands after using bathroom
• Cant just make one inseparable behavior
• Enable, disable objects for dependency
• Is there anything I need to do before using this
  object?

9
Responsibility

• Responsibility as a function of location
• Keeps player from being overwhelmed by enemies
• Responsibility as a function of the class of
  character
• Set which objects are available to be used by
  that class of character
• What is my purpose in life?

10
Relevance

• A minor characters goals should not have a major
  influence on the world!
• Provide a ceiling to the amount of influence an
  AI character can have over their surroundings
• How will my actions affect the players
  experience?

11
Priority

• Character presented with multiple opportunities
  simultaneously
• 3 categories of behavior
• Aggressive gt investigative gt relaxed
• Relative priorities within categories
• What is most important right now?

12
State of Consciousness

• Alive, Sleeping, Unconscious, Dead
• Players emit stimuli depending on state
• Can act according to previous or current state
• Am I awake?

13
Expected State

• Consciousness is an example of general concept
  State
• Objects have associated state
• Notice when an object is out of place
• Notice when a player or objects state has
  changed
• Has anything changed?

14
Presence of Others

• Behavior depends on who is present
• Characters need to take each other into account
• How will this action affect other characters?

15
Outline

• Constraining Autonomous Behavior (3.1)
• Simple Techniques for Coordinated Behavior (3.2)
• Synthetic Adversaries for Urban Combat Training
  (Laird et al.)

16
Coordinate Behavior

• Split Up
• Get out of my way
• Get down
• Whats going on

17
Split Up

• Agents are driven by same logic and therefore
  behave in the same way upon receiving a stimulus
• Same shortest path algorithm
• Nearby agents follow same path
• Crowding!

18
Split up cont.

• Path of waypoints to target
• Reservations
• Shortest path as least costly path
• A node is not released until agent completes path
• Reserve the node BEFORE the destination
• Different paths give illusion of coordination

19
Pathfinding
Reserved
20
Maximum Occupancy

• The pathfinding algorithm works well in combat
• Firing range prevents crowding
• When investigating, crowding becomes a problem
• First agent should reserve object under
  investigation
• Other agents maintain distance

21
Blackboard Communication

• Post and query public information
• Avoids overhead in adding data to actual
  pathfinding data structures themselves
• Makes code more maintainable

22
A Record on the Blackboard

• Struct BlackboardRecord
• BBRecordType eType
• ObjectID idPoster
• ObjectID idTarget
• Int nData

23
The Blackboard

• Class Cblackboard
• public
• void PostBBRecord(BBRecordType eType, ObjectID
  idPoster, ObjectID idTarget, int nData)
• void RemoveBBRecord(BBRecordType eType, ObjectID
  idPoster)
• void RemoveAllBBRecords(BBRecordType eType)
• int CountBBRecords(BBRecordType eType, ObjectID
  idTarget)
• BlackboardRecord GetBBRecord(BBRecordType
  eType, ObjectID idTarget)
• protected
• BlackboardRecord m_aBBRecordsMAX_BB

24
Agent path planning

• Clear any previously reserved volumes and reserve
  volume before destination
• g_pBlackboard -gt
• RemoveBBRecord(kBB_ReservedVolume, myID)
• g_pBlackboard -gt PostBBRecord(kBB_ReservedVolume,
  myID, volumeID,NULL)

25
Agent path planning cont.

• Check if volume is reserved and apply cost
  penalty if it is
• float fCost pVolume -gt fCost
• if(g_pBlackboard-gtGetBBRecord(kBB_ReservedVolume,
  volumeID)
• fCost 50.f

26
Get Out of My Way

• It is inevitable that agents will get in each
  others way
• Too much overhead to recalculate optimal position
• Better to deal gracefully with obstruction

27
Get out of My Way cont.

• The obstructed agent can do nothing
• The obstructed agent can fire anyway, either
  killing or going through his ally
• The obstructed agent can move
• The obstructed agents ally can move

28
Implementation

• Obviously options 3 and 4 are the way to go
• Requests ally moves
• If ally is busy (reloading) agent moves
• Leads to a dynamic environment with everyone in
  motion looking for a clear shot

29
Implementation cont.

• Previous method works well with ranged weapons
  but not well with close-ranged melee weapons
• Resort to KungFu style

30
Get Down!

• Agents have a limited number of actions
• We dont want all agents take same action
  simultaneously
• Use blackboard to post actions
• Agent posts action and time
• Before taking action agent checks to see if ally
  has done same action within a threshold time

31
Whats Going On?

• We dont want agents to stand idol while a nearby
  ally is taking damage
• The same sensory system that detects enemies can
  be used to detect each other
• When agent detects danger or takes damage, it
  emits a signal to any other agent in earshot

32
Coordination

• We have achieved simple coordination without the
  addition of a group behavior layer!

33
Outline

• Constraining Autonomous Behavior (3.1)
• Simple Techniques for Coordinated Behavior (3.2)
• Synthetic Adversaries for Urban Combat Training
  (Laird et al.)

34
Synthetic Adversaries

• Virtual training of soldiers for Military
  Operations on Urbanized Terrain (MOUT)
• System based on a bot developed to play Quake
• Uses Infiltrations (UTs) environment

35
Requirements for Synthetic Adversaries

• Competence
• Taskability
• Observational fidelity
• Behavior variability
• Transparency
• Minimal computational footprint

36
Competence

• Must perform the actions and behaviors humans
  perform in this domain
• Many different roles
• Defending a room
• Sentry
• Acting as a leader
• Focus on a small team interacting

37
Competence

• Threat data structure
• Used to maintain awareness about enemy
• Building map data structure
• Mission object
• Operators are used to take action in the world

38
Taskability

• Must be able to be assigned new missions and
  change objectives during an exercise
• All agent knowledge is independent of the
  specific mission
• MOUTBots can issue a limited amount of commands
  to change the mission of other bots

39
Observational Fidelity

• Model human behavior only in those aspects which
  can be observed
• This is one place we can limit computational time
• Subject to making the some human errors

40
Behavior Variability

• Must not be predictable
• Allows for soldiers to train in the same
  situation but experience a new encounter
• Limits the amount of surprises in real life
  combat
• Assign probability to each potential action

41
Transparency

• Allow after action review by keeping a record
  explaining actions
• Used an application specific visualization tool
  to show what the agent believes the status of the
  map to be
• Can look at an agents goal defend room goal
  among agents active goals

42
Minimal Computational Footprint

• Targeted for personal computers
• Majority of processing reserved for graphics and
  physics modeling
• Have been able to run as many as 20 bots in UT
  while keeping the games frame rate

43
Conclusion

• Fully autonomous AI is a long way off, but it is
  still possible to model human behavior

44
BONUS MATERIAL
45
Creature Smarts

• Is it possible to build an artificial
  intelligence thats as clever, as adaptive and as
  captivating as the average dog?
• C4 and SheepDog
• Duncan the Terrier

46
Autonomous AI

• Build a creature that can interact with each
  other and human participants
• Self sufficient reasoning
• Learning
• Action Selection
• Goals

47
System Architecture
Networked Devices
Local Devices
Creature 1 Mental Rep
World Model
Input Server
Creature 2 Mental Rep
Renderers
World State
48
Cognitive Architecture
The World
Sensory System
Proprioceptive System
Perception System
Working Memory
Action System
Attention Selection
Action Selection
Navigation System
Motor System
49
Sensory System

• Sensory honesty
• Input from world (creatures point of view)

50
Perception System

• Percept-perceive stimulus
• Classification hierarchy

51
Percept and Memory

• After classifying, hold onto percepts and place
  like percepts together
• Allows related events to be matched

52
Why bother with percepts?

• Causes honest mistakes
• Believable behavior
• Learning generalizable concepts
• Find rules that apply to many objects
• Subjective experience
• How stimulus is perceived depends on creature

53
Action System

• Any representation of action should answer the
  big 4
• When to do it?
• What to do and how to do it?
• What to do it to?
• For how long?

54
Action Tuples

• Trigger Context
• Action
• Object Context
• Do Until Context

Away Sound Percept
Circle Counterclockwise
Sheep Shape Percept
15s, or circled flock
55
Action Group

• Formed of similar action tuples
• Attention Group
• Chooses creatures focus
• Primary Action Group
• Determines large-scale body motion

56
Learning in c4s Action System

• Events perceived to have intrinsic values on a
  nominal scale
• Eating a cookie has a high positive value
• Meeting a lion a large negative number
• The actions that lead to these events become
  weighted (credit assignment)
• Learning reliable action pairs
• Learning cannot happen without variation of
  action and state

57
Navigation System

• Navigation overrides action system
• Must move to achieve action
• Relieves action system from handling movement
• Eat vs Approach and eat

58
Motor System

• No such thing as just a walk cycle
• Happy walk
• Sad walk
• And so on
• Simple gestures
• Locomotion
• Eye/head/body orientation

59
Doing it in Style

• Parameterized motor actions
• Different levels (heights of paw for shake)
• Support new motor actions
• Characters increase in complexity over time
• Create new animations live
• Allow characters to learn new animations during
  their lives

60
Pose Graph

• Action between poses
• Movement between poses

61
Conclusions

• Strict enforcement of sensory honesty
• Creature must make decisions based on percept of
  object, rather than knowledge of object state
• Creatures must perceive each other and interact
• Leads to a more realistic system of learning and
  honest mistakes