Agent-based Composition of Behavior Models

1
Agent-based Composition of Behavior Models

• Katia Sycara (PI) Start date 10/02/02
• Gita Sukthankar
• Anupriya Ankolekar
• The Robotics InstituteCarnegie Mellon University

2
Talk Outline

• Vision
• Limitations of Current Models
• Research Objectives
• Research Approach
• Expected Impact
• Accomplishments
• Deliverables

3
Fully automated, high fidelity Computer Generated
Forces have enormous value for military
simulation and training

• High fidelity CGFs provide realistic adversaries
  and team mates
• Utilize multi-agent architectures to go beyond
  current limited behaviors to adaptive
  opponent/teammates with human-like
  unpredictability
• Can learn from experience
• Embodying Team behaviors
• Can be used for shipboard and embedded training
• Training can be conducted using standard computer
  equipment (e.g. PCs)
• Will be cost-effective and affordable
• Automated CFGs reduce the training manning
  requirements
• Agent-oriented software engineering techniques
  promote modularity and reuse

4
Limitations of Current Models

• Current CGF training models are limited and
  inflexible
• They exhibit a small hard-coded set of behaviors
• They do not allow the coach to easily customise
  the training experience
• They are hard to develop and troubleshoot
• Current human performance modeling techniques
• Have not been successfully scaled to complex
  tasks
• Have not been applied to modeling teams
• Models are expensive to construct
• Models do not allow reuse

5
Research Objectives

• Develop techniques that
• Enable CGFs to increase range of behaviors to
  incorporate smart human-like strategies and
  adaptation
• Allow efficient reuse and composition of CGF
  models
• Allow the development of models of adversaries
  and team mates that are consistent with human
  behavior modeling
• Reduce model construction time and cost

6
Research Approach

• Integration of multi-agent architectures and
  software engineering techniques to increase CGF
  sophistication and enable reuse
• Leverage our expertise in the development of
  intelligent agents to increase the autonomy,
  range of behaviors and long-term strategic level
  thinking of the CGFs
• Use knowledge bases of composable CGF plan
  fragments that encapsulate particular behaviors
• Use libraries of reusable software components and
  connectors to create executable code
• COTS game engines and state of the art animations
  provide a realistic and affordable simulation
  platform deployable for classroom, shipboard, and
  embedded training (PCs with game software)
• Demonstration Domain Urban Warfare

7
Whats unique about our approach?

• The combination of semantically rich agent
  representation and software engineering
  development methodology
• The multi-agent architectural approach enables
  modeling of team behaviors
• This approach will result in affordable,
  coachable teams of realistic training forces

8
Functional Architecture
Trainer
Reasoner
Plan Editor
Internal Events
Knowledge Structures
Reasoner
Belief Editor
CGF Model
Trainee
Simulation Environment
9
Armies Fight in Teams and so must their Training
Simulations

• Teamwork in Open Environments Sycara et al.
  incorporates heterogeneous teams and dynamic team
  formation
• Teams are not assumed to be fixed in size or team
  members abilities
• Model accommodates dynamic role assignment
  according to current situation and individual
  capability
• Model accommodates discovery and incorporation
  into the team of new appropriate team members
  (adapts to the loss of members)
• Teams can be formed/reformed dynamically during
  execution in response to incoming/changing goals
  and environment
• Negotiation of team goals and commitments
• Has been applied to Joint Mission Planning (Agent
  Storm)

10
Our approach enables reuse at multiple levels

• Individual CGFs can be adapted for different
  scenarios and domains
• Programmers reuse already developed CGF behavior
  fragments to construct new CGFs
• Our multi-agent architecture (RETSINA) is a
  proven model of software development that has
  been reused across multiple domains

11
Composition

• Composition of agents at task level
• SE language an agent is a computational process
  (an smart component). An agent can be viewed as
  a unit of planning and execution
• Thus, composition of plan fragments and
  associated code
• Manage interdependencies between plan fragments
  by matching preconditions, beliefs, commitments,
  constraints (at reactive and cognitive levels)
• Manage interdependencies between code by matching
  inputs and outputs
• Promising approach from Software Engineering
• Use a library of adapters and connectors to
  manage interdependencies and repair violated
  dependencies between composed agents

12
Appropriate representation facilitates reuse and
composition of pre-existing plans
Knowledge base of pre-developed plan fragments
CLEAR AREA x
CLEAR INTERIOR OF x
GAIN DOMINANT POSITION
CLEAR ENTRY
IF DOOR LOCKED SHOOT BOLT IF DOOR CLOSED KICK
DOOR IF WIDE ENTRY STRAFE ENTRY
Abstract plan fragments

HUG WALL

Executable actions communicated to UT and
executed by CGF

13
Appropriate representation facilitates reuse and
composition of pre-existing plans
CLEAR BUILDING
Clearing Room
CLEAR BUILDING INTERIOR
CLEAR ROOM
CLEAR ROOM INTERIOR
GAIN DOMINANT POSITION
CLEAR ENTRY
Plan fragment reuse and composition in similar
new situations
14
Appropriate representation facilitates reuse and
composition of pre-existing plans
CLEAR CAVE
CLEAR CAVE INTERIOR
GAIN DOMINANT POSITION
CLEAR ENTRY
Clearing Cave
Plan fragment reuse and composition in similar
new situations
15
Realistic and Affordable Simulation Environment
UnrealTournament (UT)
Gamebots TCP/IP Interface
Urban Scenario
UT Engine (C/C)
16
We can embed CGFs into larger tactical
simulations
UT Game Engine To provide real-time high quality
graphics and detailed local behavior
OneSAF To simulate larger military entities,
behaviors, capabilities
Correlated entities
Correlated terrain
17
SAF Manager
Show entities information
SAF entities
Show existing OTB simulation
Show network information
Show Current PDUs in OTB
Show UT entities
18
Advantages of our Approach

• Reuse
• knowledge base of plan fragments and beliefs
  supports reuse in new situations
• Modularity
• agent-based architecture provides modularity of
  CGF plans and behaviors
• Composition
• matching algorithms enable the matching of plan
  fragments and behaviors so they can be composed
  to form more intelligent adversaries and team
  mates, as situations warrant
• Verification
• our representation formalism can be used for
  formal model-checking and verification of
  desirable properties of the software, thus
  reducing development time

19
Expected Impact

• If successful, our research will provide
  Reprogrammable and Instructable CGF teams which
• Can be Coached by training instructor using a
  simple GUI to provide trainee appropriate combat
  experiences
• Exhibit realistic team behaviors
• Considerably reduce development time and cost
  while increasing behavior realism
• Can be embedded in larger simulations (e.g.
  OneSAF)

20
Accomplishments

• Developed initial Agent Representation Scheme
• Developed initial algorithm that matches current
  situation to previously developed plan fragments
  for reuse.
• Implemented initial teamwork scenario in Unreal
  Tournament.
• Publications
• Sycara, K. et al. Integrating Agents into Human
  Teams, In Salas E. (ed.) Team Cognition, Erlbaum
  Publishers, 2003. In Press.
• Sycara K. et al. Ontologies in Agent
  Architectures, In S. Staab and R. Studer (eds.)
  Handbook on Ontologies in Information Systems,
  Springer 2003. In Press.

21
Hand Signal Behaviors
Cover Area
Listen
Wait

• Hand signals are important for team communication
  in urban warfare since the enemy is often in
  close proximity.
• Extensions to Gamebots allow AI control over
  these new behaviors.

1
2
3
4
http//www.millenniumsend.com/user/pender/articles
/hands.html
22
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
23
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
24
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
25
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
26
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
27
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
28
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
29
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
30
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
31
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
32
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
33
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
34
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
35
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
36
Composition L-Shaped Corridor and Room Clearing
MCWP 3-35.3 Military Operations on Urbanized
Terrain (MOUT)
37
Composition L-shaped Corridor Stacked 2-Man
Room Clearing
38
Milestones and Deliverables

• 4/30/03- 9/30/03
• Develop initial scenarios for CGF deployment
• Develop initial agent teamwork representation
• Implement the initial scenarios in Unreal
  Tournament
• 10/01/03-12/30/03
• Evaluate the resulting CGFs for realism
• Refine teamwork representation as a result
• 1/01/04 3/30/04
• Develop techniques for agent behavior reuse
• Continue development and testing of teamwork
  schemes
• Implement them and test them in new situations

39
Milestones and Deliverables (2)

• 4/01/04- 6/30/04
• Evaluate the resulting CGFs from previous quarter
  for realism and ease of development
• Develop and test mechanisms for agent behavior
  composition
• 7/01/04 9/30/04
• Develop techniques for resolution of mismatches
  in agent descriptions
• Develop techniques for propagation of constraints
  across plans and agent beliefs
• 10/01/04 12/30/04
• Implement techniques from previous quarter in
  Unreal Tournament and test in new situations
• Develop techniques for belief propagation across
  team members

40
Milestones and Deliverables (3)

• 1/01/05- 3/30/05
• Develop indexing scheme for agent behaviors
• Develop techniques for dynamic retrieval of agent
  behaviors and reuse
• 4/01/05 6/30/05
• Implement dynamic retrieval and reuse of agent
  behaviors in new situations
• Design and implement coachs GUI
• 7/01/05 9/30/05
• Test control of CGFs from coachs GUI
• Demonstrate embedding of CGFs in OneSAF

41
Hand Signal Behaviors