Effective Coordination of Multiple Intelligent Agents for Command and Control

1
Effective Coordination of Multiple Intelligent
Agents for Command and Control

• The Robotics Institute
• Carnegie Mellon University
• PI Katia Sycara
• http//www.cs.cmu.edu/sycara
• http//www.cs.cmu.edu/softagents

Key Personnel Onn Shehory
Terry Payne
2
Current Situation

• Vast amounts of data from distributed and
  heterogeneous sources
• Uncertain and evolving tactical situation
• Shrinking decision cycles
• Decision makers distributed in space and time

3
Overall Goal

• To develop effective agent-based system
  technology to support command and control
  decision making in time stressed and uncertain
  situations

4
What is an Agent?

• A computational system that
• has goals, sensors and effectors
• is autonomous
• is adaptive
• is long lived
• lives in a networked infrastructure
• interacts with other agents

5
Retsina Agent Architecture
6
Retsina Functional Organization
7
Middle Agent Types
Capabilities Initially Known By
8
Research Objectives

• Develop an adaptive, self-organizing collection
  of intelligent agents that interact with the
  humans and each other to
• integrate information management and decision
  support
• anticipate and satisfy human information
  processing and problem solving needs
• perform real-time synchronization of domain
  activities
• notify users and other each other about
  significant changes in the environment
• adapt to user, task and situation

9
Technical Challenges

• What coordination mechanisms are effective for
  large numbers of sophisticated agents?
• What are the scaling up properties of these
  coordination mechanisms?
• How do they perform with respect to dimensions,
  such as task complexity, interdependence, agent
  heterogeneity, solution quality?
• What guarantees do these mechanisms provide
  regarding predictability of overall system
  behavior?
• Do they mitigate against harmful system
  behaviors?
• How to achieve effective human-agent coordination?

10
Potential Impacts

• Reduce time for commanders to arrive at a
  decision
• Allow commanders to consider a broader range of
  alternatives
• Enable commanders to flexibly manage
  contingencies (replan, repair)
• Improve battle field awareness
• Enable in-context information filtering

11
Innovative Claims

• Scalable, robust and adaptive coordination and
  control multi-agent strategies
• Sophisticated individual agent control
• Reusable and customizable agent components
• Multi-agent infrastructure coordination tools and
  environment

12
Major Project Deliverables

• Prototype multiagent system that aids human
  military planners to perform effective in
  context information gathering, execution
  monitoring, and problem solving
• reusable agent shell that includes domain
  independent components for representing and
  controlling agent functionality, so that agents
  can be easily produced for different types of
  tasks
• effective multiagent coordination protocols, that
  are scalable, efficient and adaptive to user task
  and planning context
• multi agent coordination infrastructure
  consisting of a suite of tools for reliable and
  low cost building and experimenting with flexible
  multiagent systems

13
The RETSINA Multi-Agent Architecture
distributed adaptive collections of information
agents that coordinate to retrieve, filter and
fuse information relevant to the user, task and
situation, as well as anticipate user's
information needs.
14
RETSINA Individual Agent Architecture
15
Capability-Based Coordination

• Open, uncertain environment
• Agents leave and join unpredictably
• Agents have heterogeneous capabilities
• Replication increases robustness
• Agent location via Middle agents
• Matchmakers match advertised capabilities
• Blackboard agents collect requests
• Broker agents process both

16
Capability-Based Coordination (cont)

• Advertisement
• Includes agent capability, cost, etc.
• Supports interoperability
• Agent interface to the agent society independent
  of agent internal structure
• We will test scale-up properties of
  capability-based coordination

17
Cooperation

• Problems with current methods
• Mechanisms not tested in real-world MAS
• Simulations?size small (20 agents)
• Complex mechanism do not scale up
• We will provide algorithms for efficient group
  formation

18
Cooperation - Solutions (continued)

• Approach
• Very large systems (millions of agents)
• Constant complexity cooperation method
• Based on models of multi-particle interaction
• Structural organization
• Relation of organization structure and autonomy
• Effect on system flexibility, robustness

19
Cooperation - Solutions (continued)

• Communication planning
• Change communication patterns to reduce
  eavesdropping risk
• Bundle small message together
• Use networks when less congested

20
Competition and Markets

• Limited resources result in competition
• Market-based approaches
• Assume that agents can find one another
• Assume centralized auctioneer
• Otherwise, convergence results do not hold
• Approach
• Utilize financial option pricing
• Prioritize tasks by dynamic valuation
• Allows flexible contingent contracting
• Analysis of large MAS via economics methods

21
Competition and Markets (contd)

• Combine our capability-based coordination with
  market mechanisms
• Mechanism design
• Design enforceable mechanisms for self-interested
  agents
• Resolve Tragedy of Commons by pricing schemes.
• Devise mechanisms to motivate truthful behavior

22
RETSINA Testbed for Agent-Based Systems

• Continuing development of general purpose
  multi-agent infrastructure
• Agents built from domain-independent, reusable
  components
• Agent behaviors specified in declarative manner
• New agent configurations easily built and
  empirically tested.

23
Coordinating Agents With Human Users

• Problem Commanders already overloaded
• For task delegation to be effective,
  communication with agents should be
• natural
• flexible providing planning information when
  appropriate
• concise providing as little detail as possible
• interactive
• before and during task execution, agents
• provide explanations of plans
• assist users in revising plans
• during task execution, agents
• report plans progress

24
Agent Task Delegation

• Languages for task description and delegation
• Reconciling human and agent representation of
  tasks
• Structured Natural Language/Graphical task
  description
• Interactive Planning and Execution
• user input as constraints on plan formation
• execution monitor brings user into loop
• Extending RETSINAs
• graphical task editor
• planner and execution monitor

25
In-Context Information Management for C2

• Agent-Based Information Management
• Dependent on
• user preferences
• decision-making tasks
• evolving situation
• Agents responsibilities
• Represent users task environment
• Monitor significant changes
• Provide appropriate notification to user or
  responsible agent
• Learn to track and anticipate users information
  needs
• Learn appropriate times and methods for
  presenting information

26
Agent Coordination in RETSINA

• Build information management agents for C2 based
  on RETSINA mechanisms for agent coordination
• Goal and task structures provide user and agent
  context
• Information agents form and execute plans that
• involve queries for future information monitoring
• take situational constraints into account
• work around notification deadlines
• Build upon existing base of information
  management agents

27
Research Plan

• Agent Control
• mapping of task model and requirements to the
  appropriate coordination strategy
• mapping of constraints of the environment, other
  agents and available resources to appropriate
  coordination strategy
• experimental evaluation, analysis and refinement
• Agent Coordination
• design/refine coordination algorithm
• implement appropriate experimental infrastructure
• implement the coordination strategy and evaluate
  along different dimensions
• analyze the results and refine algorithm design
  and experimental process

28
Research Plan (contd.)

• User-Agent Coordination
• enhance the functionality of the current agent
  command language
• develop and implement techniques for acquisition
  and maintenance of user tasks preferences and
  intentions
• develop and implement protocols to enable an
  agent to accept task-related queries before,
  during or after task execution and generate
  natural descriptions of the unfolding execution
  of its plans
• evaluate and refine
• Information Management and Decision Support
• develop mechanisms for information management
  (e.g., filtering, integration) in the context of
  the current problem solving task
• develop mechanisms for in-context information
  monitoring and notification
• evaluate and refine

29
Major Project Deliverables

• Prototype multiagent system that aids human
  military planners to perform effective in
  context information gathering, execution
  monitoring, and problem solving
• Reusable agent shell that includes domain
  independent components for representing and
  controlling agent functionality, so that agents
  can be easily produced for different types of
  tasks
• Effective multiagent coordination protocols, that
  are scalable, efficient and adaptive to user task
  and planning context
• Multi agent coordination infrastructure
  consisting of a suite of tools for reliable and
  low cost building and experimenting with flexible
  multiagent systems

30
Dimensions of Evaluation

• Individual Agent
• eg
• reasoning sophistication
• control sophistication
• learning capability
• degree of self-interestedness
• knowledge
• data available to the agent
• Task
• eg
• task complexity
• task interdependence
• task temporal
• resource constraints
• frequency of task arrival

31
Dimensions of Evaluation (cont)

• Environment
• eg
• number of agents
• system load
• degree of uncertainty
• resource availability
• Coordination Mechanism
• eg
• degree of agent coupling
• richness of communication
• task delegation mechanism
• degree of agent cooperation/competition
• Organizational Structure
• eg
• hierarchy
• heterarchy
• federation

32
Evaluation Metrics

• Individual Agent Performance
• eg
• accuracy of information returned by an agent
• agent service responsiveness
• resource consumption
• MAS Aggregate Performance
• eg
• System efficiency
• Solution quality
• System robustness
• System stability
• Predictability
• Scalability

33
Examples of Experimental Hypothesis
34
Process for Experimentation

• 1. Formulation of the distributed coordination
  algorithm
• 2. Development of experimental infrastructure
  (eg simulation tools, making appropriate
  modifications ro RETSINA components)
• 3. Running the experiment and collecting
  statistics
• 4. Analysis of the results
• 5. Inter-mechanism evaluation the results of the
  simulations of the various mechanisms will be
  compared to determine performance landscapes of
  the different coordination mechanisms

35
Inter-Agent World Communications

• 1. The OAA Facilitator is started, followed by
  OAA Startit and OAA Monitor.
• 2. Start the InterOperator.
• a. We verify its registration as a Retsina agent
  with the Retsina ANS entry, "OAA_InterOperator".
• b. We verify its registration as an OAA agent
  via its registry and advertisement with the OAA
  Facilitator and by its name, "Retsina_InterOperato
  r", and icon showing in the OAA Monitor.
• 3. Start the Retsina agent, "KQMLMessageSenderGUI"
  and register it with theRetsina ANS under the
  name, "Retsina_Matchmaker".
• 4. Using OAA Startit, start the other OAA agents.
  As those agents come onlinethey will register
  and advertise with the OAA Facilitator. Each
  registry and advertisement will generate an event
  which is captured by the InterOperator and
  forwarded to the "Retsina_Matchmaker". In the
  future, the real Retsina Matchmaker will be the
  actual recipient of those messages.
• 5. Via the "Retsina_Matchmaker", it is possible
  to send messages to the OAA Facilitator.
• 6. OAA agents may be disconnected from the OAA
  Facilitator, or shutdown, and their status change
  will also be transmitted to the
  "Retsina_Matchmaker".

36
Inter-Agent World Communications

• 1. The "OAA Facilitator" is started, followed by
  "OAA Startit"(cf. ltLVgt/OAA_Start-It.gif) and
  "OAA Monitor" (cf. ltLVgt/OAA_Monitor.gif).
• 2. Start the InterOperator.
• a. We verify its registration as a Retsina agent
  with the Retsina ANS entry,"OAA_InterOperator"
  (cf. ltLVgt/TestANS_lookup.gif).
• b. We verify its registration as an OAA agent
  via its registry andadvertisement with the OAA
  Facilitator, Ex. OAA Facilitatorgt Knowledge
  source connected 6 OAA Facilitatorgt 6
  (Retsina_InterOperator) can solve OAA
  Facilitatorgt update_data(_6771,_6788)
• and by its name, "Retsina_InterOperator", and
  icon showing in the "OAAMonitor" (cf.
  ltLVgt/OAA_Monitor_InterOp.gif).

37
Inter-Agent World Communications (cont)

• 3. Start the Retsina agent, "KQMLMessageSenderGUI"
  and register it with the"Retsina ANS" under the
  name, "Retsina_Matchmaker"(cf.
  ltLVgt/Test_Retsina_Matchmaker.gif).
• 4. Using OAA Startit, start the other OAA
  agents(cf. ltLVgt/OAA_Start-It_AllAgentsUp.gif,
  ltLVgt/OAA_Monitor_AllAgentsUp.gif).As those
  agents come on-line they will register and
  advertise with the OAAFacilitator. Each registry
  and advertisement will generate an event whichis
  captured by the InterOperator (cf.
  ltLVgt/OAA_Monitor_StartOaaWebL.gif) and forwarded
  to the "Retsina_Matchmaker"(cf.
  ltLVgt/Test_Retsina_Matchmaker_Updates.gif). In the
  future, the realRetsina Matchmaker will be the
  actual recipient of those messages.
• 5. Via the "Retsina_Matchmaker", it is possible
  to send messages to the OAAFacilitator (cf.
  ltLVgt/Hypothetical_MsgSend.gif).
• 6. OAA agents may be disconnected from the OAA
  Facilitator, or shutdown, andtheir status change
  will also be transmitted to the
  "Retsina_Matchmaker"(cf. ltLVgt/Agent_Shutdown.gif)
  .