Integrating Intelligent Assistants into Human Teams

1
Integrating Intelligent Assistants into Human
Teams

• Katia Sycara
• The Robotics Institute
• Carnegie Mellon University
• Pittsburgh, PA 15213
• [phone]
• katia_at_cmu.edu
• www.cs.cmu.edu/softagents

• Michael Lewis
• School of Information Sciences
• University of Pittsburgh
• Pittsburgh, PA 15260
• [phone]
• ml_at_sis.pitt.edu
• www.pitt.edu/cmlewis

2
Team Members CMU

• Liren Chen
• Somesh Jha
• Rande Shern
• Dajun Zeng

• Keith Decker
• Anadeep Pannu
• Vandana Verma

Prasad Chalasani Kostya Domashnev Onn Shehory
3
Team Members U. of Pittsburgh

• Michael Lewis (PI)
• Terry Lenox
• Emily Roth

4
Talk Outline

• Goals
• Potential Impact for the Navy
• Approach
• Research Issues
• Progress
• Plan for Next Year

5
Overall Research Goal

• Increase the effectiveness of joint Command and
  Control Teams through the incorporation of Agent
  Technology in environments that are
• distributed
• time stressed
• uncertain
• open (information sources, communication links
  and agents dynamically appear and disappear)
• Team members are distributed in terms of
• time and space
• expertise

6
Impacts for Navy

• Reduce time for a C2 team to arrive at a decision
  
• Allow C2 teams to consider a broader range of
  alternatives
• Enable C2 teams to flexibly manage contingencies
  (replan, repair)
• Reduce time for a C2 team to form a shared model
  of the situation
• Reduce individual and team errors
• Support team cohesion and team work skills
• Increase overall team performance

7
Transition Opportunities

• Maritime Crisis planning
• Target identification training
• Air campaign planning
• Strike planning
• Aircraft maintenance

8
Overall Approach

• develop an adaptive, self-organizing collection
  of Intelligent Agents (the RETSINA
  infrastructure) that interact with the humans and
  each other.
• integrate multimedia information management and
  decision support
• anticipate and satisfy human information
  processing and problem solving needs
• perform real-time synchronization of human
  actions
• notify about significant changes in the
  environment
• adapt to user, task and situation
• develop model libraries of individual and team
  tasks
• develop verifiable useful human-agent interaction
  techniques

9
Overall Research Issues

• Agents and Agent Interactions
• Human Agent Interaction
• Information Filtering and Integration

10
Overall Research Issues Agents and Agent
Interactions

• interleaving planning, replanning, execution
  monitoring and information gathering in a
  multiagent setting
• single agent architecture and self-awareness
• agent coordination scheme
• finding appropriate agents
• agent interoperability
• agent-to-agent task delegation protocols
• learning through agent interactions

11
Overall Research Issues Human Agent Interaction

• agent-based team aiding
• functional allocation between humans and agents
  (insert agents into military simulations and
  perform controlled experiments with human
  subjects to assess utility)
• human-agent trust
• development of task models (graphical task
  editor)
• user-guided instantiation of agents (agent editor)

12
Insert TeamAiding.ppt
13
Overall Research Issues Information Filtering
and Integration

• learning and tracking multiple interests of users
• increase relevance of retrieved information
  (refinement key words, relevance feedback,
  summary of most important information in
  documents)
• detecting interesting'' patterns from multiple
  data sources
• information integration and conflict resolution

14
Retsina Functional Organization
15
Characteristics of RETSINA Agents

• Agents act autonomously to accomplish objectives
• Goal-directed
• Taskable
• Running unassisted for long periods
• Proactive Reactive

16
Characteristics of RETSINA Agents (Contd.)

• Agents engage in peer-to-peer interactions
• Agents are taskable, i.e. users or other agents
  can delegate tasks to them, user acceptability
  and trust an important issue
• Can interact as cooperative teams or
  self-interested individuals
• Interaction protocols
• Coordination Strategies
• Negotiation Protocols
• Agents adapt to their environment, user, task and
  each other
• Adapt both at the individual level and at the
  societal level
• Employ Alternate Methods
• Learn from (and about) users and each other

17
Progress

• RETSINA system infrastructure development
• Java implementation
• RETSINA agent architecture
• increased planning sophistication in individual
  agents
• Middle agents
• Agent interaction protocols

18
Middle Agent Types
Service providers have capabilities and service
parameters Service requesters have service
request and preferences
Service Parameters Initially Known By
19
Retsina Agent Architecture
20
RETSINA Planning Mechanisms

• hierarchical task network-based formalism
• library of task reduction schemas
• alternative task reductions
• contingent plans, loops
• incremental task reduction, interleaved with
  execution
• information gathered during execution directs
  future planning
• resource and temporal constraints

21
A task Structure (Advertisement Task Structure)
22
Progress (Contd.)

• Agent interoperability
• language for capability advertisement (Aardvark)
• agent name server and distributed matchmakingª
• Human Agent Interaction
• Task Editor
• Agent Editor
• Human Agent Trust
• Team TANDEM experiments
• ________________________
• ª www.cs.cmu.edu/softagents/retsina/ans

23
Insert Aardvark.pptlanguage for capability
advertisement
24
Insert Interact.ppt Agent Editor
25
Progress (Contd.)

• Applications
• Information filtering Webmateª, DVINA
• Agents in team aiding ModSAF, multiagent air
  patrol, agent-aided aircraft maintenance?
• ___________________________
• ª www.cs.cmu.edu/softagents/webmate
• ? This application is done in collaboration with
  the CMU wearable computer project.

26
ModSAF Vision
27
(No Transcript)
28
(No Transcript)
29
Insert AirMain.pptAircraft Maintenance Task
30
Overview of the WebMate System

• Use the multiple TF-IDF vectors to keep track of
  user interests in different domains which are
  automatically learned
• Use the trigger pair model to automatically
  extract relevant words for refining search
• The user can provide multiple pages as relevance
  guidance for information search

31
Insert WebMate.ppt(more detailed description)
32
Insert WebMateDemo.ppt(detailed description of
WebMate demo)
33
Overview of Informedia

• One of the six Digital Libraries Initiative
  projects funded by the NSF, DARPA, NASA and
  others in collaboration with WQED
• A multimedia library that will consist of over
  one thousand hours of digital video, audio,
  images, text and other related materials
• Uses combined speech, language and image
  understanding technology to transcribe, segment
  and index the linear video.

34
Plans for Next Year

• Continue enhancing the functionality of
  individual agents (e.g., more sophisticated
  planning mechanisms)
• Improve the robustness of the RETSINA
  infrastructure
• Finish the implementation of the agent
  advertisement language (Aardvark)
• Refine agent task delegation framework,
  particularly contingent task delegation
• Investigate situation-dependent agent
  coordination strategies
• Investigate information- and action-based
  conflict resolution
• Expand the ModSAF team-aiding scenarios by
  introducing agents of additional types and
  functionalities

35
Plans for Next Year (Contd.)

• Develop explicit agent tasking mechanisms
• Identify appropriate indexing mechanisms for task
  structure cases
• Expand the functionalities of agent editor
• Automatically learn individual and team
  coordination patterns from team activity traces

36
Plan for Integrating the Parts of CMU MURI

• Work with U. of Pittsburgh to identify additional
  agent requirements needed for agent-based team
  aiding
• U. of Pittsburgh will test the effectiveness of
  agent-based team aiding in ModSAF scenarios with
  human subjects
• Incorporate multimedia information from
  Informedia into agent-based team aiding
• Use the wearable computers as the platform for
  running the collaborative aircraft maintenance
  agents