A Personal Odyssey in the World of MultiAgent Research

1
A Personal Odyssey in the World of Multi-Agent
Research

• Victor R. Lesser
• Computer Science Department
• University of Massachusetts, Amherst
• July 26, 1999

2
Theme

• To provide a personal perspective on how a
  research career develops
• How ideas get created and evolve over time
• My personal research agenda
• Thoughts on where the field should be going

3
Creation and Evolution of MAS/DAI Approaches

• Functionally/Accurate Cooperative (FA/C)
  paradigm (Corkill and Carver)
• Tolerance of inconsistency
• Range of acceptable answers
• Error resolution through exchange of partial and
  tentative subproblem solutions
• Layered Agent Control (Corkill)
• Organizational structuring
• Satisficing agent coordination
• Interplay between local and non-local control

4
MAS/DAI ideas (continued)

• Partial Global Planning (PGP) and its successor
  GPGP as a framework for real-time agent
  coordination (Durfee Decker)
• Quantitative view of coordination based on a
  distributed search model
• Taxonomy of Subproblem interactions
• Agent Task Language TAEMS
• Fault-detection, diagnosis and adaptation as
  mechanism for coordination adaptation (Hudlicka
  Sugawara)
• Negotiation as a distributed search
• Multi-stage negotiation (Conry, Kuwabara Meyer)
• Level-commitment as protocol for search by
  self-interested agents (Sandholm)

5
Precursor Research

• Multi-computer operating system (1963-65)
• Excitement and complexity
• Reconfigurable Multi-processor Architecture
  (1966-1972)
• Dynamic Mapping of Process network on to
  Processor network
• Hardware kernel micro-operating system
• Mapping had to take into account of
  interrelationship among processes
• Issues of scale
• Control working set
• Importance of careful attention to empirical data
• Detail simulation

6
Hearsay-II Parallel, Cooperating
Knowledge-Source Model (1973-1975)

• Blackboard Architecture

7
Functional Descriptionof the Speech-Understandin
g KSs
8
Key Ideas in Blackboard Architecture

• Distributed, multi-level asynchronous search
• Integrated search at different abstraction levels
• Error-resolution through search and combining of
  approximate knowledge
• Sophisticated control reasoning
• Use of approximate knowledge for control
• Probabilistic view of search control
• Ideas and lesson learned in HS-II will be an
  important influence on future work

9
Transition to Distributed AI research the
importance of serendipity

• Parallel Processing Experiments (Fennel)
• The Wisdom of Allen Newell in forbidding me to
  talk
• Conversation with Bob Kahn (one of the founders
  of the Internet)

10
Distributed Hearsay-II Experiments (Erman,
1977-1979)

• Three node System with Overlapping Acoustic Data
• Communication of only High-Level Results
• No changes to basic architecture except for
  transmit and receive KSs

11
Network of Hearsay-II Systems
12
Results of Experiments

• Reproduced Results of Centralized System
• Slight Speed up and Reduced Communication
• Robustness in face of errorful communication
  channel
• Handled 35 error rate

13
Lessons It worked but...!

• Lack of coherent behavior
• Distraction
• Inappropriate Communication/Computation
• Redundant
• Lack of timeliness
• Lack of Focus
• Simplistic Local and Network Control Inadequate
• Local agent control needs to be more
  sophisticated when taking into account
  interactions with other agents

14
Key Question that Focused Research in the 1980s

• Can computationally tractable cooperative
  strategies be developed that maintain both
  coherent agent activity and system robustness?
• Implicit recognition of tension between reactive
  and reflective control

15
DVMT an Environment for Research in Cooperative
Distributed Problem Solving

• Build and evaluate more complex local agent
  control, coordination strategies and organization
  strategies

16
DVMT Agent Architecture(Corkill, 1983)

• Static Meta-level Control
• Organizational structuring
• Goal-Directed requests for information
• Integrating external and internal requests for
  processing

17
Integrating Data and Goal-Directed Control and
Organizational Structuring
18
Limitations of Static Meta-Level control (1987)

• Transmission of meta-level state information
  only partially successful
• Missing information about future activities
• Transmission of activity plans
• Partial global planning (Durfee)

19
PartialGlobalPlanningArchitecture
20
Partial Global Planning(Durfee)

• Representation of near-term agent activities
• Intermediate goals of activities
• Region and likely vehicles
• Behavioral characteristics
• Timing and likelihood of success
• Relationship of interagent activities
• Spatial overlapping and adjacent interpretation
  regions
• Basis for reorganizing local activities
• Exploiting predictive information
• Avoiding redundant activities
• Allow for load sharing

21
Some Important Digressions in Local Agent Control

• Meta-level control through Diagnosing of
  Problem-Solving Behavior (Hudlicka, 1984)
• Led to work on learning new situation specific
  coordination rules via detection and diagnosis
  (Sugawara, 1993)
• RESUN a framework for problem solving control
  based on symbolic reasoning about source of
  uncertainty (Carver, 1989)
• Led to work on DRESUN which provide a distributed
  framework for focused communication to resolve
  inconsistent agent beliefs (Carver, 1992-1997)

22
Further Work on Local Control

• Design-to-Time Scheduling (Garvey, Wagner)
• Approach to real-time agent control by
  dynamically constructing a schedule of activities
  to meet real-time deadlines
• Exploit the existence of alternative algorithms
  that trade off quality of solution for resource
  usage
• Led to deeper understanding of the issues of
  uncertainty
• Techniques for Sophisticated Local Control have
  strong implications for Non-Local Control

23
Digressions into the World of Negotiation

• Trying to understand cooperative and
  self-interested contracting
• Multi-stage negotiation (Conry et al., Lander,
  Laasris, Moehlman, Neiman)
• Cooperative dialogue among agents (1987-1997)
• Self-Interested Agent Interaction (Sandholm)
• Level commitment negotiation protocol
• Digressions are sometimes important for
  validating old intuitions and gaining new ones

24
Key Questions that Focused Research in the 1990s

• Is there some deeper theory of Agent Coordination
  implicit in this work
• Can we create infrastructure/ frameworks that
  eliminate a lot of the work in constructing MAS
  systems
• The search for the Holy Grail!

25
Generalizing PGP (Decker)
OR
AND
AND
AND
OR
AND
OR
AND
d2j1 . d2z
DATA/ Resources
d11 . d1j
A distributed goal search tree involving Agent1
and Agent2. The dotted arrows indicate
interdependencies between goals and data in
different agents, solid arrows dependencies
within an agent. The superscripts associated
with goals and data indicate the agent which
contains them (Jennings, 1993).
26
TAEMS A Domain Independent Framework for
Modeling User Activities

• The top-level goals/objectives/abstract-tasks
  that an agent intends to achieve
• One or more of the possible ways that they could
  be achieved, expressed as an abstraction
  hierarchy whose leaves are basic action
  instantiations, called methods
• A precise, quantitative definition of the degree
  of achievement in terms of measurable
  characteristics such as solution quality and time

27
TAEMS (continued)

• Task relationships that indicate how basic
  actions or abstract task achievement affect task
  characteristics (e.g., quality and time)
  elsewhere in the task structure
• Hard relationships (e.g., enables) denote when
  the result from one problem-solving activity is
  required to perform another, or when performing
  one activity precludes the performance of another
• Soft relationships (e.g. facilitates) express the
  notion that the results of one activity may be
  beneficial (or harmful) to another activity, but
  that the results are not required in order to
  perform the activity.
• The resource consumption characteristics of tasks
  and how a lack of resources affects them.

28
Information Gathering Example
29
(No Transcript)
30
Important Aspects of TAEMS

• Abstract View of Agent Activities
• level of detail necessary for understanding
  interactions and scheduling decisions
• Relationships among activities based on data
  flow enables, facilitates, favor, disable, etc.
• Relationships among activities how they
  contribute to the overall goal quality
  accumulation functions min, max, sum, etc.
• Schedule represents policy guidance for
  resource consumption and goals
• Worth Oriented
• Coordination as an Optimization Problem
• Real-time deadlines

31
WARREN Style Model of Multi-Agent Information
Gathering
32
GPGP Agent Architecture
33
Clear Separation of Local Control from
Coordination

• Coordination is generation of commitments
• Importance, utility, negotiability, decommitment
• Commitments lead to constraints on local
  scheduling
• Earliest start time of a task
• Deadline for completion of a task
• Interval when cant be executed

34
Integrating GPGP with other Approaches

• No person is an island unto themselves!

35
Putting it all together An architecture for
Large Agent Societies
36
Personal Perspective on MAS based on this
research path

• Agent Flexibility in Open Environments
• Agents need to be able to adapt their local
  problem solving to the available resources and
  goals of the system.
• Long-term learning needs to be an integral part
  of an agent architecture
• Agents not restricted to solving one goal at a
  time but may flexibly interleave their activities
  to solve multiple goals concurrently
• Error resolution/management needs to be integral
  part of agent problem solving
• Satisficing control
• Less than optimal but still acceptable levels of
  coordination among agents is traded off for a
  significant reduction in computational costs to
  implement cooperative control.
• Emphasis on satisficing behavior subtly moves the
  focus from the performance of individual agents
  to the properties and character of the aggregate
  behavior of agents.

37
Personal Perspective (continued)

• Predicting Performance of MAS systems is possible
  via probabilistic analysis
• Requires detail model of the environment
• Interaction between local and non-local agent
  control
• For effective agent coordination local agent
  control must have a certain level of
  sophistication in order to be able to understand
  what it has done, what is currently doing and
  what it intends to do
• Agent Roles and Responsibilities for large agent
  societies
• organizing agents in terms of roles and
  responsibilities can significantly decrease the
  computational burden of coordinating their
  activities.

38
Personal Perspective (continued)

• Centrality of Commitment to coordinated behavior
• Both long- and short-term coordination can be
  viewed in terms of commitments that have varying
  duration and specificity.
• Layered Control
• Modulationhigher layers providing constraints
  (policies) to lower levels that modulate
  (circumscribe) their control decisions
• Bi-directional Interaction(negotiation) among
  Layers Though constraints flow down the layers,
  information that flows in the other direction
  allows these constraints to be modified in case
  they cant be met or they lead to inappropriate
  behavior

39
Personal Perspective (continued)

• Situation-specificity
• There is no one best approach to organizing and
  controlling computational activities for all
  situations when the computational and resource
  costs of this control reasoning is taken into
  account.
• Quantitative View of Coordination
• Efficient and effective coordination must account
  for the benefits and the costs of coordination in
  the current situation.
• Coordination can be seen as a distributed
  mechanism for approximating a global optimization
  problem of task assignment

40
Personal Perspective (continued)

• Domain-independence The aspects of a domain that
  affect coordination can be abstracted and
  represented in a domain-independent language.
• An agents goals and criteria for their
  successful performance
• The performance characteristics and resource
  requirements of the alternative methods it
  possesses for accomplishing its goals,
• Qualitative and quantitative interdependencies
  among its methods and those of other agents

41
Personal Perspective (continued)

• Representing and Reasoning about Assumptions
• To the degree that the system can either
  re-derive or explicitly represent the assumptions
  behind these control decisions
• The more the system can effectively detect and
  diagnose the causes for inappropriate or
  unexpected agent behavior.
• Importance of Experimentation we are still an
  experimental science
• Dont yet have good ways to predict performance
• Statistical analysis is important but dont
  forget to look at the details

42
Current Research Projects

• Organizational Structuring, Design and Adaptation
  for Large Agent Societies (Horling, Vincent,
  Wagner)
• Real-time negotiation
• Layered, Domain-Independent Coordination
• GPGP-II (Wagner and Xuan)
• JIL to GPGP (Raja and Zhang)
• Distributed Situation Assessment/DRESUN
• Satisficing termination (Carver)
• Distributed Dynamic Bayesean Network and
  Influence Diagrams (Carver, Xiang, Zhang,
  Zilberstein)

43
Current Research Projects, Contd

• Survivable MAS Systems (Xuan)
• Coordinating for fault-tolerance
• Distributed Markov Decision Processes
• Resource Bounded Reasoning/Design-to-Criteria
  Scheduling (Raja and Wagner)
• Application Focus
• Cooperative Information Gathering
• Intelligent Home
• Supply Chain Manufacturing

44
Questions on my mind

• Is there a unified perspective with in which
  self-interest and cooperative agents can be
  understood?
• Is that perspective distributed search?
• How viable is it to think about emotions and
  power relationships as computational mechanisms
  making it possible to approximate the global
  optimal solution in a distributed way through
  local optimizations?
• Early work on skeptical nodes
• What type of meta-level framework (with limited
  and bounded computational overhead) will allow us
  reason about coordination costs as first-class
  objects so that it possible to dynamically
  balance problem-solving activities with
  coordination activities?

45
Questions on my mind (continued)

• Are Markov Decision Processes a computationally
  viable approach for dynamic multi-agent
  coordination?
• What knowledge and reasoning is necessary for
  designing top-down an agent organization?
• In what situation can bottom-up evolutionary
  organizational structuring produce good
  organizations
• Will the world of MAS/DAI be dominated in the
  future by game theoretic ideas and market
  mechanisms?
• what is the role of cooperative agents?

46
MAS in the 21st CenturyA Dominant Model

• Cooperating, Intelligent Agent Societies
• (seamless integration among people/machines)
• Constructionist perspective
• built out of heterogeneous, semi-autonomous
  agents
• having varying motivations from totally
  self-interested to benevolent
• High-level artificial language for cooperation
• Problem solving for effective cooperation will be
  as or more sophisticated than the actual domain
  problem solving
• reasoning about goals, plans, intentions, and
  knowledge of other agents

47
MAS in the 21st Century, Contd

• Operate in a satisficing mode
• Do the best they can within available resource
  constraints
• Deal with uncertainty as an integral part of
  network problem solving
• Complex organizational relationships among agents
• Highly adaptive/highly reliable
• Learning will be an important part of their
  structure (short-term/long-term)
• Able to adapt their problem-solving structure to
  respond to changing task/environmental conditions
• Profound implications for AI Computer Science!

48
Important Directions for the Field to Realize
this Vision

• Development of software infrastructure to help
  build sophisticated, interacting agents
• What will it be wrappers, languages or frameworks
  or some combination?
• Techniques for Sharing of knowledge/data among
  heterogeneous agents
• Are ontologies the answer or will there be the
  need for more sophisticated knowledge
  translation approaches or specialized languages?

49
Directions (continued)

• Mechanisms for dynamically establishing
  interaction protocols among heterogeneous agents
• Are recent ideas such as civil agent societies
  by Dellacros and Klein a viable approach?
• Analysis tool for understanding the performance
  of such systems before they are implemented
• Design rules and mechanisms for agent societies
  so that they will not evolve in ways that lead to
  inappropriate behavior or poor performance

50
Parting Thoughts

• This is a very exciting time for researchers in
  MAS
• The practical application of this technology is
  here!
• The set of ideas that the field has developed
  only scratch the surface
• There is a tremendous amount of work to be done
• There are a lot of hard problems to work on
• Let your intuitions drive you not what is
  necessarily currently in fashion