On Plan Management Issues

1
On Plan Management Issues
Ioannis Tsamardinos Martha E.
Pollack tsamard, pollack_at_cs.pitt.edu
Intelligent Systems Program and Computer Science
Dept. University of Pittsburgh, Pittsburgh, PA,
USA
2
Autonomous AgentsAutonomous Systems Capable of
Effective Goal-directed Behavior in Environments
Where

• things change (dynamic)
• new goals are (self) generated continuously
• reasoning and acting have to be interleaved
• new opportunities or failures arise
• there are other agents with whom they must
  interact or cooperate

3
Agent Applications

• Autonomous spacecraft (Deep Space 1).
• Delivery robots.
• Internet assistants.
• Disaster management agents.
• Intelligent tutors.
• Equipment monitoring agents.
• Health care tracking agents.
• Intelligent personal assistants (PMA, Nursebot).

4
Important Concepts for AA

• Commitment and stability of plans.
• Especially for distributed, cooperative agents,
  including humans.
• Continuous planning and execution.
• Need to accommodate and plan for new goals in the
  midst of executing current commitments.

5
Plan Management Capabilities

• Plan generation what actions achieve your goal?
• Plan elaboration how much detail should you
  include, and when should you add detail?
• Commitment when should you be willing to
  reconsider your existing plans?
• Environment monitoring what new opportunities
  and problems should you attend to?
• Alternative assessment whats the value of an
  alternative in context?
• Coordination/cooperation how should you
  interact with other agents?
• Meta-level control how much effort should you
  put into planning and evaluation tasks?

6
A Prototypical Architecture.
Plan Merging
Plan Schemata Library
WORLD
Goal Generator
Commitments
Plan Revision
Execution
Execution Monitoring
7
Overview

• Autonomous agents and plan management
• Plan generation
• Plan merging
• Applications
• Conclusions

8
Plan Generation andAI Planning
9
AI Plan Generation

• The basic plan generation problem
• Given a library of action descriptions L, find a
  sequence of actions that will achieve some goal G
  when executed in some known state S.
• Actions defined with preconditions and effects.

10
Some Plan Generation Approaches

• Search through space of states for a goal state.
• Search through space of plans for a plan that
  achieves the goals.
• Convert the problem to a SAT problem and solve it
  (SATPlan)
• Convert the problem to a graph and solve it
  (GraphPlan)
• Specify and solve the problem as Hierarchical
  Task Networks.

11
Classical Planning Extension

• Actions with stochastic outcomes Kushmerick et
  al., Goldman/Boddy, Blythe.
• Imperfect knowledge Etzioni, Peot/smith,
  Collins/Pryor.
• Non-categorical goals (rich utility functions)
  Boutilier, dean/Kaelbling/Littman, Haddawy,
  Williamson, Goodwin, Onder/Pollack.
• Rich temporal and resource constraints
  Bacchus/Kabanza, Ghallab, Muscettola,
  Pollack/Tsamardinos.
• Hierarchical decomposition Erol, Tsuneto,
  Young/Pollack/Moore.
• But all, still form a plan from a complete set of
  goals.

12
Plan Merging
13
Definition Plan Merging
S Steps T Temporal B Binding L Causal
Links Cond Conditional Links

• Problem Typically, considering two or more plans
  together, results in negative interactions
  (conflicts)

• Given
• P1 ltS1, T1, B1, L1, Cond1gt
• and a new plan P2 ltS2, T2, B2, L2, Cond2gt
• Find T12 and B12 such that
• T12 is consistent with T1?T2
• B12 is consistent with B1?B2
• The set P1, P2 with T12, B12 added is conflict
  free

14
Definition Plan Merging
S Steps T Temporal B Binding L Causal
Links Cond Conditional Links

• Given
• P1 ltS1, T1, B1, L1, Cond1gt
• Pn ltSn, Tn, Bn, Ln, Condngt
• T1n, B1n (previous merging solution)
• and a new plan
• Pn1 ltSn1, Tn1, Bn1, Ln1, Condn1gt
• Find T1(n1) and B1(n1) such that
• T1(n1) is consistent with T1? ?Tn1
• B1(n1) is consistent with B1? ?Bn1
• The set P1, ., Pn1 with T1(n1), B1(n1)
  imposed is conflict free

15
Desired Properties

• Stability
• T1(n1) and B1(n1) should be as close as
  possible to T1n, B1n
• In the best case T1n?T1(n1), B1n ?B1(n1), and
  S1nS1(n1)
• Efficiency
• Aim for improvement over full replanning.
• Reuse computation.
• Involve CSP technology.

16
Previous Approach Yang97 Relies on CSP
constrained variables conflicts domain values
possible resolutions
V1 A1 lt A2, D2 lt A1 V2 A2 lt A1, D1 lt A2
V3 A1 lt C, x ¹ y V4 A2 lt B, x ¹ y V5
BltC
17
Conflict-Resolution CSP
V2
It is an n-ary CSP. Yang chooses to make binary
inconsistencies explicit. During backtracking new
values are checked for consistency.
A1 lt A2 D2 lt A1
A2 lt A1 D1 lt A2
V1
A1 lt C x ? y
V4
A2 lt B x ? y
V3
BltC
V5
18
Limitations of Yangs approach
Problem Insufficient for plans with metric
temporal constraints and conditional branches.

• Metric Constraints
• C should occur at most 10 time units after B.
• E s duration is at least 20 time units.

19
Temporal Constraint Language

• Simple Temporal Networks Meiri/Dechter/Pearl
• Set of variables V, and constraints C l s - t
  u
• s, t will encode the events of starting or ending
  a plan step.
• Can encode a wide range of constraints duration
  of actions, separation period between actions,
  absolute time of execution, etc.
• Consistency check in polynomial time using
  shortest paths algorithms.

20
STN Example
Max_Thrust
Idle
Idle
21
Conditional Simple Temporal Networks (CSTN)

• Same as STNs but every node (variable) has a
  context assigned.
• Nodes are executed (occur) only when their
  context is true.
• CSTNs are used to encode temporal plans with
  conditional branches.

22
An Example CSTN
55,60
Ref
Branch
S
S
30,30
0,0
BE
BS
DS
DE
0,1
10,10
AS
AE
0,0
CE
CS
0,1
?S
?S
All edges not annotated with an interval are
assumed 0, ?
Actions A, B, C, D, and Branch
23
Conflict Identification

• Conflict exists between sp and st in
  PltS,T,B,L,Condgt iff
• ltsp,e,sugt?L
• st has e as an effect
• sp and st occur in consistently labeled contexts
• start(st) - end(su) 0 is consistent with T
• start(sp) - end(st) 0 is consistent with T
• Similar to conflict definition in IxTeT
  Ghallab, except for context labeling

st could come between sp and su
24
Plan Merging Algorithm
Identify Conflicts

• Merge-Plans (P1, P2 ).
• 1. Let P P1 ÈP2 .
• 2. Construct a CSTN C representing P.
• 3. Identify conflicts(P) in C.
• 4. Construct a conflict-resolution CSP encoding
  conflicts(P).
• 5. Find a solution, Sol, to the
  conflict-resolution CSP. (Set backtrack point.)
  If there are no more solutions, return failure.
• 6. Add the constraints in Sol to C the and
  determine consistency.
• 7. If C Sol is consistent, then return Sol.
• 8. Else, restore C to its original state,and
  backtrack at line 5.

Suggest a global solution (Conflict-CSP)
Check validity of temporal Constraints (CSTN)
25
CSTN Consistency

• Strong Consistency
• ?assignment, ?scenario
• Can we schedule CSTN variables so that all
  constraints are respected in all contexts?
• Weak Consistency
• ?scenario, ?assignment
• Can we schedule the variables when the execution
  scenario will be known?
• Dynamic Consistency
• Can we assign times to variables as context
  information pours in?

26
Efficiency Techniques

• Using problem specific heuristics for the CSP
  (subsumption relation Yang97).
• Reusing computation in weak consistency checking
  for CSTN.
• Employing better CSP techniques CDB, DCSP,
  nogood learning, etc.
• Improve communication between CSTN and
  conflict-CSP.

27
Applications
28
Plan Management Assistants

• Assist a user in managing their commitments.
• Services
• Suggest plans to achieve a goal, maximizing
  expected utility.
• Check consistency of a new goal with existing
  commitments, suggest solutions to identified
  conflicts.
• Suggest efficient ways of merging a new plan into
  existing commitments.
• Provide alerts when execution or plan elaboration
  deadlines are approaching.
• Monitor execution (with input from user), and
  suggest revisions as needed.

29
PMA Plan Management Agent
30
PMA

• Related to an intelligent, personal workflow
  manager
• tracks execution of the set of commitments
  (plans), issuing reminders as needed
• merges new plans with existing commitments
• identifies potential conflicts, and suggests
  solutions
• estimates incremental cost of alternative
  possible plans in context
• Prototype implemented for academic administrator.
• Investigating use for military command and
  control, and for astronaut activity management
  (space station).

31
Nursebot
32
Nursebot Motivation

• Elderly population is increasing.
• Demonstrated advantages to enabling elderly to
  remain at home
• emotional, cognitive, and physical advantages
• significant cost savings.
• However, many elderly have difficulties with
  Activities of Daily Living (ADL) physical or
  cognitive.
• Nursebot intelligent mobile robot that provides
  assistance when a human caregiver is not
  available.

33
Nursebot as Cognitive Prosthetic

• Nursebot assists an elder in managing their ADLs
• Tracks commitments, and provides reminders as
  needed.
• Please take your heart medication now.
• Wheel of Fortune will start in 5 minutes.
  Would you like to use the bathroom first?
• Selects plans and updates commitments in response
  to new input.
• Doctors appt. next Tues. at 2 determines that
  transportation is required, and that access van
  visits that day.
• Tracks activities and alerts elder and/or
  caregiver as needed
• You havent eaten all day. Shall I notify your
  nephew?

34
Other Functions of Nursebot

• Monitors health functions
• physical glucose level, ankle diameter, ...
• mental/cognitive game playing, simple quizzes
• Serves as a source of interaction for elders

35
Comparison With Related Paradigms

• Classical Planning Systems
• Solve one problem at a time, no agenda with
  current commitments.
• Limited expressiveness of plans.

• Our work on Plan Management
• Rich plan language including temporal constraints
  and contingencies (conditions).
• Evaluates cost in context.
• Projects current plans into the future,
  identifies conflicts, and suggests solutions.

• Reactive Systems (PRS, RAPS, etc.)
• Do not project current commitments to the future
  and therefore cannot identify conflicts.
• Limited causal information.
• Rich plan language but limited temporal
  constraints.
• No cost in context capabilities.

• Workflow Systems
• Limited capabilities for handling temporal
  uncertainty and contingencies.
• Limited plan interaction and threat resolution
  capabilities.
• No reasoning about value of alternative ways to
  perform a task.

• Calendar Systems (Microsoft Outlook)
• Have explicit time, but can only schedule simple
  events.
• Interactions limited to busy and free time slots,
  extremely limited temporal constraints.

36
Conclusions

• Agents require continuous planning and execution.
• Plan Management reasoning capabilities are
  required for Autonomous Agents
• Merging plans is central in plan management.
• We presented a algorithm for merging plans with
  rich temporal constraints and conditional
  branching .
• Current applications of our approach include
• PMA
• Nursebot.

37
What Next?

• Theory (for plan merging)
• Extend plan merging for hierarchical plans,
  include the option of adding steps.
• Address issues of adjustable autonomy.
• Extend underlying theory of CSTNs.
• Investigate use of advanced CSP techniques in
  plan merging.
• Theory for other plan management capabilities.
• Continued development of plan-management agents
  for various domains.

38
A model of Plan Management

• Library of Plan Schemata
• Agenda of commitments (I.e. instantiated plan
  schemata)
• Schemata are instantiated and merged with the
  rest of commitments.
• Plan instances (commitments) are monitored and
  executed.

39
Plan Generation shortfalls

• Plan Generation
• Generally inefficient.
• Views planning as a one time problem.
• No agenda of commitments.
• Limited expressiveness of plans.

• Desired for Aut. Agents
• Efficiency.
• Continuous planning and execution.
• Tracking and monitoring an agenda of commitments.
• Being able to represent rich temporal constraints
  and conditions.

40
Performance Issues

• The problem is NP-hard.
• Checking CSTN consistency is time-consuming
• There are different notions of consistency
• Strong Consistency O(VE)
• Weak Consistency Brute Force O(VE 2c), c number
  of context predicates.
• Dynamic Consistency ?