Cost Sensitive Reachability Heuristics for Handling State Uncertainty

1
Cost Sensitive Reachability Heuristics for
Handling State Uncertainty

• Daniel Bryce Subbarao Kambhampati
• Arizona State University
• dan.bryce, rao_at_asu.edu

2
Abstract

• While POMDPs provide a general platform for
  conditional planning under a wide range of
  quality metrics they have limited scalability. On
  the other hand, uniform probability conditional
  planners scale very well, but many lack the
  ability to optimize plan quality metrics. We
  present an innovation to planning graph based
  heuristics that helps uniform probability
  conditional planners both scale and generate high
  quality plans when using actions with non uniform
  costs. We make empirical comparisons with two
  state of the art planners to show the benefit of
  our techniques.

3
Reachability Measures
Planning Graph for Belief Space
State Space
State Space Planning Graph
4
Compressed Reachability in Belief Space
Labelled Uncertainty Graph (LUG)

• Binary Decision Diagrams
• Initialize and/projection
• Operator and/preconditions
• Literal or/supporters

D. Bryce, S. Kambhampati, and D. Smith, Planning
in Belief Space with a Labelled Uncertainty
Graph, In AAAI Workshop on Markov Processes,
2004.
5
Handling Action Costs (CLUG)

• Planning Graph represent unit action costs, but
  using cost propagation captures non-uniform
  action cost to support goals
• Cost Propagation for the LUG (CLUG)
• The cost of a goal can be different for every
  subset of possible worlds
• The of costs is exponential in of worlds
• Track cost for individual worlds
• The of costs is exponential in of literals
• Loses world cost dependencies
• Track cost over fixed partitions of worlds
• Can bound the of costs by of graph levels by
  tracking cost for worlds that are newly reached
  in a level
• Can capture world dependencies

6
Example

• A patient goes to the doctor complaining of
  feeling unrested ( r), but he is unsure if he is
  actually sick (s Ç s). The doctor has two
  treatments 1) give the patient drug B to cure
  the sickness if he is sick, and have him rest, R,
  for a week to become rested, or 2) do a blood
  test, S, to determine if he is sick if so he
  takes drug C with no need to rest, otherwise
  rests for a week. Both treatments will ensure
  that he is not sick and rested ( s Æ r).

s?r
c(B) 10
c(R) 7
s?r
r
c(S) 9
B Drug B C Drug C R Rest S Blood Test
c(C) 20
s?r
r rested s sick
Conformant Cost 17 Contingent Cost 22.5
7
Cost Propagation on the LUG
s
s
s
B
?0(B)
?0(B)
B
s
s
s
C
?0(C)
C
?0(C)
r
r
R
?0(R)
R
?0(R)
r
r
r
Relaxed Plan (bold) Cost c(B) C(R) 10 7
17
8
The Medical Specialist
Average Path Cost Total Time
High Cost Sensors Low Cost Sensors
9
Rovers
Average Path Cost Total Time
High Cost Sensors Low Cost Sensors
10
Further Information

• http//rakaposhi.eas.asu.edu/belief-search
• D. Bryce and S. Kambhampati, Heuristic Guidance
  Measures for Conformant Planning, In Proceedings
  of ICAPS04, 2004.
• W. Cushing and D. Bryce, State Agnostic Planning
  Graphs, In Proceedings of AAAI05, 2005.