Planning with Goal Utility Dependencies

1
Planning with Goal Utility Dependencies

• J. Benton
• Department of Computer Science
• Arizona State University
• Tempe, AZ
• j.benton_at_asu.edu

Subbarao Kambhampati Department of Computer
Science Arizona State University Tempe,
AZ rao_at_asu.edu
Minh Do Embedded Reasoning Area Palo Alto
Research Center Palo Alto, CA minh.do_at_parc.com
Menkes van den Briel Department of Computer
Science Arizona State University Tempe,
AZ j.benton_at_asu.edu
2
Classical vs. Over-Subscription Planning

• Over-Subscription Planning
• Initial state
• Goals with differing utilities
• Actions with differing costs
• Find a plan with highest net benefit
• (cumulative utility cumulative cost)
• (best plan may not achieve all the goals)

• Classical Planning
• Initial state
• Set of goals
• Actions
• Find a plan that achieves all goals
• (prefer plans with fewer actions)

1/19
3
Goal Dependencies
goal interactions exist as two distinct types
cost dependencies
utility dependencies
Actions achieving different goals
interact positively or negatively
Goals may complement or substitute each other
a
Handled by previous planners AltWlt,
Optiplan, SapaPS
Investigated in this paper
2/19
4
Complementary Goals
Cost 10 Util 20
Cost 100 Util 50
Cost 110 Util 300
3/19
5
Substitute Goals
Cost 15000 Util 30000
Cost 45000 Util 35000
4/19
6
Conditional Dependency
Cost 500 Util 1000
7
Challenges

• Modeling goal utility dependencies
• Doing planning in the presence of utility (and
  cost) dependencies

5/19
8
Modeling Goal Dependencies
General Additive Independence (GAI) Model
(Bacchus Grove, 1995)
Util 20
Util 50
Util 300
Utility over sets of dependent goals
6/19
9
Planning Approaches
Pursue two different approaches

• Heuristic Search
• Scale up well
• Challenge Developing good heuristics

• Bounded-horizon Optimal using
• Integer Linear Programming (ILP)
• Supports action cost and goal utility naturally
• Handles the objective function of maximizing
  utility-cost tradeoffs easily
• Concern scaling up with complex problems

7/19
10
G1SC IP Encoding
IPPlan (van den Briel et. al., 2005)
t 1
t 2
t 3
I
AT_STORE
G
AT_HOME
Shoes
IN_CAR
I
AT_OFFICE
Car
AT_HOME

• IP Constraints
• Capture the flow of each variable from I to G
• Interaction between different flows

8/19
11
iPUD Extended G1SC Encoding

• Remove hard constraints on goal achievement.
• Introduce a new binary variable for each related
  goal set S.
• Add constraints to ensure that S is achieved when
  achieved (and vice versa).
• New objective function capturing goal utility
  dependencies.

9/19
12
SPUDS Heuristic Search Approach
Extending SapaPS (2004)
S3
S1
S5
I
S4
S2
Forward State-space Planning Search using
A Node evaluation g U(GS) Cost(PS)
h(S) expected additional benefit Output better
quality solutions given more time (anytime)
10/19
13
Heuristic Relaxed Plan
Introduced in FF (Hoffman Nebel, 2001)
Step 1 Estimate the lowest cost relaxed plan P
achieving all remaining goals
A3
A2
I
S
A4
A1
all remaining goals
relaxed plan
Going backward, greedily select lowest cost
action achieving all remaining goals
11/19
14
Heuristic
Step 2 Build Cost-dependencies between actions
in P
G1
A3
G2
A2
S
A4
A1
G3
Build the supported goal set GS(A1) G3,
GS(A2) G1,G2,G3
12/19
15
Heuristic
Step 3 Extract the optimal relaxed plan within P
G1
A3
G2
A2
S
A4
G3
A1
Removing costly goals and actions (solely)
supporting them
Set up and solve an IP encoding using GS(A)
and goal utility dependencies functions f(S)
13/19
16
Heuristic Summary
Step 1 Estimate the lowest cost relaxed plan P
achieving all remaining goals
Step 2 Build cost-dependencies between goals in
P
Step 3 Find the optimal relaxed plan within P
Approximate the relaxed plan with the best
utility-cost tradeoff
14/19
17
Experimental Setup

• IPC problems in 4 benchmark domains Satellite,
  ZenoTravel, TPP, Rovers
• Goals randomly selected as hard or soft
• Goal utilities and action costs randomly
  generated within upper/lower bounds.
• Goal dependencies randomly generated
• Compared iPud, SPUDS (three heuristics) and
  SapaPS.

15/19
18
Time-bounded Solution Quality
Problems 11-20 for each domain Total of 40
hardest problems
16/19
19
Quality Time Comparison
20 problems for each domain (harder going from
left to right) Planning time limit 600 secs
17/19
20
Summary

• Goal utility dependency in GAI framework
• iPud Bounded-horizon optimal plans
• SPUDS Heuristic forward search
• cost dependencies utility dependencies
• IP encoding for heuristic improvement

18/19
21
Future Work

• Quantitative Qualitative preference (PrefPlan
  by Brafman Chernyavsky)
• PDDL3 preference model
• Residual cost as in AltWlt (Sanchez Kambhampati)

19/19
22
Thank You !