11th Feb

1
11th Feb
2
h-A
h-B
Pick-A
Pick-B
cl-A
cl-B
he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
3
h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
cl-A
cl-A
cl-B
cl-B
St-B-A
he
he
onT-A
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B
he
he
he
4
Using the planning graph to estimate the cost of
single literals
1. We can say that the cost of a single literal
is the index of the first proposition level
in which it appears. --If the literal
does not appear in any of the levels in the
currently expanded planning graph,
then the cost of that literal is
-- l1 if the graph has been expanded to l
levels, but has not yet leveled off
-- Infinity, if the graph has been
expanded
(basically, the literal cannot be achieved from
the current initial state) Examples
h(he) 1 h (On(A,B)) 2 h(he) 0
5
Estimating the cost of a set of literals (e.g. a
state in regression search)
Idea 0. Max Heuristic Hmax(p,q,r..)
maxh(p),h(q),. Admissible, but very
weak in practice Idea 2. Sum Heuristic Make
subgoal independence assumption
hind(p,q,r,...) h(p)h(q)h(r)
Much better than set-difference heuristic in
practice. --Ignores ve interactions
h(he,h-A) h(he) h(h-A) 112
But, we can achieve both the literals with just
a single action, Pickup(A). So,
the real cost is 1 --Ignores -ve interactions
h(cl(B),he) 10 1 But, there is
really no plan that can achieve these two
literals in this problem So, the real cost
is infinity!
6
Positive Interactions

• We can do a better job of accounting for ve
  interactions in two ways
• if we define the cost of a set of literals in
  terms of the level
• hlev(p,q,r) The index of the first level
  of the PG where
• p,q,r appear together
• so, h(he,h-A) 1
• Compute the length of a relaxed plan to
  supporting all the literals in the set S, and use
  it as the heuristic () hrelax
• hrelax(S) greater than or equal to hlev(S)
• Interestingly, hlev is an admissible
  heuristic, even though hind is not! (Prove) How
  about hrelax?

7
Finding a relaxed plan

• Suppose you want to find a relaxed plan for
  supporting literals g1gm on a k-length PG. You
  do it this way
• Start at kth level. Pick one action for
  supporting each gi (the actions dont have to be
  distinctone can support more than one goal). Let
  the actions chosen be a1aj
• Take the union of preconditions of a1aj. Let
  these be the set p1pv.
• Repeat the steps 1 and 2 for p1pvcontinue until
  you reach init prop list.
• The plan is called relaxed because you are
  assuming that sets of actions can be done
  together without negative interactions.
• The optimal relaxed plan is the shortest relaxed
  plan
• Finding optimal relaxed plan is NP-complete
• Greedy strategies can find close-to-shortest
  relaxed plan
• Length of relaxed plan for supporting S is often
  longer than the level of S because the former
  counts actions separately, while the later only
  considers levels (with potentially more than one
  action being present at each level)
• Of course, if we say that no more than one action
  can be done per level, then relaxed plan length
  will not be any higher than level.
• But doing this basically involves putting mutex
  relations between actions

8
Use of PG in Progression vs Regression
Remember the Altimeter metaphor..

• Progression
• Need to compute a PG for each child state
• As many PGs as there are leaf nodes!
• Lot higher cost for heuristic computation
• Can try exploiting overlap between different PGs
• However, the states in progression are
  consistent..
• So, handling negative interactions is not that
  important
• Overall, the PG gives a better guidance

• Regression
• Need to compute PG only once for the given
  initial state.
• Much lower cost in computing the heuristic
• However states in regression are partial states
  and can thus be inconsistent
• So, taking negative interactions into account
  using mutex is important
• Costlier PG construction
• Overall, PGs guidance is not as good unless
  higher order mutexes are also taken into account

Historically, the heuristic was first used with
progression planners. Then they used it with
regression planners. Then they found progression
planners do better. Then they found that
combining them is even better.
9
Progression
Regression
A PG based heuristic can give two things 1.
Goal-directedness 2. Consistency. Progression
needs 1 more --So can get by without mutex
propagation Regression needs 2 more. --So may
need even higher consistency information
than is provided by normal PG.
10
Negative Interactions

• To better account for -ve interactions, we need
  to start looking into feasibility of subsets of
  literals actually being true together in a
  proposition level.
• Specifically,in each proposition level, we want
  to mark not just which individual literals are
  feasible,
• but also which pairs, which triples, which
  quadruples, and which n-tuples are feasible. (It
  is quite possible that two literals are
  independently feasible in level k, but not
  feasible together in that level)
• --The idea then is to say that the
  cost of a set of S literals is the index of the
  first level of the planning graph, where no
  subset of S is marked infeasible
• --The full scale mark-up is very costly,
  and makes the cost of planning graph construction
  equal the cost of enumerating the full progres
  sion search tree.
• -- Since we only want estimates, it is
  okay if talk of feasibility of upto k-tuples
• -- For the special case of feasibility of
  k2 (2-sized subsets), there are some very
  efficient marking and propagation procedures.
• This is the idea of marking and
  propagating mutual exclusion relations.

11

• Rule 1. Two actions a1 and a2 are mutex if
• both of the actions are non-noop actions or
• a1 is any action supporting P, and a2 either
  needs P, or gives P.
• some precondition of a1 is marked mutex with
  some precondition of a2

Rule 2. Two propositions P1 and P2 are marked
mutex if all actions supporting P1
are pair-wise mutex with all
actions supporting P2.
12
h-A
h-B
Pick-A
Pick-B
cl-A
cl-B
he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
13
h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
cl-A
cl-A
cl-B
cl-B
St-B-A
he
he
onT-A
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B
he
he
he
14

• Here is how it goes. We know that at every time
  step we are really only going to do one non-no-op
  action. So, at the first level either pickup-A,
  or pickup-B or pickup-C are done. If one of them
  is done, the others cant be. So, we put
  red-arrows to signify that each pair of actions
  are mutually exclusive.
• Now, we can PROPAGATE the mutex relations to the
  proposition levels.
• Rule 1. Two actions a1 and a2 are mutex if
• both of the actions are non-noop actions or
• a1 is a noop action supporting P, and a2 either
  needs P, or gives P.
• some precondition of a1 is marked mutex with
  some precondition of a2
• By this rule Pick-A is mutex with Pick-B.
  Similarly, the noop action he is mutex with
  pick-A.
• Rule 2. Two propositions P1 and P2 are marked
  mutex if all actions supporting P1 are pair-wise
  mutex with all actions supporting P2.
• By this rule, h-A and h-B are mutex in level 1
  since the only action giving h-A is mutex with
  the only action giving h-B.
• cl(B) and he are mutex in the first level, but
  are not mutex in the second level (note that
• cl(B) is supported by a noop and stack-a-b
  (among others) in level 2. he is supported by
  stack-a-b, noop (among others). At least one
  action stack-a-b supporting the first is
  non-mutex with one actionstack-a-b-- supporting
  the second.

15
Some observations about the structure of the PG
1. If an action a is present in level l, it will
be present in all subsequent levels. 2.
If a literal p is present in level l, it will be
present in all subsequent levels. 3. If
two literals p,q are not mutex in level l, they
will never be mutex in subsequent levels
--Mutex relations relax monotonically as
we grow PG 1,2,3 imply that a PG can be
represented efficiently in a bi-level
structure One level for propositions and one
level for actions. For each
proposition/action, we just track the first time
instant they got into the PG. For mutex
relations we track the first time instant
they went away. A PG is said to have leveled
off if there are no differences between two
consecutive proposition levels in terms of
propositions or mutex relations. --Even
if you grow it further, no more changes can
occur..
16
Level-based heuristics on planning graph with
mutex relations
We now modify the hlev heuristic as follows
hlev(p1, pn) The index of the first level of
the PG where p1, pn appear together
and no pair of them are marked
mutex. (If there is no
such level, then hlev is set to l1 if the PG is
expanded to l levels,
and to infinity, if it has been expanded until it
leveled off)
This heuristic is admissible. With this
heuristic, we have a much better handle on both
ve and -ve interactions. In our example, this
heuristic gives the following reasonable
costs h(he, cl-A) 1 h(cl-B,he) 2
h(he, h-A) infinity (because they
will be marked mutex even in the final level of
the leveled PG)
Works very well in practice
17
But Level Doesnt always win over SUM
Solution length/ time in sec.
256MB, 500MHz
18
13th Feb
19
Qns on PG?

• Consider a set of subgoals p,q,r,s
• If the set appears at level 12 without any pair
  being mutex, is there guaranteed to be a plan to
  achive p,q,r,s using a 12-step plan?
• If p,q appear in level 12 without being mutex,
  is there a guaranteed 12-step plan to achieve
  p,q?
• If p appears in level 12 without being mutex,
  is there a guarnateed 12-step plan to achieve p
  with ?

PG does approximate reachability analysis
20
Massaging good (inadmissible) heuristics from hsum

• Hsum tends to do better sometimes than Hlev (and
  vice versa)
• Here are some heuristics that do better than both
  (can be thought of as adjusting the sum
  heuristic)
• Combo (add em both! HsumHlev )
• Hsum a negative interaction penalty
• -Degree of interaction in terms of ?
• ?(p,q) lev(p,q) - maxlev(p), lev(q)
• ?(p,q) if p and q are static mutex
• 0 lt ?(p,q) lt if p and q are level specific
  mutex
• ?(p,q) 0 otherwise (p and q are
  non-interacting)
• Relaxed plan length a negative interaction
  penality
• So, start with Hsum and

21
Adjusted Sum Heuristic

• Adjust the Sum heuristic to take positive and
  negative interactions
• HAdjSum2M(S) length(RelaxedPlan(S)) max p,q?S
  d(p,q)
• Relaxed Plan computed by ignoring all the mutex
  relations
• Second component is an approximation of a penalty
  induced by ignoring the negative interactions
• Degree of interaction
• ?(p,q) lev(p,q) - maxlev(p), lev(q)
• ?(p,q) if p and q are static mutex
• 0 lt ?(p,q) lt if p and q are level specific
  mutex
• ?(p,q) 0 otherwise (p and q are
  non-interacting)

Shown to be quite effective
Nguyen Kambhampati, AAAI 2000
22
But Level Doesnt always win over SUM
Solution length/ time in sec.
256MB, 500MHz
23
A family of Adjusted heuristics
24
AltAlt
Uses a hybrid of Level and sum Heuristics
--sacrifices admissibility --uses partial
PG to keep heuristic cost down
25
AltAlt Optimizations

• Cost of Computing the Heuristic can be high

• Bi-level Planning Graph representation
• Partial expansion of the PG

• Branching factor can still be quite high

• Use PG to prune unpromising choices
• Select actions in lev(S) vs Levels-off
• (paction pruning actions strategy)

26
Bi-Level Planning Graph

• Two Layers of Graph divided into ranks
• A Facts Array
• An Actions Array
• Increases Graph Construction efficiency
• Reduces Storage requirements
• Also quite useful for temporal planning
• problems see TGP

Rank 1
Rank 0
Pointers Level Information
Rank 1
Rank 2
Rank 2
Facts
Actions
27
Partial Expansion

• Grow the graph to level k (k lt level-off level)
• Level(S) if k1 instead of , if S is not present
  in the graph without mutex

• Can limit the space and time resources expended
  on computing the planning graph by trading
  heuristic quality

28
PGs for reducing actions

• If you just use the action instances at the final
  action level of a leveled PG, then you are
  guaranteed to preserve completeness
• Reason Any action that can be done in a state
  that is even possibly reachable from init state
  is in that last level
• Cuts down branching factor significantly
• Sometimes, you take more risky gambles
• If you are considering the goals p,q,r,s, just
  look at the actions that appear in the level
  preceding the first level where p,q,r,s appear
  for the first time without Mutex.

29
Limiting Branching Factor using Planning Graphs

• PACTION strategy Pick actions at lev(S)
• (instead of at the last level)
• Rationale Lev(S) comprises the most significant
  actions for achieving state S from the Initial
  State

C,D
Reduces BF (albeit incomplete)
Levels-off
Lev(S)
State S
Levels off
30
Do PG expansion only upto level(l) where all top
level goals come in without being mutex
AIPS-00 Schedule Domain
Quality
Cost
31
Empirical Evaluation Logistics domain
HadjSum2M heuristic
Problems and domains from AIPS-2000 Planning
Competion (AltAlt approx in top four)