Agenda

1
Agenda
2
Handling lifted actions(action schemas)

• Progression doesnt change much!
• You can generate all the applicable groundings of
  the operator
• Regression changescan be less committed!
• Consider regressing a goal state P(a),Q(b) over
  an action schema A with effects P(x) and Q(y)
• What happens if the effects were U(x)gtP(x) and
  M(y)gtQ(y)

3
(No Transcript)
4
(No Transcript)
5
Atif M. Memon, Martha E. Pollack, Mary Lou Soffa
Plan Generation for GUI Testing. AIPS 2000
226-235
6
(No Transcript)
7
(No Transcript)
8
Relevance, Rechabililty Heuristics
Reachability Given a problem I,G, a (partial)
state S is called reachable if there is a
sequence a1,a2,,ak of actions which when
executed from state I will lead to a state
where S holds Relevance Given a problem I,G, a
state S is called relevant if there is a
sequence a1,a2,,ak of actions which when
executed from S will lead to a state satisfying
(Relevance is Reachability from
goal state)

• Progression takes applicability of actions into
  account
• Specifically, it guarantees that every state in
  its search queue is reachable
• ..but has no idea whether the states are relevant
  (constitute progress towards top-level goals)
• SO, heuristics for progression need to help it
  estimate the relevance of the states in the
  search queue

• Regression takes relevance of actions into
  account
• Specifically, it makes sure that every state in
  its search queue is relevant
• .. But has not idea whether the states (more
  accurately, state sets) in its search queue are
  reachable
• SO, heuristics for regression need to help it
  estimate the reachability of the states in the
  search queue

Since relevance is nothing but reachability from
goal state, reachability analysis can form the
basis for good heuristics
9
Reachability through progression
pqr
A2
A1
pq
pq
A3
A1
pqs
A2
p
pr
psq
A1
A3
A3
ps
ps
A4
pst
ECP, 1997
10
Planning Graph Basics

• Envelope of Progression Tree (Relaxed
  Progression)
• Linear vs. Exponential Growth
• Reachable states correspond to subsets of
  proposition lists
• BUT not all subsets are states
• Can be used for estimating non-reachability
• If a state S is not a subset of kth level prop
  list, then it is definitely not reachable in k
  steps

pqr
A2
A1
pq
pq
A3
A1
pqs
A2
p
pr
psq
A1
A3
A3
ps
ps
A4
pst
p q r s
p q r s t
p
A1
A1
A2
A2
A3
A3
A4
ECP, 1997
11

• The Tonight Show with Jay Leno
• "There are now a lot of questions about the
  authenticity of these memos shown on '60 Minutes'
  about President Bush and his service in the
  National Guard. Let me tell you something, if
  there's one thing you don't want to see, it's a
  president who really didn't win the election
  being brought down by phony documents."
• "Meanwhile the White House is scrambling to
  bolster President Bush's image. They're saying
  that it's true Bush did not go to Vietnam, but he
  did attend an early screening of 'Apocalypse
  Now'!"
• "John Kerry unveiled his newest campaign slogan
  'A mind is a terrible thing to make up."'

• "The candidates are arguing about the exact
  format of the presidential debates. Kerry wants
  to stand behind a podium and Bush wants to stand
  behind Dick Cheney."
• "The nationwide ban on assault weapons is
  scheduled to expire at midnight tonight. ... John
  Kerry criticized President Bush for not fighting
  to renew the ban. Well, you can understand why
  President Bush doesn't want to renew the ban on
  assault weapons. These are the first weapons of
  mass destruction he's actually been able to
  find!"
• "One thing about Kerry -- Kerry just can't seem
  to shake his rich guy image. Like today he
  challenged President Bush to three debates and a
  yacht race!"

12
(No Transcript)
13
Planning Graph Basics

• Envelope of Progression Tree (Relaxed
  Progression)
• Linear vs. Exponential Growth
• Reachable states correspond to subsets of
  proposition lists
• BUT not all subsets are states
• Can be used for estimating non-reachability
• If a state S is not a subset of kth level prop
  list, then it is definitely not reachable in k
  steps

pqr
A2
A1
pq
pq
A3
A1
pqs
A2
p
pr
psq
A1
A3
A3
ps
ps
A4
pst
p q r s
p q r s t
p
A1
A1
A2
A2
A3
A3
A4
ECP, 1997
14
Heuristics to guide Progression/Regression

• Set difference heuristic
• Intuition The cost of a state is the number
  of goals that are not yet present in it.
• Progression The cost of a state S is G \ S
• The number of state-variable value pairs in G
  which
• are not present in S
• Regression The cost of a state S is S \ I
• The number of state-variable value pairs in S
  that are
• not present in the initial state
• Problems with Set difference heuristic
• 1. Every literal is given the same
  cost. Some literals are
• harder to achieve than others!
• 2. It is assumed that the cost of
  achieving n-literals together is n
• This ignores the interactions
  between literals (subgoals).
• -- It may be easier to
  achieve a set of literals together than to
• achieve each of them
  separately (ve interactions)
• -- It may be harder to
  achieve a set of literals together than to
• achieve them
  separately. (-ve interactions)

15
Estimating the cost of achieving individual
literals (subgoals)
Idea Unfold a data structure called planning
graph as follows 1. Start with the initial
state. This is called the zeroth level
proposition list 2. In the next level, called
first level action list, put all the actions
whose preconditions are true in the initial
state -- Have links between actions
and their preconditions 3. In the next level,
called first level propostion list, put
Note A literal appears at most once in a
proposition list. 3.1. All the
effects of all the actions in the previous
level. Links the effects to the
respective actions. (If
multiple actions give a particular effect, have
multiple links to that
effect from all those actions) 3.2.
All the conditions in the previous proposition
list (in this case zeroth
proposition list). Put
persistence links between the corresponding
literals in the previous
proposition list and the current proposition
list. 4. Repeat steps 2 and 3 until there is no
difference between two consecutive
proposition lists. At that point the graph is
said to have leveled off
The next 2 slides show this expansion upto two
levels
16
h-A
h-B
Pick-A
Pick-B
cl-A
cl-B
he
onT-A
onT-A
onT-B
onT-B
cl-A
cl-A
cl-B
cl-B
he
he
17
h-A
on-A-B
St-A-B
on-B-A
h-B
Pick-A
h-A
h-B
Pick-B
cl-A
cl-A
cl-B
cl-B
St-B-A
he
he
onT-A
onT-A
Ptdn-A
onT-A
onT-B
onT-B
onT-B
Ptdn-B
cl-A
cl-A
cl-A
Pick-A
cl-B
cl-B
cl-B
Pick-B
he
he
he
18
Graph has leveled off, when the prop list has not
changed from the previous iteration
Have(cake) eaten(cake)
Dont look at curved lines for now
The note that the graph has leveled off now since
the last two Prop lists are the same (we could
actually have stopped at the Previous level since
we already have all possible literals by step 2)
19
Planning Graph Basics

• Envelope of Progression Tree (Relaxed
  Progression)
• Linear vs. Exponential Growth
• Reachable states correspond to subsets of
  proposition lists
• BUT not all subsets are states
• Can be used for estimating non-reachability
• If a state S is not a subset of kth level prop
  list, then it is definitely not reachable in k
  steps

pqr
A2
A1
pq
pq
A3
A1
pqs
A2
p
pr
psq
A1
A3
A3
ps
ps
A4
pst
p q r s
p q r s t
p
A1
A1
A2
A2
A3
A3
A4
ECP, 1997
20
Using the planning graph to estimate the cost of
single literals
1. We can say that the cost of a single literal
is the index of the first proposition level
in which it appears. --If the literal
does not appear in any of the levels in the
currently expanded planning graph,
then the cost of that literal is
-- l1 if the graph has been expanded to l
levels, but has not yet leveled off
-- Infinity, if the graph has been
expanded
(basically, the literal cannot be achieved from
the current initial state) Examples
h(he) 1 h (On(A,B)) 2 h(he)
0 How about sets of literals? Hind(S)
h(l1)h(l2)h(ln)
21
Progression
Regression
22
Subgoal interactions
Suppose we have a set of subgoals G1,.Gn
Suppose the length of the shortest plan for
achieving the subgoals in isolation is l1,.ln
We want to know what is the length of the
shortest plan for achieving the n subgoals
together, l1n If subgoals are
independent l1..n
l1l2ln If subgoals have ve
interactions alone l1..n lt l1l2ln
If subgoals have -ve interactions alone
l1..n gt l1l2ln
23
Estimating reachability of sets with Positive
Interactions

• We can do a better job of accounting for ve
  interactions in two ways
• if we define the cost of a set of literals in
  terms of the level
• hlev(p,q,r) The index of the first level
  of the PG where p,q,r appear together
• so, h(he,h-A) 1
• Compute the length of a relaxed plan to
  supporting all the literals in the set S, and use
  it as the heuristic () hrelax
• hrelax(S) greater than or equal to hlev(S)
• Because we may be considering more than one
  action per step
• Interestingly, hlev is an admissible
  heuristic, even though hind is not! (Prove)
• How about hrelax?

24
Relaxed plan

• Suppose you want to find a relaxed plan for
  supporting literals g1gm on a k-length PG. You
  do it this way
• Start at kth level. Pick an action for supporting
  each gi (the actions dont have to be
  distinctone can support more than one goal). Let
  the actions chosen be a1aj
• Take the union of preconditions of a1aj. Let
  these be the set p1pv.
• Repeat the steps 1 and 2 for p1pvcontinue until
  you reach init prop list.
• The plan is called relaxed because you are
  assuming that sets of actions can be done
  together without negative interactions.

25
More on Relaxed Plans

• The optimal relaxed plan is the shortest relaxed
  plan
• Finding optimal relaxed plan is NP-complete
• Greedy strategies can find close-to-shortest
  relaxed plan
• Length of relaxed plan for supporting S is often
  longer than the level of S because the former
  counts actions separately, while the later only
  considers levels (with potentially more than one
  action being present at each level)
• Of course, if we say that no more than one action
  can be done per level, then relaxed plan length
  will not be any higher than level.
• But doing this basically involves putting n-ary
  mutex relations between actions
• Normal binary mutex relations wont be enough.
  Consider three actions A,B,C which give p,q,r
  respectively. Start from the init state that is
  empty. Set-level on parallel graph will say
  that level of p,q,r is 1. The set-level on serial
  graph will say that set-level of p,q,r is 2still
  not enough.

26
(No Transcript)
27
Negative Interactions

• To better account for -ve interactions, we need
  to start looking into feasibility of subsets of
  literals actually being true together in a
  proposition level.
• Specifically,in each proposition level, we want
  to mark not just which individual literals are
  feasible,
• but also which pairs, which triples, which
  quadruples, and which n-tuples are feasible. (It
  is quite possible that two literals are
  independently feasible in level k, but not
  feasible together in that level)
• The idea then is to say that the cost of a set
  of S literals is the index of the first level of
  the planning graph, where no subset of S is
  marked infeasible
• The full scale mark-up is very costly, and makes
  the cost of planning graph construction equal the
  cost of enumerating the full progres sion search
  tree.
• Since we only want estimates, it is okay if talk
  of feasibility of upto k-tuples
• For the special case of feasibility of k2
  (2-sized subsets), there are some very efficient
  marking and propagation procedures.
• This is the idea of marking and propagating
  mutual exclusion relations.

28
(No Transcript)
29
Level-off definition? When neither propositions
nor mutexes change between levels
Have(cake) eaten(cake)
Dont look at curved lines for now
30
Mutex Propagation Rules
This one is not listed in the text

• Rule 1. Two actions a1 and a2 are mutex if
• both of the actions are non-noop actions or
• a1 is any action supporting P, and a2 either
  needs P, or gives P.
• some precondition of a1 is marked mutex with
  some precondition of a2

Serial graph
interferene
Competing needs
Rule 2. Two propositions P1 and P2 are marked
mutex if all actions supporting P1
are pair-wise mutex with all
actions supporting P2.
31
Level-based heuristics on planning graph with
mutex relations
We now modify the hlev heuristic as follows
hlev(p1, pn) The index of the first level of
the PG where p1, pn appear together
and no pair of them are marked
mutex. (If there is no
such level, then hlev is set to l1 if the PG is
expanded to l levels,
and to infinity, if it has been expanded until it
leveled off)
This heuristic is admissible. With this
heuristic, we have a much better handle on both
ve and -ve interactions. In our example, this
heuristic gives the following reasonable
costs h(he, cl-A) 1 h(cl-B,he) 2
h(he, h-A) infinity (because they
will be marked mutex even in the final level of
the leveled PG)
Works very well in practice
H(have(cake),eaten(cake)) 2
32
Some observations about the structure of the PG

• 1. If an action a is present in level l, it will
  be present in
• all subsequent levels.
• 2. If a literal p is present in level l, it will
  be present in all
• subsequent levels.
• 3. If two literals p,q are not mutex in level l,
  they will never
• be mutex in subsequent levels
• --Mutex relations relax monotonically as
  we grow PG
• 1,2,3 imply that a PG can be represented
  efficiently in a bi-level
• structure One level for propositions and one
  level for actions.
• For each proposition/action, we just track
  the first time instant
• they got into the PG. For mutex relations we
  track the first time instant
• they went away.
• PG doesnt have to be grown to level-off to be
  useful for computing heuristics
• PG can be used to decide which actions are worth
  considering in the search

33
PGs for reducing actions

• If you just use the action instances at the final
  action level of a leveled PG, then you are
  guaranteed to preserve completeness
• Reason Any action that can be done in a state
  that is even possibly reachable from init state
  is in that last level
• Cuts down branching factor significantly
• Sometimes, you take more risky gambles
• If you are considering the goals p,q,r,s, just
  look at the actions that appear in the level
  preceding the first level where p,q,r,s appear
  for the first time without Mutex.

34
(No Transcript)