2/7: Knowledge-based Planning

1
2/7 Knowledge-based Planning
2
Life must be lived forward, but can only be
understood backward.--Kierkegaard
Lp relaxation
3
Should we bother with Proof-based encodings?

• Clearly Graphplan-based encodings are better than
  backward-proof based encodings
• Because Graphplan does Mutex propagation
• And mutex propagation is harder to do on direct
  SAT
• And yet, it is useful to know how to set up
  encodings directlysince if we dont know how to
  do directed consistency enforcement, then we at
  least know how to solve the problem
• Particularly useful for non-classical problems,
  where we havent yet found really effective mutex
  propagation procedures.
• If neural networks are often the second best way
  of solving any problem, you can say that direct
  SAT(or other) encodings are the second best way
  of solving planning problems
• Which is better than no way at all

4
Connections to Refinement Planning Framework
5
Disjunctive Planning

• Idea Consider Partial plans with disjunctive
  step, ordering, and auxiliary constraints
• Motivation Provides a lifted search space,
  avoids re-generating the same failures multiple
  times (also, rich connections to combinatorial
  problems)
• The solution check now involves searching
  through the space of all (exponentially many)
  minimal candidates of the plan to see if any of
  them is a solution
• Issues
• Refining disjunctive plans
• Graphplan (Blum Furst, 95)
• Solution extraction in disjunctive plans
• Direct combinatorial search
• Compilation to CSP/SAT/ILP

6
Disjunctive Representations
--Allow disjunctive step, ordering and
auxiliary constraints in partial plans
lt 1,In(A), ? gt V lt 1 ,In(B), ? gt
In(A)
?
0
1 Load(A)
Load(A)
In(x)_at_ ?
?
0
1 or
In(B)
Load(B)
?
0
1 Load(B)
In(x)_at_ ?
At(A,E)_at_1 V At(B,E)_at_1
7
Refining Disjunctive Plans (1)
Indirect unioned approach
Maximum pruning power - Exponential refinement
cost INFEASIBLE
8
Refining Disjunctive plans (2)
Direct naive approach Put all actions at all
levels --Proposition list contains all
conditions
Trivial to construct - Loss of pruning power
(progressivity) gt too many (minimal)
candidates gt Costlier solution extraction
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
USED in SATPLAN
or
5 Unload(B,E)
6 Unload(A,M)
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
9
Refining Disjunctive plans (2)
Direct naive approach Put all actions at all
levels --Proposition list contains all
conditions
Trivial to construct - Loss of pruning power
(progressivity) gt too many (minimal)
candidates gt Costlier solution extraction
USED in SATPLAN
10
Refining Disjunctive plans (3)
Enforce partial 1-consistency Proposition
list avoids unsupported conditions
Polynomial refinement time -- Loss of pruning
power (progressivity)
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
or
5 Unload(B,E)
6 Unload(A,M)
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
11
Refining Disjunctive plans (4)
Enforce (partial) 2-consistency Proposition
list maintains interactions between pairs
of conditions
Polynomial refinement time higher pruning
power Better balance between refinement cost
and solution extraction cost
1 Load(A,E)
or
2 Load(B,E)
or
3 Fly(R)
or
4 Unload(A,E)
or
5 Unload(B,E)
6 Unload(A,M)
Can do higher level consistency, but it is rarely
worth the cost
7 Unload(B,M)
8 load(A,M)
9 load(B,M)
12
CUSTOMIZING PLANNERS WITH DOMAIN SPECIFIC
KNOWLEDGE

• User-assisted customization
• Planning by Cheating??
• (accept domain-specific knowledge as input)
• 2. Automated customization
• (learn regularities of the domain through
  analysis or
• experience)

13
Digression Domain-Independent vs. Domain
Specific vs. Domain Customized
Review

• Domain independent planners only expect as input
  the description of the actions (in terms of their
  preconditions and effects), and the description
  of the goals to be achieved
• Domain dependent planners make use of additional
  knowledge beyond action and goal specification
• Domain dependent planners may either be stand
  alone programs written specifically for that
  domain OR domain independent planners customized
  to a specific domain
• In the case of domain-customized planners, the
  additional knowledge they exploit can come in
  many varieties (declarative control rules or
  procedural directives on which search choices to
  try and in what order)
• The additional knowledge can either be input
  manually or in some cases, be learned
  automatically

Unless noted otherwise, we will be talking about
domain-independent planning
14
Improving Performancethrough Customization

• Biasing the search with control knowledge
  acquired from experts
• Non-primitive actions and reduction schemas
• Automated synthesis of customized planners
• Combine formal theory of refinement planning and
  domain-specific control knowledge
• Use of learning techniques
• Search control rule learning
• Plan reuse

15
User-Assisted Customization(using
domain-specific Knowledge)

• Domain independent planners tend to miss the
  regularities in the domain
• Domain specific planners have to be built from
  scratch for every domain
• An Any-Expertise Solution Try adding domain
  specific control knowledge to the
  domain-independent planners

16
Task Decomposition (HTN) Planning

• The OLDEST approach for providing domain-specific
  knowledge
• Most of the fielded applications use HTN planning
  
• Domain model contains non-primitive actions, and
  schemas for reducing them
• Reduction schemas are given by the designer
• Can be seen as encoding user-intent
• Popularity of HTN approaches a testament of ease
  with which these schemas are available?
• Two notions of completeness
• Schema completeness
• (Partial Hierarchicalization)
• Planner completeness

17
Modeling Reduction Schemas
GobyBus(S,D)
In(B)
t1 Getin(B,S)
Hv-Tkt
t2 BuyTickt(B)
t3 Getout(B,D)
Hv-Money
At(D)
18
Stuff to add..

• Domain knowledge as prescriptive vs. suggestive
• Recipes as HTN schemas whose correctness is not
  known
• The fact that soundness of HTN schemas can be
  checked but not completeness even with full
  domain model at the prec/effect level

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
A concretization is a task network produced by
repeatedly reducing a non-primitive task
until all tasks are primitive
23
Modeling Action Reduction
Affinity between reduction schemas and plan-space
planning
24
Dual views of HTN planning

• Capturing hierarchical structure of the domain
• Motivates top-down planning
• Start with abstract plans, and reduce them
• Many technical headaches
• Respecting user-intent, maintaining systematicity
  and minimality
• Kambhampati et. al. AAAI-98
• Phantomization, filters, promiscuity,
  downward-unlinearizability..

• Capturing expert advice about desirable solutions
• Motivates bottom-up planning
• Ensure that each partial plan being considered is
  legal with respect to the reduction schemas
• Directly usable with disjunctive planning
  approaches
• Connection to efficiency is not obvious

Relative advantages are still unclear...

Barrett, 97
25
2/12
26
(No Transcript)
27
Changes to Plan-Space Refinement(in the presence
of non-primitive tasks)
28
(No Transcript)
29
HTN vs. T-STN

• Naus book talks about a special version of HTN
  called T-STN (Total-ordering STN)
• T-STN schemas are HTN schemas where all the
  ordering relations between tasks are constrained
  to be contiguity constraints
• This means that the concretizations of different
  tasks cannot be interleaved
• Bug or feature?
• From the planners point of view, it can be a
  feature Most of the difficulties in HTN planning
  come from the need to do plan-space refinement in
  the presence of non-primitive tasks
  (phantomization, conflict resolution etc..)
  eliminating that will make HTN planning a
  no-brainer
• BUT from the domain-writers point of view it is
  clearly a BUG
• Would have to write many more schemas
• Same is true of Yangs unique main sub-action
  thing (although it is less drastic than T-STN)
• (few recipes are given with contiguity
  orderingswhich means that you cant do anything
  else while following the recipe)
• The only HTN planners that use STN schemas are
  those from Naus group ?
• SIPE and O-Plan use schemas with precedence
  orderings

30
(No Transcript)
31
(No Transcript)
32
HTN vs. STRIPS planningCompeting or
Complementary?

• Much of the discussion till now has been done
  with the assumption that HTN is just a way of
  doing STRIPS planning

• HTN can also be seen as solving a different
  problem
• Can develop plans at various levels of
  abstraction
• Can model domains with partial domain knowledge
• What you consider primitive action may be
  changeable
• What you thought was a correct plan may not be..

33
9/6
34
Full procedural control The SHOP way
Nau et. al., 99
Shop provides a high-level programming language
in which the user can code his/her domain
specific planner -- Similarities to HTN
planning Uses T-STN schemas -- Order
of use of reduction schemas can be
specified The SHOP engine can be seen as
an interpreter for this language
Blurs the domain-specific/domain-independent
divide How often does one have this level of
knowledge about a domain?
35
Non-HTN Declarative Guidance
Kautz Selman, AIPS-98
Invariants A truck is at only one location
at(truck, loc1, I) loc1 ! loc2 gt at(truck,
loc2, I) Optimality Do not return a package to
a location at(pkg, loc, I)
at(pkg,loc,I1) IltJ gt at(pkg,loc,j) Simplify
ing Once a truck is loaded, it should
immediately move in(pkg,truck,I)
in(pkg,truck,I1) at(truck, loc, I1) gt
at(truck, loc, I2) Once again,
additional clauses first increase the encoding
size but make them easier to solve after
simplification (unit-propagation etc).
36
Case Study SATPLAN with domain specific knowledge

• Instantiate the domain specific rules as well,
  and then add them to the encoding.
• Solve the full encoding
• Wont the encoding size increase with domain
  spefic rules?
• Yes, but the new rules support a lot of
  simplification so that after simplification, the
  encoding size actually reduces!!
• Example in Blocks world (from Kautz Selman,
  AIPS-99)
• BW-c

Original 4258 vars (10 increase) / 30986
clauses (300 increase)
Final (after simplification)
3526 vars 22535 clauses
37
SAT encodings of HTN planning
Mali Kambampati, AIPS-98

• Abstract actions can be seen as disjunctive
  constraints
• K-step encoding has each of the steps mapped to a
  disjunction of the non-primitive tasks
• If a step s is mapped to a task N, then one of
  the reductions of N must hold (The heart of
  encoding setup)
• The normal constraints of primitive
  action-based encoding
• Causal encodings seem to be a natural fit (given
  the causal dependencies encoded in reduction
  schemas)

Solutions for action based encodings
HTN-compliant Solutions
38
Solving HTN Encodings
Puzzle How can increasing encoding sizes lead to
efficient planning? Abstract actions and their
reductions put restrictions on the amount of
step-action disjunction at the primitive level.
--Reduction in step-action disjunction
propagates e.g. Fewer causal-link
variables, Fewer exclusion clauses Savings
wont hold if each non-primitive task has MANY
reductions
Kambhampati Mali, AIPS-98
39
Rules on desirable State Sequences TLPlan
approach
TLPlan Bacchus Kabanza, 95/98 controls a
forward state-space planner Rules are
written on state sequences using the linear
temporal logic (LTL) LTL is an extension of prop
logic with temporal modalities U until
always O next
ltgt eventually Example If
you achieve on(B,A), then preserve it until
On(C,B) is achieved ( on(B,A) gt
on(B,A) U on(C,B) )
40
TLPLAN Rules can get quite boroque
Good towers are those that do not violate any
goal conditions
Keep growing good towers, and avoid bad towers
How Obvious are these rules?
The heart of TLPlan is the ability to
incrementally and effectively evaluate the truth
of LTL formulas.
41
Control rules for choice points

• UCPOP developed a language in which user can
  write control rules that tell the planner what to
  do when it faces a specific sort of choice (e.g.
  between open conditions establishers for open
  conditions etc.).
• HSTSa metric temporal planner that NASA used in
  its DeepSpace mission used similar control rule
  language.
• The user needs to know the innards of the planner
  to write these rules
• The rules are hard to debug
• Learning such rules automatically is a way out

42
Folding the Control Knowledge into the planner
CLAY approach
Srivastava Kambhampati, JAIR 97

• Control knowledge similar to TLPlans
• Knowledge is folded using KIDS semi-automated
  software synthesis tool into a generic refinement
  planning template
• Use of program optimizations such as
• Finite differencing
• Context-dependent independent simplification
• Empirical results demonstrate that folding can be
  better than interpreting rules

Caveat Current automated software synthesis
tools have a very steep learning curve
43
Many User-Customizable Planners

• Conjunctive planners
• HTN planners
• SIPE Wilkins, 85-
• NONLIN/O-Plan Tate et. al., 77-
• NOAH Sacerdoti, 75
• Also SHOP (Nau et. al., IJCAI-99)
• State-space planners
• TLPlan Bacchus Kabanza, 95 99
• TALPlan Kvarnstrom Doherty, 2000
• Customization frameworks
• CLAY Srivastava Kambhampati, 97
• Disjunctive planners
• HTN SAT Mali Kambhampati, 98
• SATPLANDom Kautz Selman, 98

How do they relate? What are the tradeoffs?
44
With right domain knowledge any level of
performance can be achieved...

• HTN-SAT, SATPLANDOM beat SATPLAN
• Expect reduction schemas, declarative knowledge
  about inoptimal plans
• TLPLAN beats SATPLAN, GRAPHPLAN
• But uses quite detailed domain knowledge
• SHOP beats TLPLAN(but not TALPlan)
• Expects user to write a program for the domain
  in its language
• Explicit instructions on the order in which
  schemas are considered and concatenated

Who gets the credit? -The provider of domain
knowledge? -The planner that is capable of
using it?
45
Types of Guidance

• Declarative knowledge about desirable or
  undesirable solutions and partial solutions
  (SATPLANDOM Cutting Planes)
• Declarative knowledge about desirable/undesirable
  search paths (TLPlan TALPlan)
• A declarative grammar of desirable solutions
  (HTN)

(largely) independent of the details of the
specific planner affinities do exist between
specific types of guidance and planners)
Planner specific. Expert needs to understand the
specific details of the planners search space

• Procedural knowledge about how the search for the
  solution should be organized (SHOP)
• Search control rules for guiding choice points in
  the planners search (NASA RAX)

46
Where does guidance come from?

• Learned from (planning) experience
• EBL-based search control rule learning
• (UCPOPEBL), PRODIGY
• inductive learning
• Case-based reasoning (DerSNLP)
• Given by humans (often, they are quite willing!)
• As declarative rules (HTN Schemas, Tlplan rules)
• Dont need to know how the planner works..
• Tend to be hard rules rather than soft
  preferences
• Whether or not a specific form of knowledge can
  be exploited by a planner depends on the type of
  knowledge and the type of planner
• As procedures (SHOP)
• Direct the planners search alternative by
  alternative..

how easy is it to write control information?
47
Ways of using the Domain Knowledge

• As search control
• HTN schemas, TLPlan rules, SHOP procedures
• Issues of Efficient Matching
• To prune unpromising partial solutions
• HTN schemas, TLPlan rules, SHOP procedures
• Issues of maintaining multiple parses
• As declartative axioms that are used along with
  other knowledge
• SATPlanDomain specific knowledge
• Cutting Planes (for ILP encodings)
• Issues of domain-knowledge driven simplification
• Folded into the domain-independent algorithm to
  generate a new domain-customized planner
• CLAY
• Issues of Program synthesis

48
Conundrums of user-assisted cutomization

• Which planners are easier to control?
• Conjunctive planners are better if you have
  search control knowledge
• Forward State Space (according to TLPlan)
• Plan-space planners (according to HTN approaches)
• Disjunctive planners are better if your
  knowledge can be posed as additional constraints
  on the valid plans
• Which SAT encoding?
• HTN knowledge is easier to add on top of causal
  encodings
• Which approach provides the best language for
  expressing domain knowledge for the lay user?
• (Mine--no, no, Mine!)
• What type of domain knowledge is easier to
  validate?
• When does it become cheating/
  wishful-thinking
• Foolish not to be able to use available knowledge
• Wishful to expect deep procedural knowledge...

49
Evaluating KB-planners

• Since the 2nd IPC, there has been a separate
  track for knowledge-based planners at the
  competition.
• Basically only the TLPlan (and its variant
  TALPlan) and SHOP took part (in comparison close
  to a dozen planners take part in the
  domain-independent track)
• In 2000, TALPlan won
• In 2002, TLPlan won
• Quite controversial
• When SHOP does better than TLplan on Gizmo
  domain, what does this mean?
• That SHOP is a better planning algorithm than
  TLplan?
• That Dana Nau knows Gizmo domain better than
  Faheim Bacchus?
• That the easily available control knowledge in
  Gizmo domain is more naturally representable as
  TLplan rules than SHOP schemas?

50
Are we comparing Dana Fahiem or SHOP and
TLPlan?(A Critique of Knowledge-based Planning
Track at ICP)

• Subbarao Kambhampati
• Dept. of Computer Science Engg.
• Arizona State University
• Tempe AZ 85287-5406

51
The I am not an anti-dentiteDisclaimers..

• I think KB planning is a swell idea
• I started my career with HTN planning
• I think the KB planning track at IPC is a swell
  idea
• has done more to increase interest in KBplanning
  than the bi-annual polemics and laments about
  lack of interest in Knowledge-based planning
• I think Fahiem and Dana are REALLY swell
• (in case they dont buy that) I may already have
  a black-belt in Karate..

Id rather learn from one bird how to singthan
teach ten thousand stars how not to dance--e.e.
cummings
52
What are the lessons of KB Track?

• If TLPlan did better than SHOP in ICP, then how
  are we supposed to interpret it?
• That TLPlan is a superior planning technology
  over SHOP?
• That the naturally available domain knowledge in
  the competition domains is easier to encode as
  linear temporal logic statements on state
  sequences than as procedures in the SHOP
  language?
• That Fahiem Bacchus and Jonas Kvarnstrom are way
  better at coming up with domain knowledge for
  blocks world (and other competition domains) than
  Dana Nau?

We are NOT asking the right questions
53
Questions worth asking in KB planner comparisons
(IMHO)

• How easy/natural (for humans) is the language in
  which the planner accepts control knowledge?
• How easy is it to validate the control
  knowledge being input to the planner?
• Is the naturally available knowledge about a
  specific domain easily encoded in the language
  accepted by the planner?
• Does the planner allow any expertise
  behaviorsolving the problems even without any
  control knowledge, but improving performance with
  added control knowledge? (or is the control
  knowledge tightly intertwined with the domain
  physics?).

54
How/Why the competition is not asking the right
questions

• The role of the knowledge-engineer is played by
  the same person(s) who wrote the planner. So, the
  question of how natural the specific language is
  for third-party knowledge engineers is largely
  unaddressed.
• No reasonable time limits are placed on coming up
  with the control knowledge. So, we dont learn
  much (or anything) about whether or not naturally
  available knowledge about a domain is easily
  representable in the language accepted by the
  planner.

55
Some suggestions for change

• Recruit third-party volunteers who will play the
  role of knowledge engineers for the KB planners.
• Ideally, we would like to have the same people
  writing the control knowledge for a given domain
  for all the competing approaches (so one
  knowledge engineer per domain rather than one
  knowledge engineer per planner).
• (Alternative to above) Specify the control
  knowledge that is available, so all planners
  encode the same general knowledge.
• One idea might be to ask the designers of the
  domains (e.g. David Smith and his cohorts for the
  Satellite and Rovers domain) to provide, in
  english, what sort of control information they
  would like the planner to use.
• Measure the time taken to write and validate the
  control knowledge.
• Analyze the knowledge encoded by the different KB
  planners for the same domain
• Characterize it in terms of (a) whether the
  knowledge is procedural or declarative and (b)
  how hard would it be to learn the same
  knowledge.

56
Slides from here-on about learning domain
knowledge were not used
57
Automated Customization of Planners

• Domain pre-processing
• Invariant detection Relevance detection
• Choice elimination, Type analysis
• STAN/TIM, DISCOPLAN etc.
• RIFO ONLP
• Abstraction
• ALPINE ABSTRIPS, STAN/TIM etc.
• Learning Search Control rules
• UCPOPEBL,
• PRODIGYEBL, (GraphplanEBL)
• Case-based planning (plan reuse)
• DerSNLP, Prodigy/Analogy

Most of the work has been done in the context of
Conjunctive Refinement Planners
Read Terry Zimmermans Learning
assisted planning Looking back, Taking Stock and
Going Forward
58
Symmetry Invariant Detection

• Generate potential invariants and test them
• DISCOPLAN Gerevini et. al.
• Allows detection of higher-order mutexes
• Rintanens planner
• Uses model-verification techniques
• STAN/TIM
• Type analysis of the domain is used to generate
  invariants
• ONLP (Peot Smith)
• Use operator graph analysis to eliminate
  non-viable choices

59
Abstraction

• Idea
• Abstract some details of the problem or actions.
• Solve the abstracted version.
• Extend the solution to the detailed version
• Precondition Abstraction
• Work on satisfying important preconditions first
• Importance judged by
• Length of plans for subgoals ABSTRIPS, PABLO
• Inter-goal relations ALPINE
• Distribution-based HighPoint
• Strong abstractions (with downward refinement
  property) are rare
• Effectiveness is planner-dependent
• Clashes with other heuristics such as most
  constrained first

60
Example Abstracting Resources

• Most planners thrash by addressing planning and
  scheduling considerations together
• Eg. Blocks world, with multiple robot hands
• Idea Abstract resources away during planning
• Plan assuming infinite resources
• Do a post-planning resource allocation phase
• Re-plan if needed

(with Biplav Srivastava)
61
Learning Search Control Rules with EBL
Explain leaf level failures Regress the
explanations to compute interior node failure
explanations Use failure explanations to set up
control rules Problems -- Most branches
end in depth-limits gtNo
analytical explanation gtUse preference
rules? -- THE Utility problem gtLearn
general rules gtKeep usage statistics
prune useless rules
UCPOPEBL
If Polished(x)_at_S Initially-True(Polished(x))
Then REJECT
Stepadd(Roll(x),Cylindrical(x)_at_s)
(Kambhampati, Katukam, Qu, 95)
62
Example Rules (Learned)
UCPOP
Prodigy
If (and (current-node node)
(candidate-goal node
(shape obj Cyl))
(candidate-goal node
(surface-condition obj
polished))) Then Prefer (shape obj cyl) to
(surface-condition obj polished)
If Polished(x)_at_S Initially-True(Polished(x))
Then REJECT
Stepadd(Roll(x),Cylindrical(x)_at_s)
Pruning rule
Preference rule
63
Case-based Planning Macrops, Reuse, Replay

• Structures being reused
• Opaque vs. Modifiable
• Solution vs. Solving process (derivation)
• Acquisition of structures to be reused
• Human given vs. Automatically acquired
• Mechanics of reuse
• Phased vs. simultaneous
• Costs
• Storage Retrieval costs Solution quality

64
Case-study DerSNLP
( Ihrig Kambhampati, JAIR 97)

• Modifiable derivational traces are reused
• Traces are automatically acquired during problem
  solving
• Analyze the interactions among the parts of a
  plan, and store plans for non-interacting
  subgoals separately
• Reduces retrieval cost
• Use of EBL failure analysis to detect
  interactions
• All relevant trace fragments are retrieved and
  replayed before the control is given to
  from-scratch planner
• Extension failures are traced to individual
  replayed traces, and their storage indices are
  modified appropriately
• Improves retrieval accuracy

Old cases
65
DerSNLP Results
Performance with increased Training
Solvability with increased traning
Problem size (goals)
5
Library Size
3
1
(JAIR, 97)
66
Reuse in Disjunctive Planning

• Harder to make a disjunctive planner commit to
  extending a specific plan first
• Options
• Support opaque macros along with primitive
  actions
• Increases the size of k-step disjunctive plan
• But a solution may be found at smaller k
• Modify the problem/domain specification so the
  old plans constraints will be respected in any
  solution (Bailotti et. al.)
• MAX-SAT formulations of reuse problem
• Constrain the encoding so that certain steps may
  have smaller step-action mapping and ordering
  choices
• Causal encodings provide better support

with Amol Mali