Planning as Satisfiability

1
Planning as Satisfiability

• CS672

2
Outline
0. Overview of Planning 1. Modeling and Solving
Planning Problems as SAT - SATPLAN 2. Improved
Encodings using Graph Analysis - BLACKBOX 3.
Improved Encodings using Compiled Control
Knowledge
3
Overview of Planning

• Find a sequence of operators that transform an
  initial state to a goal state
• State complete truth assignment to a set of
  variables (fluents)
• Goal partial truth assignment (set of states)
• Action a partial function State State
• specified by three sets of variables preconditio
  n, add list, delete list

4
Abdundance of Negative Complexity Results

• I. Domain-independent planning PSPACE-complete
• (Chapman 1987 Bylander 1991 Backstrom 1993)
• II. Domain-dependent planning NP-complete
• (Chenoweth 1991 Gupta and Nau 1992)
• III. Approximate planning NP-complete
• (Selman 1994)

5
Planning as Inference

• Planning as first-order theorem proving (Green
  1969)
• computationally infeasible
• STRIPS (Fikes Nilsson 1971)
• very hard
• Partial-order planning (modal truth criteria)
  (Tate 1977, Chapman 1985, McAllester 1991, Smith
  Peot 1993)
• can be more efficient, but still hard (Minton,
  Bresina, Drummond 1994)
• SATPLAN planning as propositional reasoning

6
Part 1 Modeling and Solving Planning Problems
as SAT
7
SAT Encodings

• Planning Problem -gt Propositional CNF by axiom
  schemas
• Discrete time, modeled by integers
• state predicates indexed by time at which they
  hold
• action predicates indexed by time at which
  action begins
• each action takes 1 time step
• many actions may occur at the same step

8
Encoding Conventions

• Actions imply preconditions and effects
• fly(x,y,i) ? at(x,i) route(x,y) at(y,i1)
• Conflicting actions cannot occur at same time (A
  deletes a precondition of B)
• fly(x,y,i) y?z ? ?fly(x,z,i)
• If something changes, an action must have caused
  it (Explanatory Frame Axioms)
• at(x,i) ?at(x,i1) ? ? y . route(x,y)
  fly(x,y,i)
• Initial and final states hold
• at(NY,0) ... at(LA,9) ...

9
Modeling Tricks

• Can often dramatically reduce size of problem by
  modeling techniques
• move(x,y,z,i) requires n4 vars
• pickup(x,y,i), putdown(x,z,i) requires 2n3 vars
• State-based encodings eliminate all action
  variables (compile away)
• at(x,i) ? at(x,i1) ? ? y . route(x,y)
  at(y,i1)
• at(x,i) x?y ? ?at(y,i)

10
Solution to a Planning Problem

• A solution is specified by any model (satisfying
  truth assignment) of the conjunction of the
  axioms describing the initial state, goal state,
  and operators
• Easy to convert back to a STRIPS-style plan

11
SATPLAN
instantiated propositional clauses
instantiate
axiom schemas
problem description
length
mapping
SAT engine(s)
interpret
satisfying model
plan
12
SAT Algorithms

• Systematic Search
• DP (Davis Putnam Logemann Loveland)backtrack
  search unit propagation
• satz (Chu Min Li) - variable selection by forward
  checking max unit props
• relsat (Bayardo) - dependency directed
  backtracking add new clauses at dead-ends
• Local Search
• Inspired by Mins-Conflict algorithm (Adorf,
  Johnson, Minton, Philips, Laird)
• GSAT (Selman), Walksat (Selman, Kautz
  Cohen)greedy local search noise to escape
  minima

13
Planning Benchmark Test Set

• Extension of Graphplan test set
• blocks world - up to 18 blocks, 1019 states
• logistics - complex, highly-parallel
  transportation domain.
• Logistics.d
• 2,165 possible actions per time slot
• 1016 legal configurations (22000 states)
• optimal solution contains 74 distinct actions
  over 14 time slots
• Problems of this size never previously handled by
  state-space planning systems

14
Scaling Up Logistics Planning
10000
1000
100
Graphplan
DP
log solution time
10
DP/Satz
Walksat
1
0.1
0.01
log.d
log.b
log.a
log.c
rocket.a
rocket.b
15
Randomized Restarts

• Solution randomize the systematic solver
• Add noise to the heuristic branching (variable
  choice) function
• Cutoff and restart search after a fixed number of
  backtracks
• In practice rapid restarts with low cutoff can
  dramatically improve performance
• (Gomes 1996, Gomes, Kautz, and Selman 1997,
  1998)

16
Increased Predictability
10000
1000
100
Satz
log solution time
10
Satz/Rand
1
0.1
0.01
log.d
log.b
log.a
log.c
rocket.a
rocket.b
17
What SATPLAN Shows

• General propositional theorem provers can compete
  with state of the art specialized planning
  systems
• New, highly tuned variations of DP surprising
  powerful
• result of sharing ideas and code in large SAT/CSP
  research community
• specialized engines can catch up, but by then new
  general techniques
• Radically new stochastic approaches to SAT can
  provide very low exponential scaling
• 2 orders magnitude speedup on hard benchmark
  problems

18
Why SATPLAN Works

• More flexible than forward or backward chaining
• Systematic most unit propagation at most highly
  constrained states
• Stochastic iterative repair
• Randomized algorithms less likely to get trapped
  along bad paths

19
Part 2 Improved Encodings by Graph Analysis
The BLACKBOX Planner
20
Graphplan

• Planning as graph search (Blum Furst 1995)
• Set new paradigm for planning
• Like SATPLAN...
• Two phases instantiation of propositional
  structure, followed by search
• Unlike SATPLAN...
• Interleaves instantiation and pruning of plan
  graph
• Employs specialized search engine
• Graphplan - better instantiation
• SATPLAN - better search

21
Graph Pruning

• Graphplan instantiates in a forward direction,
  pruning unreachable nodes
• conflicting actions are mutex
• if all actions that add two facts are mutex, the
  facts are mutex
• if the preconditions for an action are mutex, the
  action is unreachable!
• In logical terms limited application of negative
  binary propagation
• given ? P V ? Q, P V R V S V ...
• infer ? Q V R V S V ...

22
The Plan Graph
Facts
Facts
Actions
Facts
Facts
Actions
...
...
...
...
mutually exclusive
preconditions
add effects
delete effects
23
Translation of Plan Graph
Act1
Pre1
Fact
Pre2
Act2
Fact ? Act1 ? Act2 Act1 ? Pre1 ? Pre2 Act1 ?
Act2
24
General Limited Inference

• Generated wff can be further simplified by
  consistency propagation techniques
• Compact (Crawford Auton 1996)
• unit propagation is Wff inconsistant by
  resolution against unit clauses?
• O(n)
• failed literal rule is Wff P inconsistant
  by unit propagation?
• O(n2)
• binary failed literal rule is Wff P V Q
  inconsistant by unit propagation?
• O(n3)
• Complements domain specific limited inference
• Discovers hidden local structure!

25
General Limited Inference
26
Blackbox
Plan Graph
Mutex computation
STRIPS
Translator
CNF
Simplifier
General Stochastic / Systematic SAT engines
Solution
CNF
27
Blackbox Results
28
Applicability

• When is the BlackBox approach not a good idea?
• when domain too large for propositional planning
  approaches
• when long sequential plans are needed
• when solution time dominated by reachability
  analysis (plan-graph generation), not extraction
• when optimal or near optimal planning not
  necessary

29
Part 3 Improved Encodings Compiling Control
Knowledge
30
Kinds of Control Knowledge

• About domain itself
• a truck is only in one location
• About good plans
• do not remove a package from its destination
  location
• About how to search
• plan air routes before land routes

31
Expressing Knowledge

• Such information is traditionally incorporated in
  the planning algorithm itself
• or in a special programming language
• Instead use additional declarative axioms
• (Bacchus 1995 Kautz 1998 Chen, Kautz, Selman
  1999)
• Problem instance operator axioms initial and
  goal axioms control axioms
• Control knowledge constraints on search and
  solution spaces
• Independent of any search engine strategy

32
Axiomatic Control Knowledge

• State Invariant A truck is at only one location
• at(truck,loc1,i) loc1 ¹ loc2 É Ø
  at(truck,loc2,i)
• Optimality Do not return a package to a location
• at(pkg,loc,i) Ø at(pkg,loc,i1) iltj É Ø
  at(pkg,loc,j)
• Simplifying Assumption Once a truck is loaded,
  it should immediately move
• Ø in(pkg,truck,i) in(pkg,truck,i1)
  at(truck,loc,i1) É Ø at(truck,loc,i2)

33
Adding Control Kx to SATPLAN
Problem Specification Axioms
Control Knowledge Axioms
Instantiated Clauses
As control knowledge increases, Core shrinks!
SAT Simplifier
SAT Core
SAT Engine
34
Tradeoffs of Control Knowledge

• If the planning domain is inherently intractable,
  how can any amount of control knowledge make
  planning tractable?
• by reducing solution quality
• optimal planning - NP-Hard
• non-optimal - (maybe) Polynomial
• Issue speed / quality tradeoff
• Case study Control Knowledge in TLPLAN and
  BlackBox
• TLPLAN (Bacchus 1996) simple forward-chaining
  search with strong control rules

35
TLPlan
Temporal Logic Control Formula
36
Temporal Logic for Control

• I. Rules involves only static information
• II. Rules depends on the current state
• III. Rules depends on the current state and
  requires dynamic user-defined predicates

37
Category I Control Rules
Goal
Initial
a
a
a
SFO
ORL
NYC
Do NOT unload an object from an airplane unless
the object is at its goal destination
38
Pruning the Planning GraphCategory I Rules
Facts
Facts
Actions
Facts
Facts
Actions
...
...
...
...
39
Effect of Graph Pruning
40
Category II Control Rules
a
SFO
ORL
NYC
Do NOT move an airplane if there is an object in
the airplane that needs to be unloaded at that
location.
41
Control by Adding Constraints
Control Rules
Planning Formula
Constraints Clauses
42
Blackbox with Control Knowledge(Logistics domain)
43
Comparison between Blackbox and TLPlan(Parallel
Plan Length)
44
Comparison between Blackbox and TLPlan(Running
Time)
45
Comparison

• TLPlan (without Control)
• Intractable.
• TLPlan (with Control)
• fastest, but limited parallelism
• Blackbox (without Control)
• slower, high parallelism
• Blackbox (with Control)
• faster, high parallelism

46
Summary

• Easy to encode domain-specific knowledge in the
  planning as satisfiablity frame
• Key to order-of-magnitude scaling
• Propositional logic, temporal logic, ...
• Can be applied before/after SAT encoding
• Can control time / quality tradeoff
• Power of underlying SAT engines gives option of
  finding higher quality solutions
• Heuristics are independent from the SAT engine
• Can use same axioms for radically different
  problem solvers

47
How to Generate Control Kx

• Introspection
• Try to capture obvious inferences that are hard
  to deduce
• EBL (Minton, Kambhampati)
• Generalize trace of previous problem solving
• Static analysis (Smith, Etzioni, Knoblock, Peot)
• Analyze operators
• Inductive Logic Programming (Huang, Selman,
  Kautz)
• Find rules that hold for a set of previous
  high-quality solution plans

48
Conclusions

• Propositional approaches to Open-Loop planning
  using general SAT engines are highly competitive
  with specialized planning algorithms
• Synergy with Plan Graph approaches
• Can effectively employ purely declarative control
  knowledge
• Biggest limitation domains where number of
  objects is too large to instantiate