Planning with Loops

1
Planning with Loops

• Some New Results

Yuxiao (Toby) Hu and Hector Levesque University
of Toronto
2
Outline

• Introduction
• Representing plans with loops
• FSA plan a type of finite state controller
• Constructing plans with loops
• Search in the space of FSA plans
• Potential for improvements
• Conclusion

3
The Planning Problem

• Finitely many functional fluents, f1,...,fm,
  whose domain may be finite or infinite
• Finitely many actions, a1,...,an (world-changing
  or sensing)
• Initial state the possible values of each
  fluent
• Goal achieve some goal condition in all
  contingencies.

Like contingent planning solve a class of
problems Incomplete initial state with possibly
infinite cases
4
Motivating Example(a variant of striped-tower in
Srivastava et al. 2008)

• Fluents
• stackA (a list of block colors)
• stackB (a list of block colors)
• stackC (a list of block colors)
• hand (empty/red/blue)
• World changing actions
• pickA, pickB, putB, putC
• Sensing actions
• testA? (empty/nonempty)
• testB? (empty/nonempty)
• testH? (red/blue)

5
Motivating Example(a variant of striped-tower in
Srivastava et al. 2008)

• Initially
• stackAblue,red,red,blue
• stackB
• stackC
• handempty
• Goal
• stackA
• stackB
• striped(stackC)
• handempty

striped(X) is true iff Xred,blue,,red,blue
6
Example 1

• A linear solution
• pickA
• putB
• pickA
• putC
• pickB
• putC
• pickA
• putC
• pickA
• putC.

7
Example 2
8
Example 2 (cont.)

• pickA
• CASE testH? OF
•     - red
•         putC pickA
•         CASE testH? OF
•             -red
•                 putB ... ...
•             -blue
•                 putC ... ...
•     - blue
•         putB pickA
•         CASE testH? OF
•             -red
•                 putC ... ...
•             -blue
•                 putB ... ...

9
Example 3
Need LOOPS
Question Is there a generalized plan solving all
problems in this class?
10
Some of the Existing Approaches

• KPLANNER (Levesque 2005)generates robot programs
  by winding found conditional plans.
• Aranda (Srivastava et al. 2008)obtains
  generalized plans by winding an abstracted
  example plan.
• loopDISTILL (Winner and Veloso 2007)learns a
  dsPlanner by merging matching sub-plans of an
  example partial-order plan.

11
Outline

• Introduction
• Representing plans with loops
• FSA plan a type of finite state controller
• Constructing plans with loops
• Search in the space of FSA plans
• Potential for improvements
• Conclusion

12
Plan Representation Robot Programs
(Levesque 1996, 2005)

• A robot program is defined inductively by
• nil
• seq(A,P)
• case(A,R1P1,,RnPn)
• loop(P,Q).

KPLANNER (Levesque 2005) uses the robot program
representation.
13
Robot Program Some Examples

• pickA
• CASE testH? OF
•     - red
•         putC pickA
•         CASE testH? OF
•             -red
•                 putB ... ...
•             -blue
•                 putC ... ...
•     - blue
•         putB pickA
•         CASE testH? OF
•             -red
•                 putC ... ...
•             -blue
•                 putB ... ...

pickA putB pickA putC pickB putC pickA putC
pickA putC.
14
Plan Representation

• An FSA plan is a directed graph
• Each node represents a program state
• One unique "start" state
• One unique "final" state
• Non-final state associated with an action
• Each edge is associated with a sensing result
• Sensing result of world-changing actions can be
  omitted
•  

15
Plan Execution(for a single complete initial
world)

1. Use the "start state" as current program state
2. If current state is the "final state", then stop
   
3. Execute action associated to the current state
4. Follow the edge with returned sensing result
5. Make the node pointed to by this edge the current
   program state, and repeat from Step 2.

16
Robot Program vs. FSA Plan

• It can be shown that all robot programs can be
  represented by equivalent FSA plans.
• What about the reverse direction?

17
Robot Program vs. FSA Plan
CASE feel? OF - thirsty drink go2bed sle
ep - hungry eat go2bed sleep
18
Robot Program vs. FSA Plan
fail
revise?
get?
X
suggestion
instruction
fail
unworkable
ok
follow?
think?
succeed
workable
19
Outline

• Motivation
• Representing plans with loops
• FSA plan a type of finite state controller
• Constructing plans with loops
• Search in the space of FSA plans
• Potential for improvements
• Conclusion

20
Generating Plans with Loops

1. Start with the smallest FSA plan with only one
   non-final state.
2. If the current program state is final, the goal
   must be satisfied.
3. Otherwise, execute the action associated to the
   current program state, non-deterministically pick
   an applicable one if none is associated.
4. For each possible sensing result of the action,
   follow the transition and change the current
   state to the transition target. If no transition
   is associated to the sensing result,
   non-deterministically pick one for it.
5. Repeat from step 2.

21
Search in the Space of FSA Plans

(Possible values of stackA)
X
?
22
Search in the Space of FSA Plans

23
Search in the Space of FSA Plans

pickA
X
?
24
Search in the Space of FSA Plans

testA?
?
empty
?
25
Search in the Space of FSA Plans
testA?
empty
?
X
nonempty
26
Search in the Space of FSA Plans
testA?
X
?
empty
nonempty
27
Search in the Space of FSA Plans

testA?
empty
nonempty
X
?
pickB
28
Search in the Space of FSA Plans

testA?
empty
nonempty
X
?
testA?
29
Search in the Space of FSA Plans

testA?
empty
nonempty
pickA
30
Search in the Space of FSA Plans
testA?
empty
nonempty
?
X
pickA

31
Search in the Space of FSA Plans

testA?
?
empty
X
nonempty
pickA
32
Search in the Space of FSA Plans
testA?
empty
nonempty
?
X
pickA

33
Search in the Space of FSA Plans
testA?
empty
nonempty
pickA
34
And finally
35
Experimental Results

•  

(Using the same pruning rules.)
36
Experimental Results
Statistics for Aranda are estimation from figures
in (Srivastava et al. 08) without redoing their
experiments.
37
Potential for Improvements

• We have formulated planning with loops as a
  search problem, and obtained a baseline
  implementation.
• It is using blind depth-first (iterative
  deepening) search, and does not scale well
  without effective pruning rules.
• However, the baseline implementation is a good
  starting point for adding heuristics and trying
  other improvement.

38
Heuristics for Action Selection

• We tried a variant of the additive heuristics
  (Bonet and Geffner 2001)
• Assume all fluents are independent
• Count the number of action steps to change each
  fluent to a value that may satisfy the goal
• Sum of steps across all fluents used as goal
  distance
• Try successor states with shorter distance first

39
Heuristics for Action Selection

• Appear relatively effective, but still limited
• The domain of each fluent may be infinite, so
  exponential BFS was used
• There are two non-deterministic choices in the
  search algorithm (choice of action and choice of
  transition). Greedily improving only one does not
  always lead to a good decision.

40
Discussion

• Planning with loops is an interesting problem
• We formally defined FSA plans, representation for
  loopy plans. And a logical account for planning
  problems, FSA plans and correctness.
• We formulate planning with loops as a search
  problem.
• FSA planner, the baseline implementation,
  outperforms KPLANNER, and is good starting point
  for heuristics and other improvements.