Planning

1
Planning

• Shyh-Kang Jeng
• Department of Electrical Engineering/
• Graduate Institute of Communication Engineering
• National Taiwan University

2
Reference

• S. Russell and P. Norvig, Artificial
  Intelligence A Modern Approach, Englewood
  Cliffs, NJ Prentice Hall, 1995

3
Goal-Based Agents
Sensors
Environment
State
Environment Model
Options
Decision Maker
Goals
Agent
Effectors
4
Problem-Solving Agent (1)

• class ProblemSolvingAgent
• ActionSequence s
• WorldState state
• Goal g
• Problem problem
• public ProblemSolvingAgent()
• s null

5
Problem-Solving Agent (2)

• public Action run(Percept percept)
• state.update(percept)
• if( s null )
• g new Goal(state)
• problem new Problem(state, g)
• s search(problem)
• action recommend(s, state)
• s s.remainder(state)
• return action

6
Planning Agents

• Agents that constructs plans that achieves its
  goals, and then execute them
• Similar to problem-solving agents
• Difference from the problem-solving agents
• Representations of goals, states, and actions
• Ways in representing and searching for solutions

7
Simple Planning Agent (1)

• class SimplePlanningAgent
• KnowledgeBase kb
• KBMS kbms
• Plan p
• counter t // indicating time
• IdealPlanner planner
• public KBAgent()
• kb new KnowledgeBase()
• kbms new KBMS( kb )
• p null
• t 0

8
Simple Planning Agent (2)

• public Action run(Percept percept)
• kbms.tell(new
• PerceptSentence(percept, t))
• WorldState current new
• WorldState(kbms, t)
• if( p null )
• Goal g kbms.ask(
• new GoalQuery(t) )
• p planner.makePlan(current, g, kbms)

9
Simple Planning Agent (3)

• Action action
• if( p null ) action null
• else
• action p.first()
• p p.rest()
• kbms.tell( new
• ActionSentence(action,t) )
• t
• return action

10
A Shopping Problem

• Get a quart of milk and a bunch of bananas and a
  variable-speed drill
• Initial state
• The agent is at home, without any desired objects
• Operator set
• All the things that the agent can do

11
Basic Elements of Search-Based Problem-Solver

• Representation of states
• Complete
• Used for successor generation, heuristic function
  evaluation, and goal testing
• Representation of actions
• Generates successor state
• Representation of goals
• Goal test and heuristic function evaluation
• Representation of plans
• Sequence of actions

12
Search through Space of Situations in the World
Talk to Parrot
Go To Pet Store
Buy a Dog
Go To School
Go To Class
Go To Supermarket
Buy Tuna Fish
Start
Sleep
Buy Arugula
Finish
Read A Book
Buy Milk
Sit in Chair
Sit Some More
Etc. Etc. . . .
Read A Book
13
Difficulties of the Problem-Solving Agent
Approach (1)

• Actual branching factor would be in thousands or
  millions
• There are too many actions and too many states to
  consider
• Even if the heuristic function is available, the
  agent has to evaluate and rank almost all
  successor states (the guessing process)
• The agent has no idea what to try next and must
  start the guessing process all over again

14
Difficulties of the Problem-Solving Agent
Approach (2)

• The agent must consider sequence of actions
  starting from the initial state
• It forces the agent to decide first what to do in
  the initial state
• Until the agent has figured out how to obtain the
  various terms, it can not really decide where to
  go

15
Key Ideas behind Planning

• Use formal language, usually first-order logic,
  to represent states, goals, and actions
• Enables the planner to make direct connections
  between states and actions
• Free to add actions to the plan wherever they are
  needed
• No necessary connection between the order of
  planning and the order of execution
• Can make obvious or important decisions first
  to reduce the branching factor and to reduce to
  backtrack over arbitrary decisions
• Take conjunctive goals and solve them with a
  divide-and-conquer strategy

16
STRIP Representations for States

• States are represented by the conjunctions of
  function-free ground literals, that is predicates
  applied to constant symbols, possibly negated
• Example
• Initial state for the shopping problem

17
STRIP Representations for Goals

• Goals are also described by conjunctions of
  literals
• Can also contains variables (existentially
  quantified)
• Examples
• The shopping goal
• Being at a store that sells milk

18
STRIP Representations for Operators (Actions)

• Action description
• Name for a possible action
• Precondition
• Conjunction of atoms (positive literals) that
  says what must be true before the operator can be
  applied
• Effect
• Conjunction of literals (positive or negative)
  that describes how the situation changes when the
  operator is applied

19
Example of a STRIP Representation of Operators
Go(there)
20
Planning by Searching through Plan Space

• Start with a simple, incomplete plan (partial
  plan)
• Then we consider ways of expanding the partial
  plan until we come up with a complete plan that
  solves the problem
• Operators on plans
• Refinement operators
• Modification operators

21
Partial Order Planner vs. Total Order Planner

• Partial order planner
• Represents plan in which some steps are ordered
  (before or after) with respect to each other and
  some steps are unordered
• Total order planner
• Plans consist of a simple list of steps

22
Representations for Plans

• A plan defines a data structure consisting of the
  following components
• A set of steps (operators) STEPS
• A set of step ordering constraints ORDERINGS
• A set of variable binding constraints BINDINGS
• A set of causal links LINKS

23
Problem of Putting on a Pair of Shoes

• Goal
• Operators

24
Initial Plan for the Shoe-and-Sock Problem
25
Problems Defined by Partial Plans
Start
Start
Initial
State
LeftShoeOn
RightShoeOn
Goal
State
Finish
Finish
26
A Partial-Order Plan
Start
LeftSock
RightSock
RightSockOn
LeftSockOn
LeftShoe
RightShoe
LeftShoeOn,
RightShoeOn
Finish
27
Total Order Plans
Start
Start
Start
Start
Start
Start
Right Sock
Right Sock
Left Sock
Left Sock
Right Sock
Left Sock
Left Sock
Left Sock
Right Sock
Right Sock
Right Shoe
Left Shoe
Right Shoe
Left Shoe
Right Shoe
Left Shoe
Left Sock
Right Sock
Left Shoe
Right Shoe
Left Shoe
Right Shoe
Left Shoe
Right Shoe
Finish
Finish
Finish
Finish
Finish
Finish
28
Solution

• A solution is a plan that an agent can execute
• To include partial plan, we define a solution to
  be a complete, consistent plan
• Complete plan
• Every precondition of every step is achieved by
  some other step
• Consistent plan
• There are no contradictions in ordering or
  binding constraints

29
Initial Plan for the Shopping Problem
Start
At(Home) Sells(SM,Banana)
Sells(SM,Milk) Sells(HWS,Drill)
Have(Drill) Have(Milk)
Have(Banana) At(Home)
Finish
30
Actions
31
A Partial Plan
Start
At(s), Sells(s,Drill)
At(s), Sells(s,Milk)
At(s), Sells(s,Banana)
Buy(Drill)
Buy(Milk)
Buy(Banana)
Have(Drill), Have(Milk), Have(Banana), At(Home)
Finish
32
Refined Partial Plan
Start
At(HWS), Sells(HWS,Drill)
At(SM), Sells(SM,Milk)
At(SM), Sells(SM,Banana)
Buy(Drill)
Buy(Milk)
Buy(Banana)
Have(Drill), Have(Milk), Have(Banana), At(Home)
Finish
33
A Partial Plan that Achieves At Preconditions
Start
At(x)
At(x)
Go(HWS)
Go(SM)
At(HWS), Sells(HWS,Drill)
At(SM), Sells(SM,Milk)
At(SM), Sells(SM,Banana)
Buy(Drill)
Buy(Milk)
Buy(Banana)
Have(Drill), Have(Milk), Have(Banana), At(Home)
Finish
34
A Partial Plan that Gets the Agent to Stores
Start
At(Home)
At(Home)
Go(HWS)
Go(SM)
At(SM), Sells(SM,Milk)
At(HWS), Sells(HWS,Drill)
At(SM), Sells(SM,Banana)
Buy(Drill)
Buy(Milk)
Buy(Banana)
Have(Drill), Have(Milk), Have(Banana), At(Home)
Finish
35
Threats of Causal Links
S1
S3
S2
36
Causal Link Protection
S3
S1
S1
S2
S3
S2
Promotion
Demotion
37
A Solution Plan
Start
At(Home)
At(HWS)
Go(HWS)
Go(SM)
At(SM), Sells(SM,Milk)
At(SM)
At(SM), Sells(SM,Banana)
At(HWS), Sells(HWS,Drill)
Buy(Drill)
Buy(Milk)
Buy(Banana)
Go(Home)
Have(Drill), Have(Milk), Have(Banana), At(Home)
Finish
38
A Partial-Order Planning Algorithm (POP)

• function POP(initial, goal, operators) returns
  plan
• plan? MAKE_MINIMAL_PLAN(initial, goal)
• loop do
• if SOLUTION?(plan) then return plan
• Sneed,C ? SELECT_SUB_GOAL(plan)
• CHOOSE_OPERATOR(plan, operators, Sneed,C)
• RESOLVE_THREATS(plan)
• end

39
Function SELECT_SUBGOAL

• function SELECT_SUBGOAL(plan) returns Sneed,C
• pick a plan step Sneed from STEPS(plan)
• with a precondition C that has not
• been achieved
• return Sneed,C

40
Procedure CHOOSE_OPERATOR

• procedure CHOOSE_OPERATOR(plan,operators,
  Sneed,C)
• choose a step from operators or
• STEPS(plan) that has c as an effect
• if there is no such step then fail
• add the causal link to LINKS(plan)
• add the ordering constraint to
• ORDERINGS(plan)
• if is a newly added step from
• operators then
• add to STEPS(plan)
• add to ORDERINGS(plan)

41
Procedure RESOLVE_THREATS

• procedure RESOLVE_THREATS(plan)
• for each that threatens a link
• in LINKS(plan) do
• choose either
• Promotion Add to
• ORDERINGS(plan)
• Demotion Add to
• ORDERINGS(plan)
• if not CONSISTENT(plan) then fail
• end

42
Abstract Operators and Primitive Operators

• Abstract operators
• Can be decomposed into a group of steps that
  forms a plan that implements the operator
• Example Go(Supermarket)
• Primitive operators
• Steps that can be directly executed by the agent
• Example Forward(1cm), Turn(1deg)

43
Hierarchical Decomposition (1)
Build House
decomposes to
Obtain Permit
Construction
Pay Builder
Hire Builder
decomposes to
44
Hierarchical Decomposition (2)
Build Roof
Build Foundation
Build Frame
Build Interior
Build Walls
45
Extending the Language

• Partition the set of operators into primitive and
  non-primitive operators
• Add a set of decomposition methods
• Each method is of the form Decompose(o,p) which
  means that a non-primitive operator that unifies
  with o can be decomposed into a plan p

46
Decomposition Example

• Decompose(Construction,
• Plan(STEPSS1Build(Foundation),
• S2Build(Frame),
• S3Build(Roof),
• S4Build(Walls),
• S5Build(Interior)
• ORDERINGS
  , ,
• BINDINGS,
• LINKS
  
• 
  
• ) )

47
A Hierarchical Decomposition Planner

• function HD_POP(plan, operators, methods) returns
  plan
• inputs plan, an abstract plan with start and
  goal steps (and possibly other steps)
• loop do
• if SOLUTION?(plan) then return plan
• Sneed,C ? SELECT_SUB_GOAL(plan)
• CHOOSE_OPERATOR(plan,operators, Sneed,C)
• Snonprim ? SELECT_NONPRIMITIVE(plan)
• CHOOSE_DECOMPOSITION(plan, methods,
• Snonprim)
• RESOLVE_THREATS(plan)
• end

48
Change of Fields of planin CHOOSE_DECOMPOSITION
(1)

• If method is chosen as the decomposition for the
  step Snonprim,
• STEPS
• Add all the steps of method to the plan, but
  remove Snonprim
• BINDINGS
• Add all the variable binding constraints of
  method to the plan. Fail if this introduces a
  contradiction

49
Change of Fields of planin CHOOSE_DECOMPOSITION
(2)

• ORDERINGS
• Replace each constraints
  with constraints that order before the
  latest step of method
• Replace each constraints of the form
• with the constraints that order after
  the
• earliest step of method

50
Change of Fields of planin CHOOSE_DECOMPOSITION
(3)

• LINKS
• If was a causal link in
  plan, replace it by a set of links
  , where each is a step of method
  that has C as a precondition, and there is no
  earlier step of method that has C as a
  precondition
• For each link in plan,
  replace it with a set of links
  , where is a step of method that has C as an
  effect and there is no later step of method with
  C as an effect

51
Detailed Decomposition of a Plan Step (1)
Buy Land
Own(Land)
Have(House)
Build House
Move In
Get Loan
Have(Money)
52
Detailed Decomposition of a Plan Step (2)
Own(Land)
Buy Land
Have(House)
Obtain Permit
Construction
Move In
Pay Builder
Hire Builder
Have(Money)
Get Loan
53
Blocks World

• Consists of a set of cubic blocks sitting on a
  table
• Blocks can be stacked, but only one block can fit
  directly on top of another
• A robot arm can pick up a block and move it to
  another position, either on the table or on top
  of another block
• The arm can only pick up one block at a time, so
  it can not pick up a block that has another one
  on it

54
Conditional Effects

• Op(ACTION Move(b,x,y),
• PRECOND
  ,
• EFFECT
  )
  
• Op(ACTION MoveToTable(b,x),
• PRECOND
  ,
• EFFECT
  )
• Op(ACTION Move(b,x,y),
• PRECOND
  ,
• EFFECT
• 
  
• 
  )

55
Measures and Resources

• Measures
• Amounts of something, such as money or volume
• Referred to as (1.5) or Gallons(6) or GasLevel
• Measure fluents
• Expression whose value is a measure
• Example GasLevel
• Measure functions
• Applied to objects to yield measures
• Example Volume(GasInCar)
• Resources
• Measures that can be produced or consumed
• Example Cash, GasLevel

56
Preconditions Including Measures
57
Operators with Effects Including Measures
58
Temporal Constraints

• Time as a resource
• Examples
• Time different from other resources
• Actions that are executed in parallel consume the
  maximum of their respective times
• Constraints on the time have to be consistent
  with ordering constraints
• Time never goes backward

59
Problems with Incomplete and Incorrect Information

• Incompleteness
• The world is inaccessible
• Example
• In the shopping world, the agent may not know
  where the milk is kept unless it asks
• Incorrectness
• The world does not necessarily match the agents
  model of it
• Example
• The price of milk may have doubled overnight

60
Dealing with Incomplete and Incorrect Information

• Conditional planning
• Constructing a conditional plan that accounts for
  each possible situation or contingency that could
  arise
• Execution monitoring
• By monitoring what is happening while the agent
  executes the plan, the agent can tell when things
  go wrong and do re-planning to find a way to
  achieve its goals from the new situation

61
Problem of Fixing a Flat Tire (1)

• Initial state
• Goal

62
Problem of Fixing a Flat Tire (2)

• Operators

63
Conditional Planning for Tire-Fixing Problem

• Conditional plan
• Sensing action

64
Context of a Step

• A steps context is simply the union of the
  conditions that must be hold in order for the
  step to be executed
• Essentially, it describes the branch on which
  the step lies
• Once it is established that a step has a certain
  context, then subsequent steps in the plan
  inherit that context

65
Initial Plan State for the Flat-Tire Problem
On(Tire1)
On(x)
Start
Finish
Flat(Tire1)
Inflated(x)
Inflated(Spare)
(True)
66
Plan after Adding Inflate(Tire1) Step
On(Tire1)
On(Tire1)
Start
Finish
Flat(Tire1)
Inflated(Tire1)
Inflated(Spare)
(True)
Flat(Tire1)
Inflate(Tire1)
Intact(Tire1)
67
Adding a Conditional Link
On(Tire1)
On(Tire1)
Start
Finish
Flat(Tire1)
Inflated(Tire1)
Inflated(Spare)
(Intact(Tire1))
Flat(Tire1)
Inflate(Tire1)
Intact(Tire1)
Intact(Tire1)
(Intact(Tire1))
Check(Tire1)
68
Plan for the Case Where Tire1 Is Not Intact
On(Tire1)
On(Tire1)
Start
Finish
Flat(Tire1)
Inflated(Tire1)
Inflated(Spare)
(Intact(Tire1))
Flat(Tire1)
Inflate(Tire1)
Intact(Tire1)
Intact(Tire1)
(Intact(Tire1))
Check(Tire1)
Finish
On(Spare)
Inflated(Spare)
( Intact(Tire1))
69
Complete Plan for Fixing the Tire
On(Tire1)
On(Tire1)
Start
Finish
Flat(Tire1)
Inflated(Tire1)
Inflated(Spare)
(Intact(Tire1))
Check(Tire1)
Flat(Tire1)
Inflate(Tire1)
Intact(Tire1)
Intact(Tire1)
(Intact(Tire1))
( Intact(Tire1))
Remove(Tire1) PutOn(Spare)
Finish
On(Spare)
Inflated(Spare)
( Intact(Tire1))
( Intact(Tire1))
70
Execution Monitoring

• Detect a potential failure by comparing the
  current preconditions of the plan segment with
  the state description generated from the percept
  sequence
• Preconditions of a plan segment are all those
  preconditions of the steps in the segment that
  are not established by other steps in the segment
• Annotate a plan at each step with the
  preconditions required for successful completion
  of the remaining steps

71
A Simple Replanning Agent (1)

• class ReplanningAgent
• KnowledgeBase kb
• KBMS kbms
• Plan p
• Plan q
• counter t // indicating time
• Planner planner
• public KBAgent()
• kb new KnowledgeBase()
• kbms new KBMS( kb )
• p null
• t 0

72
A Simple Replanning Agent (2)

• public Action run(Percept percept)
• kbms.tell(new
• PerceptSentence(percept, t))
• WorldState current new
• WorldState(kbms, t)
• if( p null )
• Goal g kbms.ask(
• new GoalQuery(t) )
• p planner.makePlan(current, g, kbms)
• q p
• if( p null ) return null

73
A Simple Replanning Agent (3)

• if( !p.preconditions.check(kbms) )
• Plan p
• q.chooseBestContinuation(current)
• p p.append( planner(current,
• p.preconditions(), kbms) )
• q p
• Action action p.first()
• p p.rest()
• kbms.tell( new
• ActionSentence(action,t) )
• t
• return action

74
A Situational Planning Agent in Blocks World
Goal
B
C
D
A
E
F
G
75
Action
76
Initial Plan
On(C,F) Clear(C) Clear(D)
Move(C,D)
Ontable(A) On(B,E) On(C,F) On(D,G) Clear(A) Clear(
C) Clear(D) Clear(B)
Finish
Start
On(C,D) On(D,B)
On(D,G) Clear(D) Clear(B)
Move(D,B)
77
Another Agent Moves D onto B
D
B
C
A
E
F
G
78
New Plan
On(C,F) Clear(C) Clear(D)
Move(C,D)
Ontable(A) On(B,E) On(C,F) On(D,B) Clear(A) Clear(
C) Clear(D) Clear(G)
Finish
Start
On(C,D) On(D,B)
On(D,y) Clear(D) Clear(B)
Move(D,B)
79
After Removing Redundant Step
On(C,F) Clear(C) Clear(D)
Move(C,D)
Ontable(A) On(B,E) On(C,F) On(D,B) Clear(A) Clear(
C) Clear(D) Clear(G)
Finish
Start
On(C,D) On(D,B)
80
The Agent Drops C onto A
D
B
C
A
E
F
G
81
New Plan
Ontable(A) On(B,E) On(C,A) On(D,B) Clear(F) Clear(
C) Clear(D) Clear(G)
Finish
Start
On(C,D) On(D,B)
82
Add a New Step to Deal with Open Condition
On(C,A) Clear(C) Clear(D)
Move(C,D)
Ontable(A) On(B,E) On(C,A) On(D,B) Clear(F) Clear(
C) Clear(D) Clear(G)
Finish
Start
On(C,D) On(D,B)
83
After Move(C,D) Is Executed
Ontable(A) On(B,E) On(C,D) On(D,B) Clear(F) Clear(
C) Clear(A) Clear(G)
Finish
On(C,D) On(D,B)
Start
84
Final State
C
D
B
A
E
F
G