Counterexampleguided Planning

1
Counterexample-guided Planning
Krishnendu Chatterjee Tom Henzinger
UC Berkeley UC Berkeley and EPFL Ranjit
Jhala Rupak Majumdar UC San Diego
UC Los Angeles
2
Game Models

• Model of interaction
• between agents
• Deterministic games
• with 2 players
• -set of (player 1) states S1
• -set of (player 2) states S2
• -set of actions A
• -transition relation
• ? S1 A S2 A! S1 S2

• Two Dual Problems
• Safety Control
• Given game graph, and set of bad states, check
  there is a player 1 strategy such that no matter
  what player 2 plays, the outcome does not get
  into bad states
• Conformal Planning
• Given game graph, and set of goal states, find a
  player 1 strategy (plan) such that no matter what
  player 2 plays, the outcome reaches a goal state

3
Technique Abstract Controllability

• CONTROLLABILITY
• Start with goal states
• Iterate CPre operator until fixpoint
• s2 Cpre(X) s2 S1 Æ 9 a. ?(s,a)2 X
• or s2 S2 Æ 8 a.?(s,a)2 X

• ABSTRACTION
• Partition states
• Perform controllability on reduced graph

Large State Space Expensive!
Small State Space Fast!
4
Abstractions of Games
Requirement If player 1 wins the abstract game,
then she wins also the concrete game.
8 abstraction (underapproximation)
-player 1 can be weakened (fewer moves)
-player 2 can be strengthened (more moves)
9 abstraction (overapproximation)
5
What if Abstraction is too Coarse?
Have to refine abstract plan until it becomes
concretely feasible Refinement Finer partition
on states
Abstract plan may not be concretely realizable.
No plan in this abstraction implies no concrete
plan
6
Concretize Error Trace
Focus(G,A') ? feasible plan

Focus(G,A') refine abstraction!

• In general, plans are trees (winning strategies
  of player 1)
• Can implement Focus symbolically on trees

7
Refine Abstraction (Finer Partition)
G
A"A\A'
B"B\B'
I
B' Focus(B,I)
A' Focus(A,B')
Shatter(B,E) B',B\B'

• Run the controllability algorithm on the refined
  abstraction
• Until either feasible plan is found, or there is
  no feasible plan

8
The Abstract-Check-Refine Loop
Seed abstraction
No plan possible
Abstract Controllability
Abstract plan
Concrete plan
Concretize Plan
Refine Abstraction
Not feasible
9
Perfect Information Stochastic Game Models
Model 2 players Randomization -set of player
1states S1 -set of player 2 states S2 -set of
random states Sr S S1 S2 Sr -set of
actions A -transition relation ? (S1 A ! S)
(S2 A! S) (Sr ! P(S)) -reward function r
S ! R If S2 , this is an MDP If Sr it is a
deterministic game
Winning Conditions 1.Discounted Reward 2.
Average Reward
Probability Distribution
10
Counterexample Refinement

• Observation Discounted and average reward games
  have pure, memoryless winning strategies
• i.e., Pure and memoryless plans can be found
• Can apply counterexample guided refinement on
  abstract plans
• Focus1 divides states according to available
  moves
• Focus2 divides states according to available
  rewards
• Details in the paper

11
Conclusions
Counterexample-guided abstraction refinement is a
powerful technique from verification It can be
applied uniformly and fully symbolically to
classical (reachability), conformal
(controllability), stochastic (MDP), and robust
stochastic (perfect information game) planning
problems
For non-probabilistic cases, can implement
algorithm using BDD-based reachability and SAT
based refinement http//www.cs.ucla.edu/rupak/Pow
erpoint/UAI05.ppt