Controller Synthesis with Budget Constraints

1
Controller Synthesis withBudget Constraints
Krishnendu Chatterjee Rupak Majumdar
Thomas A. Henzinger
UC Santa Cruz
UC Los Angeles
EPFL
2
The Control Problem

• In the finite-state, discrete setting

Map history to action
Controller
Strategy
Actuate
Action to change state
Observe
Current state
Plant
Finite State Machine
What happens when observations have costs?
3
Control as a Game

• Control Problem
• Given a plant, does the controller
• have a strategy such that
• no matter how the plant behaves,
• the system satisfies the control objective
• 2-person game on graphs
• - Set of states
• - Controller chooses action at each state
• - Plant resolves non-determinism to get the next
  state
• (Infinite) Sequence of states determines a play
• Controller wins if play satisfies the control
  objective

4
The Model

• Game with n-variables
• Plant has n Boolean variables (bits)
• State Valuation to the bits
• Actions Finite alphabet of control actions
• Transitions Given state and action, return a set
  of possible next states
• Intent Controller chooses action, plant resolves
  non-determinism

5
Example
a
b
b
a
a
b
b
a
Win
Lose
6
Control Objectives

• Specified as functions from plays to payoffs
• - ?-regular languages (or a suitable temporal
  logic) or more generally as Borel sets Payoff is
  1 if play is in the set, 0 o/w
• - quantitative objectives such as long-run
  average cost

7
?-regular Objectives

• Robust class of interesting specifications

Safety Stay forever in good states
Reachability Eventually reach good states
Canonical representation for omega regular
properties
8
Example
a
b
b
a
a
b
b
a
Win
Lose
9
Example Winning Strategies
a
b
a
b
b
b
a
a
b
a
a
b
b
b
a
a
Win
Lose
Win
Lose
10
Solving Games

• Well-studied theory of ?-regular games
• Buchi-Landweber,Pnueli-Rosner,Emerson-Jutla
• Model assumes controller has exact knowledge of
  the entire state of the plant
• This may not hold in practice
• Because part of the model is unobservable
• (games of partial information)
• Or because of budget constraints in the
  implementation This talk

11
Partial Information

• Can only observe a subset of bits
• only have particular sensors
• Observation-based strategy if two plays agree on
  observations, the strategy plays the same action

12
Example
a
b
b
a
a
b
b
a
Win
Lose
13
Example Strategies
a
b
a
b
b
b
a
a
b
a
a
b
b
b
a
a
Win
Lose
Win
Lose
No longer winning
Winning observation-based strategy
14
Solving Partial Information Games

• Basic Idea Subset construction
• Reif84,ChatterjeeDoyenHenzingerRaskin
• Controller maintains set of possible states of
  the system
• Makes sure strategy enforces objective for all
  states in current set
• For any ?-regular game, there is an
  observation-based winning strategy iff
• there is a winning strategy in the subset
  game
• Similar result for quantitative payoffs

15
Budget Constraints
Controller
Actuate
Observe
Plant

• Budgets arise due to
• Time required to sense
• Network bandwidth
• Power required to sense

16
Two Problems

• Budget constrained synthesis
• Budget optimization (design)

17
Budget Constrained Synthesis

• Given
• A game
• A cost function mapping bits to costs
• A budget B
• Construct a controller for a parity objective
  with budget B
• At each round, the controller can ask for a set
  of bits whose combined cost is at most B
• Unlike partial information, the bits can change
  from round to round

18
Example
Cost 1 to know color Cost 2 to know shape Budget
1
a
b
b
a
a
b
b
a
Win
Lose
19
Special Case Partial Information

• Suppose you can only observe bits 1..k
• Set costs 1 for observing bits 1..k,
• Cost ? for bits k1..n
• Set budget k

20
Example 2
Cost ? to know color Cost 1 to know shape Budget
1
a
b
b
a
a
b
b
a
Win
Lose
21
Main Reduction

• Given a budget constrained synthesis problem (G,
  Cost, B),
• there is a game of partial information G such
  that
• the controller has an observation-based strategy
  in G
• iff
• there is a budget constrained strategy in G

22
One Step
State (v, Set) Observe vSet
Make sure Cost of sensing the bits in Set is
less than B
(10011, 1,2)
a
State v v --a ? v Observe ?
11011 ?
1,4,5
State (v, Set) Observe vSet
(11011, 1,4,5)
Plus, some massaging of the parity objective
23
From Budgets to Subsets

• Given game G, budget B
• States of the partial information game
• (state of G, subset of 1,..,n such that
• total cost of sensing B)
• Copy of states of G
• Actions
• Actions of G and subsets of 1,..,n
• Observables
• (v,I) the valuations in set I
• v new observation ?

24
Solving Budget Constrained Synthesis

• Reduce to partial information game
• Solve a parity game by constructing the subset
  game
• Can transfer the winning strategy back to a
  budget-constrained strategy
• EXPTIME-complete

25
Budget Optimization

• A design problem How much budget to allocate?
• Given
• A game
• A cost function mapping bits to real costs
• Construct a controller that satisfies a parity
  objective while minimizing cost
• At each round, the controller can ask for any set
  of bits, but pays the sensing cost

26
Budget Optimization

• Construct a controller that satisfies a parity
  objective while minimizing cost
• Goal 1 Minimize the maximum cost along the play
• Goal 2 Minimize the long run average cost along
  the play

27
From Budgets to Games

• Given a budget optimization problem (G,cost), can
  construct a partial information game G with
  rewards
• such that
• There is a controller for G with budget B
• iff
• There is an observation-based strategy on G that
  satisfies the parity condition within a budget B

28
Solving Games

• For minimizing long-run average The winning
  condition is a conjunction of parity and long-run
  average payoffs
• Use the algorithm of ChatterjeeHenzingerJurdzinsk
  i
• For minimizing the maximum cost, use a binary
  search over costs, and solve parity games

29
Infinite State

• If an infinite state problem has a bisimulation
  relation of finite index, we can solve the
  budget-constrained synthesis problem on the
  finite quotient
• Discrete time rectangular automata have a
  bisimulation relation of finite index
• Henzinger-Kopke
• The budget-constrained synthesis problem for
  discrete time control for rectangular automata is
  decidable

30
Conclusion

• Budget-constrained synthesis and optimization are
  natural models for control system design
• Budgets capture implementation constraints
  (sensing costs, network bandwidths)
• Old solution techniques apply
• Without additional complexity penalties

31
Thank you

• http//www.cs.ucla.edu/rupak