Hybrid Systems Modeling, Analysis, Control

1
Hybrid Systems Modeling, Analysis, Control

• Datta Godbole, John Lygeros, Claire Tomlin,
  Gerardo Lafferiere, George Pappas, John Koo
• Jianghai Hu, Rene Vidal, Shawn Shaffert, Jun
  Zhang,
• Slobodan Simic, Kalle Johansson, Maria Prandini
• David Shim, Jin Kim, Omid Shakernia, Cedric Ma,
  Judy Liebmann and Ben Horowitz
• (with the interference of) Shankar Sastry

2
What Are Hybrid Systems?

• Dynamical systems with interacting continuous and
  discrete dynamics

3
Why Hybrid Systems?

• Modeling abstraction of
• Continuous systems with phased operation (e.g.
  walking robots, mechanical systems with
  collisions, circuits with diodes)
• Continuous systems controlled by discrete inputs
  (e.g. switches, valves, digital computers)
• Coordinating processes (multi-agent systems)
• Important in applications
• Hardware verification/CAD, real time software
• Manufacturing, chemical process control,
• communication networks, multimedia
• Large scale, multi-agent systems
• Automated Highway Systems (AHS)
• Air Traffic Management Systems (ATM)
• Uninhabited Aerial Vehicles (UAV), Power Networks

4
Control Challenges

• Large number of semiautonomous agents
• Coordinate to
• Make efficient use of common resource
• Achieve a common goal
• Individual agents have various modes of operation
  
• Agents optimize locally, coordinate to resolve
  conflicts
• System architecture is hierarchical and
  distributed
• Safety critical systems
• Challenge Develop models, analysis, and
  synthesis tools for designing and verifying the
  safety of multi-agent systems

5
Proposed Framework
6
Different Approaches

7
Research Issues

• Modeling Simulation
• Control classify discrete phenomena, existence
  and uniqueness of execution, Zeno Branicky,
  Brockett, van der Schaft, Astrom
• Computer Science composition and abstraction
  operations Alur-Henzinger, Lynch, Sifakis,
  Sztipanovits,Varaiya
• Analysis Verification
• Control stability, Lyapunov techniques
  Antsaklis, Branicky, Michel, LMI techniques
  Johansson-Rantzer, optimal control Branicky,
  Sussmann, Caines
• Computer Science Algorithmic Alur-Henzinger,
  Sifakis, Pappas-Lafferrier-Sastry or deductive
  methods Lynch, Manna, Pnuelli
• Controller Synthesis
• Control optimal control Branicky-Mitter,
  Bensoussan-Menaldi, hierarchical control
  Caines, Pappas-Sastry, supervisory control
  Lemmon-Antsaklis, model predictive techniques
  Morari Bemporad, safety specifications
  Lygeros-Tomlin-Sastry
• Computer Science algorithmic synthesis Maler,
  Pnueli, Asarin, Wong-Toi
• Observability and Diagnosability
• Control observersBemporad, Koutsoukos, Vidal
• Computer Science Biswas, Karsai, Zhao

8
Talk Outline

• Motivating Applications
• Automated Highway Systems
• Air Traffic Management Systems
• Modeling
• Basic formalism
• Existence uniqueness
• Controller synthesis
• Safety specifications
• Applications to ATM and AHS
• Analysis
• Bisimulations of transition systems
• O-minimal and linear hybrid systems
• Conclusions Future Research

9
Automated Highway Systems

• Goal
• Increase highway throughput
• Same highway infrastructure
• Same level of safety
• Same level of passenger comfort
• Introduce automation
• Partial driver assistance, intelligent cruise
  control, warning system
• Full individual vehicles, mixed traffic,
  platooning
• Complex problem
• Technological issues (is it possible with current
  technology)
• Social/Political issues (insurance and legal
  issues, equality)

10
Safety-Throughput Tradeoff

• Contradictory demands
• Safety vehicles far and moving slowly
• Throughput vehicles close and moving fast
• Proposed compromise
• Allow low relative velocity collisions
• In emergency situations
• Two possible safe arrangements
• Large spacing (leader mode)
• Small spacing (follower mode)
• Platooning concept

11
Control Hierarchy

• Implementation requires automatic control
• Control hierarchy proposed in Varaiya 93
• Regulation layer braking, acceleration and
  steering
• Coordination layer maneuvers implemented by
  communication protocols
• Link layer flow control, lane assignment
• Network layer routing
• Hybrid phenomena appear throughout
• Switching controllers for regulation
• Switching between maneuvers
• Lane and maneuver assignment
• Degraded modes of operation

12
Air Traffic Management Systems

• Studied by NEXTOR and NASA
• Increased demand for air travel
• Higher aircraft density/operator workload
• Severe degradation in adverse conditions
• High business volume
• Technological advances Guidance, Navigation
  Control
• GPS, advanced avionics, on-board electronics
• Communication capabilities
• Air Traffic Controller (ATC) computation
  capabilities
• Greater demand and possibilities for automation
• Operator assistance
• Decentralization
• Free flight

13
Automated Platoons on I-15
14
Current ATM System
CENTER B
CENTER A
TRACON
VOR
SUA
20 Centers, 185 TRACONs, 400 Airport Towers Size
of TRACON 30-50 miles radius, 11,000ft Centers/TR
ACONs are subdivided to sectors Approximately
1200 fixed VOR nodes Separation Standards
Inside TRACON 3 miles, 1,000 ft Below 29,000
ft 5 miles, 1,000ft Above 29,000 ft 5
miles, 2,000ft
TRACON
GATES
15
Current ATM System Limitations

• Inefficient Airspace Utilization
• Nondirect, wind independent, nonoptimal routes
• Centralized System Architecture
• Increased controller workload resulting in
  holding patterns
• Obsolete Technology and Communications
• Frequent computer and display failures
• Limitations amplified in oceanic airspace
• Separation standards in oceanic airspace are very
  conservative

16
A Future ATM Concept
CENTER B
CENTER A
TRACON
ALERT ZONE
PROTECTED ZONE

• Free Flight from TRACON to TRACON
• Increases airspace utilization
• Tools for optimizing TRACON capacity
• Increases terminal area capacity and throughput
• Decentralized Conflict Prediction Resolution
• Reduces controller workload and increases safety

TRACON
17
Hybrid Systems in ATM

• Automation requires interaction between
• Hardware (aircraft, communication devices,
  sensors, computers)
• Software (communication protocols, autopilots)
• Operators (pilots, air traffic controllers,
  airline dispatchers)
• Interaction is hybrid
• Mode switching at the autopilot level
• Coordination for conflict resolution
• Scheduling at the ATC level
• Degraded operation
• Requirement for formal design and analysis
  techniques
• Safety critical system
• Large scale system

18
Control Hierarchy

• Flight Management System (FMS)
• Regulation trajectory tracking
• Trajectory planning
• Tactical planning
• Strategic planning
• Decentralized conflict detection
  and resolution
• Coordination, through
  communication protocols
• Air Traffic Control
• Scheduling
• Global conflict detection and resolution

19
Hybrid Research Issues

• Hierarchy design
• FMS level
• Mode switching
• Aerodynamic envelope protection
• Strategic level
• Design of conflict resolution maneuvers
• Implementation by communication protocols
• ATC level
• Scheduling algorithms (e.g. for take-offs and
  landings)
• Global conflict resolution algorithms
• Software verification
• Probabilistic analysis and degraded modes of
  operation

20
Other Applications

• Uninhabited Aerial Vehicles (UAV)
• Automated aerial vehicles (airplanes and/or
  helicopters)
• Coordinate for search and rescue, or seek and
  destroy missions
• Control hierarchy similar to ATM
• Mode switching, discrete coordination, flight
  envelope protection
• Power Electronic Building Blocks (PEBB)
• Power electronics, with sensing, control,
  communication
• Improve power network efficiency and reliability
  for utilities, hybrid electric vehicle, universal
  power ships
• Control hierarchy load balancing/shedding,
  network stabilization, pulse width modulation
• Hybrid phenomena modulation, input
  characteristic switching, scheduling

21
UAV Laboratory Configuration
22
Motivation

• Goal
• Design a multi-agent multi-modal control system
  for Unmanned Aerial Vehicles (UAVs)
• Intelligent coordination among agents
• Rapid adaptation to changing environments
• Interaction of models of operation
• Guarantee
• Safety
• Performance
• Fault tolerance
• Mission completion

Conflict Resolution Collision Avoidance Envelope
Protection
Tracking Error Fuel Consumption Response Time
Sensor Failure Actuator Failure
Path Following Object Searching Pursuit-Evasion
23
Hierarchical Hybrid Systems

• Envelope Protecting Mode
• Normal Flight Mode

Tactical Planner
Safety Invariant ?? Liveness Reachability
24
The UAV Aerobot Club at Berkeley

• Architecture for multi-level rotorcraft UAVs
  1996- to date
• Pursuit-evasion games 2000- to date
• Landing autonomously using vision on pitching
  decks 2001- to date
• Multi-target tracking 2001- to date
• Formation flying and formation change 2002

25
Flight Control System Experiments
Landing scenario with SAS (Dec 1999)
PositionHeading Lock (Dec 1999)
PositionHeading Lock (May 2000)
Attitude control with mu-syn (July 2000)
26
Pursuit-Evasion Game Experiment using Simulink

• PEG with four UGVs
• Global-Max pursuit policy
• Simulated camera view
• (radius 7.5m with 50degree conic view)
• Pursuer0.3m/s Evader0.5m/s MAX

27
Demo of RL controller doing acrobatic maneuvers
(Spring 02)
28
Set of Manuevers

• Any variation of the following maneuvers in x-y
  direction
• Any combination of the following maneuvers

Nose-in During circling
Heading kept the same
29
Video tape of Maneuvers
30
Talk Outline

• Motivating Applications
• Automated Highway Systems
• Air Traffic Management Systems
• Modeling
• Basic formalism
• Existence uniqueness
• Controller synthesis
• Safety specifications
• Applications to ATM and AHS
• Analysis
• Bisimulations of transition systems
• O-minimal and linear hybrid systems
• Conclusions Future Research

31
Hybrid Automata

• Hybrid Automaton
• State space
• Input space
• Initial states
• Vector field
• Invariant set
• Transition relation
• Remarks
• countable,
• State
• Can add outputs, etc. (not needed here)

32
Executions

• Hybrid time trajectory,
  , finite or infinite with
• Execution with
  and
• Initial Condition
• Discrete Evolution
• Continuous Evolution over ,
  continuous, piecewise continuous,
  and
• Remarks
• x, v not function, multiple transitions possible
• q constant along continuous evolution
• Can study existence uniqueness

33
Talk Outline

• Motivating Applications
• Automated Highway Systems
• Air Traffic Management Systems
• Modeling
• Basic formalism
• Existence uniqueness
• Controller synthesis
• Safety specifications
• Applications to ATM and AHS
• Analysis
• Bisimulations of transition systems
• O-minimal and linear hybrid systems
• Conclusions Future Research

34
Controller Synthesis Example

• 2D conflict resolution
• Ensure aircraft remain more than 5nmi from each
  other

35
Hybrid Automaton Specification

• Discrete input variable determines maneuver
  initiation
• Safety specification

36
More Abstractly ...

• Consider plant hybrid automaton, inputs
  partitioned to
• Controls, U
• Disturbances, D
• Controls specified by us
• Disturbances specified by the environment
• Unmodeled dynamics
• Noise, reference signals
• Actions of other agents
• Memoryless controller is a map
• The closed loop executions are

37
Controller Synthesis Problem

• Given H and find g such that
• A set is controlled invariant if
  there exists a controller such that all
  executions starting in remain in
• Proposition The synthesis problem can be solved
  iff there exists a unique maximal controlled
  invariant set with
• Seek maximal controlled invariant sets (least
  restrictive) controllers that render them
  invariant
• Proposed solution treat the synthesis problem as
  a non-cooperative game between the control and
  the disturbance

38
Gaming Synthesis Procedure

• Discrete Systems games on graphs, Bellman
  equation
• Continuous Systems pursuit-evasion games, Isaacs
  PDE
• Hybrid Systems for define
• states that can be
  forced to jump to for some
• states that may
  jump out of for some
• states that
  whatever does can be continuously driven to
  avoiding by
• Initialization
• while do
• end

39
Algorithm Interpretation
X

Proposition If the algorithm terminates, the
fixed point is the maximal controlled invariant
subset of F
40
Computation

• One needs to compute ,
  and
• Computation of the Pre is straight forward
  (conceptually!) invert the transition relation
• Computation of Reach through a pair of coupled
  Hamilton-Jacobi partial differential equations
• Semi-decidable if Pre, Reach are computable
• Decidable if hybrid automata are rectangular,
  initialized.

41
Application Control of Automated Highway Systems

• Design of vehicle controllers performance
  estimation
• Two concepts
• platooning individual vehicles

Join
Speed, vehicle following
Lane Change
Platoon Following
Split
Exit
42
Vehicle Following Lane Changing

• Control actions (vehicle i)
• -- braking, lane change
• Disturbances (generated by neighboring vehicles)
• -- deceleration of the preceding vehicle
• -- preceding vehicle colliding with the
  vehicle ahead of it
• -- lane change resulting in a different
  preceding vehicles
• -- appearance of an obstacle in front
• Operational conditions
• state of vehicle i with respect to traffic

i
i-1
i-2
j
43
Game Theoretic Formulation

• Requirements
• Safety (no collision)
• Passenger Comfort
• Efficiency
• trajectory tracking (depends on the maneuver)
• Safe controller (J1) Solve a two-person zero-sum
  game
• saddle solution (u1,d1) given by
• Both vehicles i and i-1 applying maximum braking
• Both collisions occur at T0 and with maximum
  impact

44
Safe Vehicle Following Controller

• Partition the state space into safe unsafe sets

• Design comfortable and
• efficient controllers in
• the interior
• IEEE TVT 11/94

• Safe set characterization
• also provides sufficient
• conditions for lane change
• CDC 97, CDC98

45
Automated Highway System Safety

• Theorem 1 (Individual vehicle based AHS)
• An individual vehicle based AHS can be designed
  to produce no inter-vehicle collisions,
• moreover disturbances attenuate along the vehicle
  string.
• Theorem 2 (Platoon based AHS)
• Assuming that platoon follower operation does not
  result in any collisions even with a possible
  inter-platoon collision during join/split, a
  platoon based AHS can be safe under low relative
  velocity collision criterion.
• References
• Lygeros, Godbole, Sastry, IEEE TAC, April 1998
• Godbole, Lygeros, IEEE TVT, Nov. 1994

46
Example Aircraft Collision Avoidance

• Two identical aircraft at fixed altitude speed

y
v
y
u
x
v
d
47
Continuous Reachable Set
Mitchell, Bayen, Tomlin 2001 Tomlin, Lygeros,
Sastry 2000
48
Fast Wavefront Approximation Methods (Tomlin,
Mitchell)
49
Visualization of Unsafe SetMitchell-Tomlin
50
Talk Outline

• Motivating Applications
• Automated Highway Systems
• Air Traffic Management Systems
• Modeling
• Basic formalism
• Existence uniqueness
• Controller synthesis
• Safety specifications
• Applications to ATM and AHS
• Analysis and Computability
• Bisimulations of transition systems
• O-minimal and linear hybrid systems
• Conclusions Future Research

51
Transition Systems

• Transition System
• Define for

• Given equivalence relation
  define

• A block is a union of equivalence classes

52
Bisimulations of Transition Systems

• A partition is a bisimulation iff
• are blocks
• For all and all blocks
  is a block

• Alternatively, for

• Why are bisimulations important?

53
Bisimulation Algorithm

• initialize
• while such that
• define
• refine

• If algorithm terminates, we obtain a finite
  bisimulation

54
Transitions of Hybrid Systems

• Transitions of hybrid systems are concatenations
  of
• Discrete transitions
• Continuous transitions
• Because of initialized transitions
• If invariants, guards, resets are blocks, then
  no refinement is necessary due to discrete
  transitions

55
Bisimulation Algorithm

• Refinement process is therefore decoupled
• Consider for each discrete state the finite
  collection of sets
• Let be a partition compatible with

• Initialize
• for each
• while such that
• define
• refine
• end while end for

• Algorithm must terminate for each discrete
  location

56
Computability Finitiness

• Decidability requires the bisimulation algorithm
  to
• Terminate in finite number of steps and
• Be computable
• For the bisimulation algorithm to be computable
  we need to
• Represent sets symbollically,
• Perform boolean combinations on sets
• Check emptyness of a set,
• Compute Pre(P) of a set P
• Class of sets and vector fields must be
  topologically simple
• Set operations must not produce pathological sets
• Sets must have desirable finiteness properties

57
A simple example

• Spiraling, linear vector field
• Refinement process does not terminate
• Intersection generated set with infinite number
  of components

58
Mathematical Logic

• Every theory of the reals has an associated
  language
• Decidable theories
• Every formula is equivalent to a quantifier free
  formula
• Quantifier free formulas can be decided
• Quanitifier elimination
• Computational tools (REDLOG, QEPCAD)

59
O-Minimal Theories

• A definable set is
• A theory of the reals is called o-minimal if
  every definable subset of the reals is a finite
  union of points and intervals
• Example for
  polynomial
• Recent o-minimal theories

Semilinear sets
Semialgebraic sets
Exponential flows
Bounded Subanalytic sets
Spirals ?
60
O-Minimal Hybrid Systems

• A hybrid system H is said to be o-minimal if
• the continuous state lives in
• For each discrete state, the flow of the vector
  field is complete
• For each discrete state, all relevant sets and
  the flow of the vector field are definable in the
  same o-minimal theory
• Main Theorem
• Every o-minimal hybrid system admits a finite
  bisimulation.
• Bisimulation alg. terminates for o-minimal hybrid
  systems
• Various corollaries for each o-minimal theory

61
O-Minimal Hybrid Systems

• Consider hybrid
  systems where
• All relevant sets are polyhedral
• All vector fields have linear flows
• Then the bisimulation algorithm terminates

• Consider hybrid
  systems where
• All relevant sets are semialgebraic
• All vector fields have polynomial flows
• Then the bisimulation algorithm terminates

62
O-Minimal Hybrid Systems

• Consider
  hybrid systems where
• All relevant sets are subanalytic
• Vector fields are linear with purely imaginary
  eigenvalues
• Then the bisimulation algorithm terminates

• 
  Consider hybrid systems where
• All relevant sets are semialgebraic
• Vector fields are linear with real eigenvalues
• Then the bisimulation algorithm terminates

63
O-Minimal Hybrid Systems

• 
  Consider hybrid systems where
• All relevant sets are subanalytic
• Vector fields are linear with real or purely
  imaginary eigenvalues
• Then the bisimulation algorithm terminates

• New o-minimal theories result in new finiteness
  results
• Can we find constructive subclasses?
• Must remain within decidable theory
• Sets must be semialgebraic
• Need to perfrom reachability computations
• Reals with exp. does not have quantifier
  elimination

64
Linear Hybrid Systems

• A hybrid system H is said to be linear if
• the continuous state lives in
• For each discrete state, all relevant sets are
  semialgebraic
• For each discrete state, the vector field is of
  the form
• where matrix has
  rational entries
• Let . Then we can
  express
• Focus on the subformula

65
Nilpotent Linear Systems

• Nilpotent matrices
• Let be a linear vector
  field, rational, nilpotent.
• Then is definable in the
  decidable theory of reals
• Example

66
Diagonalizable, Rational Eigenvalues

• The flow of is
• Consider the formula
• Let and consider the equivalent
  formula
• Consider . Then
• Then for each component of we have

67
Diagonalizable, Rational Eigenvalues

• The next step rescales time to get integer
  exponents
• The substitution results in the
  equivalent formula
• The last step eliminates negative powers
• The above sequence results in the following

68
Diagonalizable, Rational Eigenvalues

• h

Let be a linear vector
field, rational, diagonalizable with
rational eigenvalues. Then is
definable in the decidable theory of
reals Example
69
Diagonalizable, Imaginary Eigenvalues

• Procedure is conceptually similar if is
  diagonalizable with purely imaginary, rational
  eigenvalues
• Equivalence is obtained by
• Suffices to compute over a period
• Composing all the constructive results together
  gives in

Let be a linear vector
field, rational, diagonalizable with purely
imaginary rational eigenvalues. Then
is definable in the decidable theory of reals
70
Semidecidable Linear Hybrid Systems

• Let H be a linear hybrid system H where for each
  discrete
• location the vector field is of the form F(x)Ax
  where
• A is rational and nilpotent
• A is rational, diagonalizable, with rational
  eigenvalues
• A is rational, diagonalizable, with purely
  imaginary, rational eigenvalues
• Then the reachability problem for H is
  semidecidable.
• Above result also holds if discrete transitions
  are not necessarily initialized but computable

71
Decidable Linear Hybrid Systems

• Let H be a linear hybrid system H where for each
  discrete
• location the vector field is of the form F(x)Ax
  where
• A is rational and nilpotent
• A is rational, diagonalizable, with rational
  eigenvalues
• A is rational, diagonalizable, with purely
  imaginary, rational eigenvalues
• Then the reachability problem for H is
  decidable.

72
Linear Hybrid Systems with Inputs

• Let H be a linear hybrid system H where for each
  discrete
• location, the dynamics are
  where A,B are
• rational matrices and one of the following holds
  
• A is nilpotent, and
• A is diagonalizable with rational eigenvalues,
  and
• A is diagonalizable with purely imaginary
  eigenvalues and
• Then the reachability problem for H is
  decidable.

73
Linear DTS (compare with Morari Bemporad)

• X ?n, U uEu??, D dGd??, f
  AxBuCd,
• F xMx??.
• Pre(Wl) x ?l(x)
• ?l(x) ?u ?d Mlx??lcEu???
• (Gdgt?)?(MlAxMlBuMl
  Cd ??l)
• Implementation
• Quantifier Elimination on d Linear Programming
• Quantifier Elimination on u Linear Algebra
• Emptiness Linear Programming
• Redundancy Linear Programming

74
Implementation for Linear DTS

• Q.E. on d (Gdgt?)?(MlAxMlBuMlCd ? ?l) ?
  MlAxMlBumaxMlCd Gd????l)
• Q.E. on u Eu?? ? MlAxMlBu?(MlC) ? ?l) ?
  ?l(MlAx?(MlC)) ? ?l?l where ?lMlB0,
  ?lE0, ?l??0, ?l?0
• Emptiness mint Mx ? ?(1...1)Tt gt
  0 where M Ml ?lMlA and ? ?l
  ?l(?l -?(MlC))
• Redundancy maxmiT x Mx ? ? ? ?i

75
Decidability Results for Algorithm

• The controlled invariant set calculation problem
  is
• Semi-decidable in general.
• Decidable when F is a rectangle, and A,b is
  in controllable canonical form for single input
  single disturbance.
• Extensions
• Hybrid systems with continuous state evolving
  according to discrete time dynamics difficulties
  arise because sets may not be convex or
  connected.
• There are other classes of decidable systems
  which need to be identified.

76
(No Transcript)
77
Talk Outline

• Motivating Applications
• Automated Highway Systems
• Air Traffic Management Systems
• Modeling
• Basic formalism
• Existence uniqueness
• Controller synthesis
• Safety specifications
• Applications to ATM and AHS
• Analysis
• Bisimulations of transition systems
• O-minimal and linear hybrid systems
• Conclusions Future Research

78
Summary

• Methodology
• Modeling Framework
• Game theoretic approach to controller synthesis
• Linear hybrid systems and computability
• Applications
• Synthesis of safe conflict resolution maneuvers
• Safe controllers for automated highways
• Verification of avionic software (CTAS, TCAS)
• Flight Envelope Protection
• Flight Mode Switching

79
Newer Research

• Modeling
• Robustness, Zeno (Zhang, Simic, Johansson)
• Simulation, on-line event detection (Johannson,
  Ames)
• Control
• Extension to more general properties (liveness,
  stability) (Koo)
• Links to viability theory and viscosity solutions
  (Lygeros, Tomlin, Mitchell, Bayen)
• Numerical solution of PDEs (Tomlin, Mitchell)
• Analysis
• Develop (exact/approximate) reachability tools
  (Vidal, Shaffert)
• Complexity analysis (Pappas, Kumar)
• Probabilistic Hybrid Systems (Hu)
• Observability of Hybrid Systems (Vidal)