Optimal Control for Systems with Order 1 State Space Constraints

1
Optimal Control for Systems with Order 1 State
Space Constraints
Part 2 Local Fields of Bang-Bang Extremals

• Heinz Schättler
• Dept. of Electrical and Systems Engineering
• Washington University
• St. Louis, Missouri, USA

Research Training Network Evolution Equations
for Deterministic and Stochastic Systems
Technische Universität Wien, June 2006
2
Example 1 Minimization of the base transit time
in HBTs Rinaldi Schättler
Kroemer, 1985 Suzuki, 1991,2001

• over all Lebesgue measurable functions u,
• subject to
• initial conditions
• terminal conditions
• and state constraints

3
optimal synthesis semiconductor problem with
saturating singular arc
local synthesis near bang-boundary-bang junction
WB
optimal singular arc
local synthesis near singular-boundary-bang
junction
4
Cell-Cycle and Chemotherapy

• Drugs act in various stages of the cell cycle
• Killing agents in G2/M
• (Taxol, Spindle poisons,)
• Blocking agents in S
• (hydroxyurea,)
• Recruiting agent in G0
• (interleukin-3, granulate
• colony stimulation
• factors,)

5
Example 2 A Class of Compartmental Models for
Cancer Chemotherapy with U. Ledzewicz

• over all Lebesgue measurable functions u,
• subject to

where
non-negative
not all zero
6
Example 2 ctd. with State Constraints

• (a) soft modeling using penalty terms
• where
• (b) hard modeling as state space constraint

7
optimal synthesis cancer chemotherapy models
local synthesis near bang-boundary-bang junction
switching surfaces
T
8
Optimal Control Problem

• over all Lebesgue measurable
  functions ,
• subject to
• terminal constraints
• and state constraints
• simplifying assumptions regular geometric
  objects
• single-input, control-linear system (multi-input
  problems are harder)
• regularity assumptions on the data
• terminal constraint and state constraints are
  given by embedded
• submanifolds e.g. has linearly
  independent gradients on

9
equivalent formulation (P)
add the Lagrangian as extra state variable,
and redefine the state as
(P)

• over all Lebesgue measurable functions
• subject to
• terminal constraint
• and state constraints

10
equivalent formulation (Q)
still add time as extra state variable,
and redefine the state as
(Q)

• over all Lebesgue measurable functions
• subject to
• terminal constraint
• and state constraints

11

• formulation (Q) is the cleanest geometrically
• formulation (P) more shows the role played by
  time
• here I prefer (P) since the applications I am
  interested in are problems with fixed terminal
  time T
• trajectories of (Q) graphs of
  trajectories of (P)

flow
12
Control-Invariant Manifolds for Problem (P)

• linearly independent gradients on active
  constraints embedded submanifolds
• control-invariant there exists an admissible
  control which leaves invariant
• relative degree 1 the Lie derivative of ha along
  g doesnt vanish

13
If we define vector fields on -space,
then the Lie derivatives of along these
vector fields are given by
So the unique control that keeps
invariant is given by
14
Example 1 Minimization of Base Transit Time

• Constraints
• control-invariant of relative degree 1
• boundary control
• (in interior of control set)

15
Example 2 Compartmental Models for Chemotherapy

• Constraints
• 2-compartment model
• control-invariant of relative degree 1
• boundary control
• (may not be admissible)

16
Assumptions

• (A) all constraint manifolds are
  control-invariant and of relative degree 1
• (B) the reference extremal is normal

17
Necessary Conditions for Optimality for Problem
(P)

• Suppose is an
  optimal control with corresponding trajectory
  and assume no constraint is active at the
  terminal time.
• If is a normal extremal, then there
  exist an absolutely continuous function
• and non-negative Radon measures
  with support in
• such that

18

• with
• and
• the following conditions hold
• (a) the adjoint equation holds in the form
• with transversality conditions
• and

19

• (b) the Hamiltonian
• is minimized by along ,
• i.e.
• hence

Switching function
20
Absolute Continuity of Boundary Measure

• Proposition Maurer, 1977,79
• Suppose condition (A) is satisfied.
• Let be an -boundary arc defined over
  an open interval and suppose
  the boundary control takes values in the
  interior of the control set.
• Then the Radon measure is absolutely
  continuous with respect to Lebesgue measure on
  with continuous and non-negative Radon-Nikodym
  derivative .

21

• Pf. switching function
• minimum condition - on
• (the boundary control lies in interior of
  control set)
• hence
• integration by parts gives that
• relative degree 1 constraint

22

• denote the Radon-Nikodym derivative of by
  and
• differentiate the switching function to get

where
hence
23
Junction Conditions

• Proposition Maurer, 1977
• Suppose condition (A) is satisfied.
• Let be an entry or exit junction time between
  an interior arc and an -boundary arc and
  suppose the reference control has a limit
  along the interior arc.
• Then the interior arc is transversal to
  if and only if the control is discontinuous
  at in this case the multiplier remains
  continuous at .

24
Pf. Let be the limit of the reference control
along the interior arc
the interior arc is transversal if and only if
along the boundary arc
25
Continuity of at junction times w.l.o.g
consider an entry time
In principle could have a jump
But
suppose
transversal
on constraint
hence
contradiction
26
Construction of a Local Field of Bang-Bang
Extremals around a Boundary Arc
AIM imbed the reference trajectory into a field
of extremals
27
(E-ref) Let be an -boundary arc of a
normal extremal defined over an interval
with corresponding multipliers
and . suppose and are the
entry- and exit-times, respectively, and assume
there exists an so that the control is
constant over the intervals and
.

• W.l.o.g. assume the entry control is
• and the exit control is

28
(E-boundary) for all times in the closed
interval the boundary control that
keeps invariant, , takes
values in the interior of the control set and the
multiplier is positive on ,

• entry and exit are transversal, is
  continuous at
• the normalization of the entry- and
  exit-controls is
• equivalent to
  and

29
embedding of interior portion near exit junction
choose as a sufficiently small
neighborhood of and integrate the
system backward from time from points in
with the same constant control denote the
corresponding trajectories by
30

• in order to construct a local field we need to
  be able to propagate a field from the interval
  further backward
• need the correct values for the multipliers

31
auxiliary problem

• over all Lebesgue measurable functions
• subject to
• and state constraints
  .

where is the value
function for the problem over the interval
,
the set of all admissible controls whose
trajectories with initial condition at time
remain in some neighborhood of
the reference trajectory
32

• essentially,
• the requirement is that the terminal interior
  portion of the reference trajectory can be
  embedded into a local field for the
  (unconstrained) problem in such a way that the
  controls and multipliers depend smoothly on the
  parameters
• classical theory for problems without state
  space constraints applies
• fields of extremal
• method of characteristics

33

• (E-adjoint) the value function is twice
  continuously
• differentiable in the state and the multiplier
  is defined as

define as the solution of the
corresponding adjoint equation
34
switching surface at exit junction

• parameterized switching function
  
• switching surface

35
regular and transversal crossing

• Definition A parameterized family of extremals,
• has a regular and transversal crossing at
  , , if
• for
• is non-singular
• the flows cross
  transversally

36
transversal crossing
locally optimal
transversal fold
not optimal
conjugate points
37
regular and transversal crossing, ctd.
(E-transversal) The family
has a regular and transversal crossing at
38
activation of boundary control

• has a unique local solution

39
local synthesis on constraint

• since Ma is control invariant of order 1 and
  the control takes values in the interior of the
  control set, the boundary arc of the reference
  trajectory can be embedded into a local flow on
  Ma
• flow box theorem from ODE

40
propagation of embedding along ?-arc
recall that is positive on

• on the constraint Ma integrate the system and
  adjoint equations backward using the boundary
  control this defines a parameter dependent
  multiplier ,
• for in a sufficiently small neighborhood of
  in , will be bounded away
  from zero on an interval
• integrating the system and adjoint equations
  backward from points on Ma using the
  entry-control generates extremals and the
  corresponding flow is regular

41
Strong Local Optimality of the Reference
Trajectory

• Theorem Under assumptions (E) there exists a
  neighborhood of the restriction of the
  reference trajectory to the interval
  in
• such that every is optimal for the auxiliary
  problem compared with any other admissible
  trajectory which lies in .
• If the local embedding of boundary arcs can be
  joined with local embeddings of the interior
  arcs, the reference trajectory is a strong local
  minimum.