Microscopic Traffic Modeling by Optimal Control and Differential Games

1
Microscopic Traffic Modeling by Optimal Control
and Differential Games

• Using Differential Game Theoryto derive a
  behavior-basedMicroscopic Flow Model

Prof. Dr. Ir. S. P. Hoogendoorn
www.tracingcongestion.tudelft.nl www.pedestrian
s.tudelft.nl
Faculty of Civil Engineering and Geosciences
2
Main contributions

• Developing a generic microscopic theory of
  driving behavior based on subjective predicted
  utility maximization / disutility minimization
• Using theory of differential games to derive
  mathematical microscopic model, including
• Longitudinal driving task (free driving and
  car-following)
• Lateral driving task (lane changing)
• Show model characteristics by simple case example

3
Principle of disutility minimization

• Several authors have proposed describing driver
  task execution as on subjective utility
  optimization problem, where the (dis-) utility
  reflects objectives such as
• Maximize safety and minimize risks
• Maximize travel efficiency
• Minimize lane-changing maneuvers
• Maximize smoothness and comfort
• Minimize stress, inconvenience, fuel consumption,
  etc.
• The importance of each of these objectives will
  vary among individuals, given capabilities of the
  drivers and the possibilities of their vehicles

4
Psychological foundations?

• Considered driving tasks take place on
  operational level and tactical level
  (Michon,1985)
• Both are based on immediate driver environment
• Decisions are to be made in seconds or
  milliseconds
• Experienced has skilled drivers in making
  user-optimal decisions subconsciously
• Description of drivers are optimal controllers
  due to a.o. Minderhoud, Weverinke and to Hogema

5
Behavioral assumptions

• Drivers control actions stem from minimizing
  generalized predicted costs reflecting their
  objectives
• Predicted costs are determined by a.o.
• Not driving at the free driving speed, in the
  desired lane
• Driving too close (or too far) to leader (or
  follower!)
• Acceleration and braking (fuel consumption and
  smoothness)
• Lane changing effort
• Drivers (re-) consider their control decisions at
  time instants tk
• Drivers may anticipate on control actions of
  other drivers
• Drivers have limited prediction capabilities and
  make errors
• Operational control objectives may change over
  time (adaptation)

6
Drivers as optimal controllers
Observation
Trafficsystem
kk1
Stateestimation
Prediction
Candidatecontrol
Predictedutility / cost
7
State and control definition

• In this model, state z(t) summarizes system
  state, including
• Longitudinal positions xj(t) of vehicles j
• Speeds vj(t) of vehicles j
• Lateral positions yj(t) of vehicles j
• Controls u(t) of driver are
• Acceleration ai(t)
• Lane change decision Di(t) (-1,0,1)
• Driver i makes assumptions regarding controls of
  other drivers j ? i (acceleration and lane
  change decisions)
• Driver state prediction model

8
Control objective

• Driving objectives can be formalized into
  objective function J
• Cost discount factor ? ? 0 describes importance
  of future cost
• Running cost L(x(s),u(s)) denotes the
  contribution of driver situation at time s ? tk
  to the total predicted cost J
• Driver aims to find optimal control

9
Control objective

• Example specification of running cost L assuming
  linear relation
• Smoothness factor
• Quickness factor
• Distance to leader(s) factor
• Easy to include other cost components, including
  those reflecting satisficing driver strategies

10
Derivation optimal control law

• Two well-known (and strongly related) techniques
  exist
• Dynamic programming
• Pontryagins minimum principle
• We use the latter to derive the optimal
  acceleration (assuming no lane changes) and the
  former to determine whether lane change is
  beneficial
• Both lean on so-called Hamilton function H
• where ? are the co-states or marginal costs of
  the state z

11
Optimal control law for single lane

• Stationary conditions used to determine optimal
  acceleration
• So-called co-state equations allow for
  determination of co-states
• Some relatively simple mathematics then yield the
  optimal acceleration for driver i

12
Intermezzo walker models

• Same approach was used to derive walker model
  NOMAD
• Resulting model is similar to the social forces
  model of Helbing and is the basis of the NOMAD
  pedestrian simulation software
• Software will be available soon on the TU Delft
  pedestrian website (www.pedestrians.tudelft.nl)

13
(No Transcript)
14
Optimal control for lane changing

• Dynamic programming entails solving so-called
  Hamilton-Jacobi-Bellman equation for infinite
  horizon discounted cost problem
• Let W(z) denote minimum value of objective
  function (so-called value function), i.e. upon
  applying optimal control law u, W satisfies the
  HJB equation
• Relation between costates and value function
• allows us to determine W(z) from H

15
Derivation of optimal control law

• Predicted cost can be computed per lane to see if
  a lane change is beneficial or not
• Lane change occurs when predicted cost of current
  lane is larger that predicted cost on target lane
  switching cost ?
• Lane changing depends on
• Expected acceleration / deceleration on target
  lane
• Expected changes in the proximity costs

16
So, what is so generic?

• Many known car-following models can be formalized
  as an optimal control model
• E.g. model of Bexelius (multi-anticipatory using
  two leaders) can be found using
• For these models, lane-changing criteria can be
  derived which are consistent with this model
• Note strong relation with the MOBIL model of
  Kesting, Treiber and Helbing, but lane changing
  criteria are different

17
Example application

• Three lane motorway
• On each lane, platoon leader with speed 16, 24
  and 32 m/s
• Example application shows plausible behavior of
  drivers modelled according to control formulation
• In particular
• Considered driver chooses to change lanes because
  expected costs are less on target lane than on
  current lane

18
Example application
19
Model calibration and benchmarking

• Calibration of model can be achieved either by
  considering macroscopic properties or by using
  microscopic trajectory data (such as those
  collected from the helicopter)
• See www.tracingcongestion.tudelft.nl

20
Model extensions

• Several simple model extensions can be easily
  included, in particular to describe driving
  behavior near discontinuities (merges, diverges
  and weaving areas)
• Anticipated reaction of drivers to control action
  of driver i
• Cooperative driving (common objective function)
• Inter-driver variability (differences between
  drivers) and intra-driver variability (adaptive
  driving behavior, e.g. by slowly changing cost
  weights allowing prediction of capacity funnel,
  capacity drop, etc.)
• Empirical investigations show importance of
  these!
• Inclusion of explicit reaction times, errors in
  observing, state estimation, predicting, decision
  making, etc.

21
(No Transcript)