Outline

1
Outline

• What is controls and feedback? Tim
• History (1 slide)
• Motivation Open-loop vs. Closed-loop
• Examples path-planning (1)
• (feed-forward efferent copy)
• General idea why these are important?
• Stability (1 slide total)
• Performance
• Robustness
• Mathematics and modeling
• ODEs difference equations (3) Mike
• Second-order systems (stability, performance)
• Dynamics, time-evolution
• Population dynamics (predator-prey)
• Neural circuits (e.g. RLC circuits)
• State-space (3) Mike
• What is a state variable? (e.g. acceleration)
• Predator-prey model
• Access to linear algebra tools

• Overview of Tools Mike
• Stability of eigenvalues (Routh-Hurwitz) (1)
• Bode, RL, Nyquist, etc (2)
• Linearization of Nonlinear Systems (1)
• MATLAB (SISOTOOLS) (1)
• Controller Design Tim
• Classical (PID) (1)
• Loop-shaping (1)
• State-space (LQR) (1)
• Optimal control
• Seans work
• Modeling (1)
• Matched filter
• Output feedback (as opposed to state feedback) (1)

2
Feedback and Control Systems, Part II

• More on Modeling
• Finite-state machines
• Markov models for state transitions
• Hybrid systems
• Uncertainty in Systems
• Decision theory
• Perturbation Models
• Controllers and Synthesis
• PID control
• Loop shaping
• Optimal control (LQR)

3
Finite State Machines

• Finite state machines model discrete transitions
  between finite of states
• Represent each configuration of system as a state
• Model transition between states using a graph
• Inputs force transition between states
• Example Traffic light logic

Car arrives on E-W St
Timer expires
Timer expires
Car arrives on N-S St
State Inputs Outputs
current pattern of lights that are on internal
timers presence of car at intersections current
pattern of lights that are on
Slide source RMM, CDS101/110
4
Markov Process Models

• Markov processes can be used to model
  probabilistic transitions between states.
• A stochastic process is Markovian if future
  states, given the present state, depends only on
  the current state.
• The transition probability represents the
  probability of going to state j in the next time
  step if currently in state i.
• The transition probability matrix has as its
  (i,j)th element, Pij.

Greenspan, Ferveur. Annual Review Genetics, 2000.
1
3
n 11 fly pairs
2
Courtesy M.J.Dunlop
5
Markov Process Models

• Markov processes can be used to model
  probabilistic transitions between states.
• A stochastic process is Markovian if future
  states, given the present state, depends only on
  the current state.
• The transition probability represents the
  probability of going to state j in the next time
  step if currently in state i.
• The transition probability matrix has as its
  (i,j)th element, Pij.
• Rows sum to one (thanks to the Law of Total
  Probability)
• Columns give relative rates

6
Other Markov model types

• TYPE II
• Accounts for transition to floor before any other
  cone
• Loses previously-visited cone information (treats
  all cone visits as independent)
• Might require a 2nd-order Markov chain model
  (store current and previous states).

• TYPE III
• Accounts for duration of time spent on cone/floor
• Requires tweaking of time resolution parameter
• Might attribute too much probability to remaining
  on cone/floor

7
Hybrid Systems

• Hybrid systems is a tool for modeling systems
  possessing both discrete and continuous states.
• Discrete States
• Can evolve independently
• Used to model behavior modes
• Continuous States
• Describe dynamical states and observation
  variables
• Control Signals
• Can be continuous or discrete
• Guards
• Govern transitions between modes

8
Uncertainty Modeling

• Real systems have uncertainty
• Errors in modeling
• Random disturbances due the environment
• Noise in measurements
• Gaussian noise
• Ubiquitous thanks to the Central Limit Theorem
• Makes math pretty (e.g. Kalman filter)
• Most common way to model uncertainty
• Possibly unbounded
• Bounded uncertainty
• Physically more realistic
• Worst-case analysis

1s confidence ellipse
Worst-case bound
9
Decision-Making using Statistical Learning

• Classical Decision Theory
• Binary hypotheses
• Neyman-Pearson optimal rule for specified false
  alarm rate
• Assumes all measurements are simultaneously
  present
• Sequential Probability Ratio Test
• Binary hypothesis with grey, indeterminate
  region
• Addresses the need to gather or integrate more
  information before making a decision

10
Decision-Making using Statistical Learning
Sequential Decision-Making

• Sequential integration of data
• Gather more observations, make better decision
• Neyman-Pearson-like optimality Given desired
  false-alarm rate, SPRT gives expected of
  measurements required
• Measurements processed real-time

• Plausible model for biological sensory decision
  systems
• Incorporates temporal notions of integration
• Either neuronal signal level or behavioral level
  modeling

11
Scaring fruit flies

• Examine the jumping escape response
• Decision-making requires integration of sensory
  inputs
• SPRT-like behavior!

Collaboration Gwyneth Card and Tim Chung
12
Decision-making and Neuroscience

• Shadlen and Gold
• Weight of evidence ? signals are accumulated
  over time
• Diffusion-to-barrier ? decisions are made by
  crossing thresholds (see Fig. 88.4)
• Neurobiology of decision-making
• Monkey recordings in visual eye-saccade tasks
• Neurons in LIP (lateral intraparietal area)
• Not purely motor and not purely sensory
• Firing rate in LIP neuron represents a decision
  variable

Reference Shadlen, M.N. Gold, J.I., The
Neurophysiology of Decision-Making as a Window on
Cognition, in The Cognitive Neurosciences, MIT
Press, 2004.
13
Decision-making and Neuroscience

• Reddi and Carpenter
• LATER model
• Linear signal growth, with Gaussian perturbations
  in slope

Reference Reddi, B.A.J. Carpenter, R.H.S.,
The Influence of Urgency on Decision Time,
Nature Neuroscience, 2000 , 3 , 827 830.
14
Models for Uncertainty
Additive uncertainty
Multiplicative uncertainty
Feedback uncertainty
W2
P
D
P
D
W2
P

• Each model describes a class of process dynamics
• Additive
• Multiplicative
• Feedback
• Robust stability conditions given by small gain
  theorem
• Compute transfer function around ? block and
  require that this be lt 1
• (If not, can choose ? with ?? ? 1 to
  destabilize)

Use W2 to shape theunmodeled
dynamics ?? lt 1 in all cases
Slide source RMM, CDS101/110
15
Overview PID control

• Intuition
• Proportional term provides inputs that correct
  for current errors
• Integral term insures steady state error goes to
  zero
• Derivative term provides anticipation of
  upcoming changes
• A bit of history on three term control
• First appeared in 1922 paper by Minorsky
  Directional stability of automatically steered
  bodies under the name three term control
• Also realized that small deviations
  (linearization) could be used to understand the
  (nonlinear) system dynamics under control
• Utility of PID
• PID control is most common feedback structure in
  engineering systems
• For many systems, only need PI or PD (special
  case)
• Many tools for tuning PID loops and designing
  gains (see reading)

Slide source RMM, CDS101/110
16
Overview of PID Feedback

• Different systems require different controllers
• Proportional control
• Simplest choice u Kpe
• Effect lifts gain with no phase change
• Corrects for current errors
• Proportional-Integral control
• Effect gives zero steady state error
• Corrects for aggregated error
• Proportional-Integral-Derivative control
• Effect gives high gain at low frequency plus
  phase lead at high frequency
• Corrects for anticipated changes

17
Summary Frequency Domain Design using PID

• Loop Shaping for Stability Performance
• Steady state error, bandwidth, tracking

• Main ideas
• Performance specs give bounds on loop transfer
  function
• Use controller to shape response
• Gain/phase relationships constrain design
  approach
• Standard compensators proportional, PI, PID

Slide source RMM, CDS101/110
18
Second Order System Response

• Second order system response
• Spring mass dynamics, written in canonical form
• Performance specifications

• Guidelines for pole placement
• Damping ratio gives Re/Im ratio
• Setting time determined by Re(?)

Ts lt x
Desired region for closed loop poles
Mp lt y
Slide source RMM, CDS101/110
19
Summary PID and Root Locus

• PID control design
• Very common (and classical) control technique
• Good tools for choosing gains

• Root locus
• Show closed loop poles as function of a free
  parameter
• Performance limits
• RHP poles and zeros place limits on achievable
  performance
• Waterbed effect

Slide source RMM, CDS101/110
20
Optimal Control Linear Quadratic Regulator (LQR)
Process
Controller
Trajectory Generation

• Trajectory Generation via Optimal Control
• Focus on special case of a linear quadratic
  regulator

Slide source RMM, CDS101/110
21
Finite Time LQR Summary
X

• Problem find trajectory that minimizes
• Solution time-varying linear feedback
• Note this is in feedback form ? can actually
  eliminate the controller (!)

Slide source RMM, CDS101/110
22
Infinite Time LQR

• Extend horizon to T ? and eliminate terminal
  constraint
• Solution same form, but can show P is constant
• Remarks
• ?In MATLAB, K lqr(A, B, Q, R)
• Require R gt 0 but Q ? 0 must satisfy
  observability condition
• Alternative form minimize output y H x
• Require that (A, H) is observable. Intuition if
  not, dynamics may not affect cost ? ill-posed.
  We will study this in more detail when we cover
  observers

State feedback (constant gain)
Algebraic Riccati equation
Slide source RMM, CDS101/110
23
Applying LQR Control
Process
Controller
Estimator
Trajectory Generation

• Application 1 trajectory generation
• Solve for (xd, yd) that minimize quadratic cost
  over finite horizon (requires linear process)
• Use local controller to regulate to desired
  trajectory
• Application 2 trajectory tracking
• Solve LQR problem to stabilize the system to the
  origin ? feedback u K x
• Can use this for local stabilization of any
  desired trajectory
• Missing so far, have assumed we want to keep x
  small (versus x ? xd)

Slide source RMM, CDS101/110
24
Choosing LQR weights

• Most common case diagonal weights
• Weight each state/input according to how much it
  contributes to cost
• Eg if error in x1 is 10x as bad as error in x2,
  then q1 10 q2
• OK to set some state weights to zero, but all
  input weights must be gt 0
• MATLAB K lqr(A, B, Q, R)
• Remarks
• LQR will always give a stabilizing controller,
  but no guaranteed margins
• LQR shifts design problem from loop shaping to
  weight choices
• Most practical design uses LQR as a first cut,
  and then tune based on system performance

Slide source RMM, CDS101/110
25
Until Next Time.

• Sean Humberts work on optic flow

26
BACKUP

• BACKUP

27
Robotics Lab Introducing the ER1s

• Simple two-wheeled robots
• Evolution Robotics
• Sensing Capabilities
• LADAR
• Odometry
• Localization
• Cameras
• K.I.S.S. philosophy
• Working with Jeremy and Noel in Robotics Lab
  (0014 Thomas)
• Feel free to stop by!

28
Player/Stage

• Robot sensor control and simulation software
• Player serves as an interface between control
  software and robot
• Open-source community (over 33 robotics labs)
• Plug-N-Play environment

http//playerstage.sourceforge.net
29
Teams of Decision-Makers

• Is sensor fusion better than decision fusion?
• Study how mobility plays a role in decision-making