PWLApproximation of Nonlinear Dynamical Systems PartII: Identification Issues

1
PWL-Approximation ofNonlinear Dynamical
SystemsPart-II Identification Issues

• Oscar De Feo
• Swiss Federal Institute of Technology
• Marco Storace
• University of Genova

2
Overview

• From samples to functions (old)
• RBF linear regression
• simplicial selection (regression trees)
• From time series to samples (old)
• state space reconstruction
• A good sampling (new)
• exploring the state/parameters space
• excitation algorithm
• squaring the circle
• Results (new)
• parameter dependence
• chaotic behavior
• Conclusions

3
From Samples to Functions ILinear regression
given N the problem is easily solved in Least
Squares sense
the overfitting
but it is not taking into account the
bias-variance problem
4
From Samples to Functions IIRegularized RBF
linear regression Ridge regression

• estimated prediction error is obtained debiasing
  the variance S with the actual number of degrees
  of freedom
• s2S/(k-m)
• m depends on ? and is estimated trough heuristics
  as
• Generalized Cross Validation
• Bayesan Information Criterion
• Maximal Marginal Likelihood

large ? ? smooth functions small ? ? wrinkled
functions
Problem select ? minimal estimated prediction
error - bias-variance problem - model
selection criteria
We could solve that for the PWL basis but it
would be like reinventing the wheel
Problem solved for RBF ? Ridge regression
(Orr) (numerically efficient technique)
?-basis are RBF then if ? are needed just
convert
RECYCLING
but we can do even more
5
From Samples to Functions IIISimplicial
selection regression trees
recursively partition the domain in two and
approximate the function in each half by the
average output value of the samples it contains
Combining RBF and regression trees (Orr) we can
obtain also the best simplicial
partition Direction (xi) to split and stop
criterion by combination with RLS (trees of RLS
solved) efficient algorithm using bordered
matrices
n18
n213
6
From Time Series to ODEsIdentification

• now we know how to obtain a PWL approximation
• from samples of domain - codomain
• but usually we do not have the ODEs functions
• obtain them from time series ? identification
• approximate the ODEs obtained with specialized
  nonlinear identification techniques
• not a good idea ? double errors
• often identification techniques are based on RBF
• exploit directly PWL-RBF (?-bases)
• and then convert to ? if necessary

7
From Time Series to SamplesState space
reconstruction
The first step of identification is the State
Space Reconstruction then the time shift or
numerical differentiation gives the samples of
domain (state space) / codomain (diff/state
space)

• State space reconstruction techniques
• Delay embedding
• not good since sensitive to real data noise
• Projection of large delay embedding into the
  eigendirections
• PCA good because also denoising

from a scalar time series to a state space
delay embedding
of the covariance matrix
time series
PCA projection
8
Summary of IdentificationFrom measures to a PWL
ODE
state space reconstruction
time series
projection on a PWL base
identified PWL chaotic system
9
How to Fill Domain and Codomain? Obtaining good
samples

• Two questions
• if there are accessible parameters
• how to perturb to get the maximal insights with
  the minimal number of measures?
• if the system is not excitable (necessarily
  chaotic)
• how to fit the strange attractor in a
  hyper-rectangle?

or transients are available
10
A Good SamplingExploring the state/parameters
space I

• In linear identification (identificability)
• best perturbation is white noise
• In nonlinear it is not necessarily the best
• we developed a driving technique
• based on state transition speed estimation

idea switch the parameters when the state is at
a known value and mark the visited boxes
11
A Good SamplingExploring the state/parameters
space II
already visited cells
where you are
where you will be
comparison driven vs. pure random 30 shorter _at_
same cover
where you can go
where you want to go
parameter step
12
A Good SamplingSquaring the circle I

• static nonlinear identification is possible only
  with enough diversity
• transients are available ? similar to perturbed
  case
• chaotic systems
• chaotic systems are circular-like and not
  hypercubes
• to fill all the domain-codomain simplices
• ?nonlinear invertible distortion
• circle ? square (saturation)

13
A Good SamplingSquaring the circle II
cover 44.8
saturation
cover 68.1
14
ResultsParameter dependence and exploration
Cusp
Simplicial partition obtained with regression
tree (2,5,9)
15
ResultsParameter dependence and exploration
Bautin
Simplicial partition obtained with regression
tree (6,5,5)
16
ResultsChaotic system Colpitts oscillator
Simplicial partition obtained with regression
tree (7,5,6)
state space reconstruction
observed system
desaturation
PWL identification
saturation
17
Conclusions
but we had one in the presentation!!

• Applied PWL RBF (?-basis) to identification
• ? ?-basis (better for analog HW implementation)
• Recycled RBF regression trees for simplicial
  partition
• simplicial partition from samples
• Recycling theory Mark J.L. Orr
• (1996) Introduction to radial basis function
  networks
• (1999) Recent advances in radial basis function
  networks
• Developed an excitation algorithm
• parameter dependence and coexisting invariants
• Applied successfully to academic examples
• identification of chaotic system
• identification of parameter dependence

We did not need e for identification SORRY!!!