General Linear Process GLP

1
General Linear Process (GLP)

• A zero mean random process Xn is a GLP if its
  sample functions can be represented as a output
  of a LTI system whose input is sample function of
  a zero mean (Gaussian) white noise process.

h(n)
w(n)
x(n)
2
How one can construct a model of random process
from a measured sample function?
Assumptions Measured signal originates from a
linear system driven by white noise. Effective
task Transform the white noise by a linear
system model so that the output has
characteristics of the observed signal.
h(n)
w(n)
x(n)
3
How one can construct a model of random process
from a measured sample function?
Assumptions Measured signal originates from a
linear system driven by white noise. Effective
task Transform the white noise by a linear
system model so that the output has
characteristics of the observed signal.
h(n)
w(n)
x(n)
But what properties to model? Mean?
No, its too coarse, has no temporal information.
Sample autocorrelation is a better candidate
which represents temporal variations and the
memory in the system.
Then the effective task How does one determine
the linear system which transforms uncorrelated
white noise into an output having same sample
correlation?
4
AutoRegressive Moving-Average (ARMA) Model
Autoregressive part
Moving average part
Model parameters pq1
Transfer function of ARMA(p,q) model
5
Linear Regression

• Consider the following model
• y(k) is the response or the dependent variable
  (output)
• u(k) is the independent variable (input)
• e(k) is the noise or model uncertainty

6
In vector notation h data vector ? -y(k-1)
-y(k-N) u(k-1) u(k-N)T Q parameter vector ?
a1 a2 aN b1 b2 bNT ??n
Measurements from time (k-m1) to k
In matrix notation
? Y HQ D
7
Least Squares Estimator (LSE)
M sets of observations Y and H, the objective is
to estimate n parameters of Q LS estimation is
based on minimizing the scalar cost function
Expanding,
Then,
Thus, LS estimate QLS
2HTHQLS 2HTY ? QLS HTH-1 HTY
8
Properties of LSE

• (Y-HQLS)TH DTH 0 ? orthogonality condition,
  i.e. QLS is chosen such that error vector D
  is orthogonal to each of the columns of H
• LSE is a linear transformation on Y, thus LSE is
  called a linear estimator
• LSE requires mgtn, i.e. number of measurements (m)
  has to be greater than the number of unknown
  parameters. In practice, m ?? n
• The quality of LSE depends on
• The richness of information in H
• The statistical properties of error D
• Algebraically, lack of information is manifested
  in the rank deficiency of H
• Multicolinearity ? redundancy in columns of H
• Too little variations ? redundancy in rows of H

9
Noise Characteristics

• Say, error D is zero mean with covariance matrix
  R.
• ED 0, ED DT R
• If D and H are statistically independent, LSE
  will be unbiased.
• Let, the error in estimation,
• Qer Qtr Qls Qtr True parameter
• Qls LS estimate
• Now, Qls (HTH)-1 HTY
• Then, Qer Qtr - (HTH)-1 HTY
• Qtr - (HTH)-1 HT(H Qtr D)
• -(HTH)-1HTD
• EQer -(HTH)-1HTED 0
• That is, LSE is an unbiased estimator, EQls
  EQtr

10
Covariance of estimation error P EQerQerT
E(HTH)-1HTDDTH(HTH)-1
(HTH)-1HTRH(HTH)-1 Thus, P, an indicative of
performance of the estimator, does not depend
on Y
11
Why Nonlinear Analysis?

• The system under study is believed to be
  nonlinear
• Our linear model was found to be not sufficient
  - lack of parsimonious modeling - failure in
  prediction - poor control
• The data show interesting structures, not
  explained by linear dynamics
• The uncertainties cannot be explained by assuming
  random perturbations

12
Pulsations in NH3-FIR Laser
Hübner et al (1989) Phys. Rev. A
Laser Output
Time samples
Laser Output
x(k2)
x(k)
Time samples
13
Continuous Blood Pressure Waveform Healthy
Subject
Bhattacharya et al (1998) IEEE-SMC
Time samples
x(k7)
x(k)
Time samples
14
EEG Signals from 3 Human Subjects
Bhattacharya et al (2001) IEEE-BME
15
Sunspot Oscillations
Original Data
Monthly Sunspot Number
Surrogate Data
Year
16
Dynamical Systems

• Any system whose state changes with time is a
  dynamical system.
• They are characterized by a function relating the
  systems past to its future, i.e. setting the
  rule for state transition.

If the state transition rule is mathematical
function devoid of any uncertainty, it
is called a deterministic dynamical system.
Given its present state, the future is
precisely determined. Given its present state,
the past is also precisely determined if the
state transition rule is invertible.
Implicit in the definition The system resides in
some m-dimensional state space M where the
transition rule is continuously applied to yield
an orbit or state space trajectory.
17
Dynamical systems Equation
where s(t) ? M, the state vector at time t
Fm is the velocity vector or the state
transition m is a vector of parameters

If Fm depends on time,
Nonautonomous system which is nonstationary
Stationary system must be autonomous, but the
inverse is not always true!
Our systems are autonomous and stationary ?
18
Delay differential equation (DDE)
Here, current state of the system depends on all
states between s(t-t) to s(t). Thus, DDE is
effectively infinite dimensional !
Say, the state transitions are not continuous in
time, but discrete.
19
Maps and Discretised Flows
Continuous flow
Discretising the flow
Thus, a continuous flow is transformed into a
discrete map

• In practice
• Take a set of differential equations
• Integrate with numerical algorithm (i.e.
  Runge-Kutta method) with constant
  integration time Dn
• Subsample the series such that the final signal
  corresponds to a time increment of ns Dn
  between consecutive values ( ns p.Dn, p an
  integer)

20
Discretized solution is only an approximation,
not a true solution.
Now, chaotic systems show sensitive dependence on
initial condition Then, how can we expect to get
the solution?
Thanks to Shadowing Theorem
Though a numerically generated trajectory will
typically not be the original or true trajectory,
there still exists true trajectories which stay
very close to the numericallygenerated
trajectory for long enough times. As a result,
numerically generated series can be considered as
a validapproximations of the true solution.
21
Flow ? Map
Attractor of the system lies in m-dimensional
state space M
Take a (m-1) dimensional surface and insert it
into the attractor in such a way that the
systems trajectory always intersects the surface
transversely. This surface is called Poincare
Section.
Find the points where the surface is intersected
by the trajectory at a chosen direction and
denote their locations as ri, i 1,2,.
r1
r(i1) P(r(i)) P ? Poincare First Return Map
r2
If the trajectory is strictly periodic, then
Poincare section shows a single point
22
Depending on the structure of Fm, there are
various possibilities for t ? ?, i,e,
asymptotic behavior after discarding the initial
transients.
If the solution diverges, s(t) ? ?, forget it !
If the system is dissipative, i.e. loosing
energy, volumes in state space will be reduced
with time but after sufficient long time, the
system will evolve onto a reduced set of
states, called the attractor (A).
Attractor A subset of the state space upon
which trajectories of a system accumulate in
the long term. It is invariant under the
action of flow, Fm Fm(A) A ? Any
trajectory that starts out on A will remain on
A A attracts an open set of initial
conditions. A is minimal, i.e. there is no
proper subset of A satisfying the condition
of an attractor.
Basin of Attraction The set of all initial sets
whose trajectories evolve into A
23
Remarks

• The basin of attraction occupies a large portion
  of the state space, but the attractor occupies
  only a small portion.
• Attractor is consisted of few degrees of freedom
  (lt10) regardless of the dimension of the state
  space
• Phase space is a state space without an explicit
  time reference. Since the dynamics is time
  independent (autonomous), phase space and state
  space are equivalent.

24
Classes of Attractor
Fixed Point
It is a state p such that Ft(p) p, ?t A system
which has fixed point attractor is motionless,
and the time series is constant. Dimension 0
Limit Cycle
A periodic orbit with period T is a limit set C
such that Ft(c) FtT(c), ?t, c? C A limit
cycle is an attracting periodic orbit. The time
series is periodic. Dimension 1
25
Limit Tori
More than one frequency in the periodic
trajectory and at least two of the frequencies
are incommensurate. N-torus if there are N
incommensurate frequencies. The time series will
be quasiperiodic. Dimension N
Strange Attractor
They are like other types of attractors for
dissipative systems but they also grow for some
directions in state space. The time series is
aperiodic, broad-band, and chaotic. Dimension
Non-integer value
26
Requirement to Produce Interesting Response
Nonlinearity has to be there!
For flow The asymptotic behavior of continuous
dynamical system in two or less dimension is
limited.
Poincare-Benedixon Theorem A differentiable
two-dimensional continuous dynamical system has
only fixed point and limit cycle atrractors.
Thus, chaotic dynamics cannot arise in flow in
less than 3 dimensions.
But, it can arise in discrete map with two or
less dimensions.
27
Rösler Flow
Rösler (1976) Phys. Lett. A
a 0.2, b 0.2
c 3.5
c 2.5
c 4
c 5
28
Lorenz (1963) J. Atmos. Sci.
Lorenz Flow
s 16, r 45.92, b 4
x
z
z
y
x
29
May (1976) Nature
Logistic Map
r is the control scalar parameter, ? 0,4
x
r
For r lt rc 3.5699456, the series is
periodic. Afterwards, series is chaotic with some
periodic windows.
30
Hénon (1976) Comm. Math. Phys.
Henon Map
a 1.4, b 0.3
y(n)
x(n)
n
x(n)
31
Ikeda Map
x
y
y
x
n
a 5.4, b 0.9, k 0.92
32
Equations with simple nonlinearity can
produce Complex Irregular Solution