ESTIMATION OF LYAPUNOV SPECTRA FROM A TIME SERIES

1
ESTIMATION OF LYAPUNOV SPECTRAFROM A TIME SERIES
S. Srinivasan, S. Prasad, S. Patil, G. Lazarou
and J. Picone Intelligent Electronic
Systems Center for Advanced Vehicular
Systems Mississippi State University URL
http//www.cavs.msstate.edu/hse/ies/publications/c
onferences/ieee_secon/2006/lyapunov_exponents/
2

• Motivation

• Analysis of chaotic signals
• Reconstruction a phase-space from a scalar
  observable
• Lyapunov exponents as a tool to analyze chaos
• Lyapunov spectra of chaotic and non-chaotic time
  series
• Optimize parameters of Lyapunov spectra
  estimation

3

• Definitions

• A deterministic signal or system every event is
  the result of preceding events and actions hence
  predictable completely
• Stochastic noise signal that is not
  deterministic, i.e., inherently unpredictable
• A chaotic signal or system sensitive to initial
  conditions (Butterfly Effect)
• Chaos says predictability holds only in
  principle, hence chaotic signals are also called
  deterministic noise.
• Dimension of a system number degrees of freedom
  possessed by the system
• Deterministic Chaos or Stochastic Noise?
• Both have continuous power spectra (and not
  easily distinguishable)
• Noise is infinite-dimensional.
• Chaotic signals are finite dimensional, but
  dimension no longer associated with number of
  independent frequencies, but a statistical
  feature related to both temporal evolution and
  geometric aspect (self-similar structure of the
  attractor)

4

• Power Spectrum of a Lorentz Signal

• Power spectra of chaotic signals are continuous,
  though the system is finite dimensional. For
  example, the power spectrum of a 3-dimensional
  chaotic Lorentz signal is shown below. Stochastic
  systems have similar spectra even though they are
  infinite dimensional.

5

• Attractors for Dynamical Systems

• System Attractor Trajectories approach a limit
  with increasing time, irrespective of the initial
  conditions within a region
• Basin of Attraction Set of initial conditions
  converging to a particular attractor
• Attractors Non-chaotic (point, limit cycle or
  torus),or chaotic (strange attactors)
• Example point and limit cycle attractors of a
  logistic map (a discrete nonlinear chaotic map)

6

• Strange Attractors

• Strange Attractors attractors whose shapes are
  neither points nor limit cycles. They typically
  have a fractal structure (i.e., they have
  dimensions that are not integers but fractional)
• Example a Lorentz system with parameters

7

• Characterizing Chaos

• Exploit geometrical (self-similar structure)
  aspects of an attractor or the temporal evolution
  for system characterization
• Geometry of a Strange Attractor
• Most strange attractors show a similar structure
  at various scales, i.e., parts are similar to the
  whole.
• Fractal dimensions can be used to quantify this
  self-similarity.
• e.g., Hausdorff, correlation dimensions.
• Temporal Aspect of Chaos
• Characteristic exponents or Lyapunov Exponents
  (LEs) - captures rate of divergence (or
  convergence) of nearby trajectories
• Also Correlation Entropy captures similar
  information.
• Any characterization presupposes that phase-space
  is available.
• What if only one scalar time series measurement
  of the system (and not its actual phase space) is
  available?

8

• Reconstructed Phase Space (RPS) Embedding

• Embedding A mapping from an one-dimensional
  signal to an m-dimensional signal
• Takens Theorem
• Can reconstruct a phase space equivalent to the
  original phase space by embedding with m  2d1
  (d is the system dimension)
• Embedding Dimension a theoretically sufficient
  bound in practice, embedding with a smaller
  dimension is adequate.
• Equivalence
• means the system invariants characterizing the
  attractor are the same
• does not mean reconstructed phase space (RPS) is
  exactly the same as original phase space
• RPS Construction techniques include differential
  embedding, integral embedding, time delay
  embedding, and SVD embedding

9

• Reconstructed Phase Space (RPS) Time Delay
  Embedding

• Uses delayed copies of the original time series
  as components of RPS to form a matrix
• m embedding dimension, delay parameter
• Each row of the matrix is a point in the RPS

10

• Reconstructed Phase Space (RPS)

Time Delay Embedding of a Lorentz time series
11

• Reconstructed Phase Space (RPS) Time Delay
  Embedding

• Setting very small delay value leads to highly
  correlated vector elements, concentrated around
  the diagonal in embedding space. Structure
  perpendicular to the diagonal not captured
  adequately.
• Setting very large delay value leads elements of
  the vector to behave as if they are independent.
  Evolutionary information in the system is lost.
• Quantitative tools for fixing delay plots of
  autocorrelation and auto-mutual information are
  useful guides.
• Advantages easy to compute the attractor
  structure is not distorted since no extra
  processing is done on it.
• Disadvantages choice of delay parameter value is
  not obvious leads to poor RPS in presence of
  noise.

12

• Reconstructed Phase Space (RPS) SVD-based
  Embedding

• Works in two stages
• Delay embed, with one sample delay, to a
  dimension larger than twice the actual embedding
  dimension
• Reduce this matrix using SVD to finally have
  number of columns equal to embedding dimension.
• (SVD-based matrix reduction is done by
  projecting each row onto only the first few
  eigenvectors and then reconstructing it to a
  lower-dimensional space)
• SVD window size dimension of time delayed
  embedded matrix over which SVD operates
• Advantages No delay parameter value to be set
  more robust to noise due to SVD stage
• Disadvantages Noise reducing property of SVD may
  also distort the attractor properties

13
Reconstructed Phase Space (RPS) Reconstruction
Attractor reconstruction using SVD embedding
(for a Lorentz system)

14

• Lyapunov Exponents

• Quantifies separation in time between
  trajectories, assuming rate of growth (or decay)
  is exponential in time, as
• where J is the Jacobian matrix at point p.
• Captures sensitivity to initial conditions.
• Analyzes separation in time of two trajectories
  with close initial points
• where is the systems evolution function.

15

• Lyapunov Exponents Some Properties

• m-dimensional system has m LEs
• LE is a measure averaged over the whole attractor
• Sum of first k LEs rate of growth of
  k-dimensional Euclidean volume element
• Bounded attractor Sum of all LEs equals zero
  (conservative) or negative (dissipative)
• Zero exponents indicate periodic attractor (limit
  cycle) or a flow
• Negative exponents pull points in the basin of
  attraction to the attractor
• Positive exponents indicate divergence signature
  for existence of chaos

16

• Lyapunov Exponents Computation

• Embed time series to form RPS matrix. Rows
  represent points in phase space
• Take first point as center
• Form neighborhood matrix, each row obtained by
  subtracting a neighbor from the centre
• Find evolution of each neighbor and form the
  evolved neighborhood matrix by subtracting each
  evolved neighbor from the evolved centre
• Compute trajectory matrix at the center by
  multiplying pseudo-inverse of neighborhood matrix
  with evolved neighborhood matrix
• Advance center to a new point and go to step 3,
  averaging the trajectory matrix in each iteration
• The LEs are given by the average of the
  eigenvalues from each R matrix. Direct averaging
  has numerical issues, hence an iterative QR
  decomposition method (treppen-iteration) is used.

17

• Lyapunov Exponents Computation Flowchart

18

• Experimental Design

• Three systems tested two chaotic (Lorentz and
  Rossler) and one non-chaotic (sine signal)
• Two test conditions clean and noisy (10 dB white
  noise)
• Lorentz system
• Parameters
• Expected LEs (1.37, 0, -22.37)
• Rossler system
• Parameters a 0.15, b 0.2, c 10
• Expected LEs (0.090, 0.00, -9.8)
• Sine Signal
• Parameters Freq1Hz, Samp freq16Hz, Amp1
• Expected LEs (0.00, 0.00, -1.85)

19

• Experimental Design

• Experiments performed to optimize parameters of
  estimation algorithm
• 30,000 points were generated for each series in
  both the conditions
• 5,000 iterations (or the number of evolution
  steps) were used for averaging using QR
  treppen-iteration
• Variation of LEs with SVD window size and number
  of nearest neighbors
• Varied number of neighbors with SVD window size
  15 for clean data 50 for noisy data
• Varied SVD window size with number of neighbors
  15 for clean data 50 for noisy data

20

• Experimental Results

Lyapunov Exponents (LEs) for a Lorentz System

• For clean data Positive and zero exponents are
  almost constant at the expected values
• For noisy data Positive and zero exponents
  converge to the expected value for SVD window
  size about 50 and number of neighbors also about
  50
• Negative LE estimation not reliable

21

• Experimental Results

Lyapunov Exponents (LEs) for a Rossler System

• For clean data Positive and zero exponents are
  almost constant at the expected values
• For noisy data Positive and zero exponents
  converge to the expected value for SVD window
  size about 60 and number of neighbors also about
  50
• Negative LE estimation not reliable

22

• Experimental Results

Lyapunov Exponents (LEs) for Sine Signal

• For clean data Positive and zero exponents are
  almost constant at the expected values
• For clean data Positive and zero exponents
  converge to the expected value for SVD window
  size about 40 and number of neighbors also about
  30
• Negative LE estimation not reliable

23

• Summary and Future Work

• LEs are useful in quantifying how chaotic a
  system is.
• SVD embedding helps reconstructing phase spaces
  in noisy conditions.
• Parameters of the LE computation algorithm are
  optimized experimentally to get reliable
  estimates.
• Both the positive and zero LEs are estimated
  near the actual values using optimized
  parameters.
• Negative LE estimation is unreliable (but this is
  of little concern in chaotic systems).
• The code for LE estimation is publicly available.
• Our future work will be to apply Lyapunov
  exponents to model nonlinearities in speech for
  better automatic speech recognition.

24

• Resources

25

• References

• J. P. Eckmann and D. Ruelle, Ergodic Theory of
  Chaos and Strange Attractors, Reviews of Modern
  Physics, vol. 57, pp. 617-656, July 1985.
• M. Banbrook, Nonlinear analysis of speech from
  a synthesis perspective, PhD Thesis, The
  University of Edinburgh, Edinburgh, UK, 1996.
• E. Ott, T. Sauer, J. A. Yorke, Coping with
  chaos, Wiley Interscience, New York, New York,
  USA, 1994.
• M. Sano and Y. Sawada, Measurement of the
  Lyapunov Spectrum from a Chaotic Time Series,
  Physical Review Letters, vol. 55, pp. 1082-1085,
  1985.
• G. Ushaw, Sigma delta modulation of a chaotic
  signal, PhD Thesis, The University of Edinburgh,
  Edinburgh, UK, 1996.