Simple Practical Criteria for Time Delay t

1
Simple Practical Criteria for Time Delay t
1. The reconstructed attractor must be expanded
from the diagonal but not too much so that
it folds back
2. The components of any state vector x(t) must
be as uncorrelated as possible
2
Autocorrelation Mutual Information
Zero crossing of autocorrelation
First minimum of mutual information
Beloushov-Zhabotinskii Chemical Reaction
3
Buzug, Pfister (1992) Phys. Rev. A
Based on Fill Factor Algorithm
Choose randomly nref sets of (d1) vectors on the
attractor.
Calculate the volumes V(d,t) of the chosen sets
(hyper-parallelepipeds).
Calculate the average volume. Normalize by the
volume of the hypercube formed by using one half
of the maximumdifference between any two values
of the time series as the edge length.
Then, the Fill Factor (F) is defined as
where the average is done across all nref sets.
As a function of t, F will increase from large
negative values (at small t) to one or several
maxima indicating reconstructions with a larger
volume.
topt ? First maximum of F
As a function of the embedding dimension d, F is
non-increasing until d do, and then it
decreases linearly in d.
4
Fill Factor Rössler Attractor
d2
nref 2000
fill factor (F)
d15
time delay
The periodic collapses of F is caused by the
presence of dominant periodic structures in
Rössler attractor
Optimum time delay, topt ? 23
5
False Nearest Neighbors
Kennel et al. (1992) Phys. Rev. A
Assume, do dimensional time delay reconstruction
is a valid embedding but do-1 is not a true
embedding.
If it is not a true embedding, reconstruction is
a mere projection. As a consequence, state
vectors which are not geometrically correlated in
the original state space can map into vectors in
the reconstructed state space which can lie
close to each other.
These vectors are not true neighbors, but are
false neighbors.
But the problem is, how can we detect the true or
false neighbors? After all, we do not have the
access to original state space.
The answer ? If do dimensional time delay
reconstruction ensures an embedding, any
embedding dimension greater than do also ensures
the same.
One starts with low d and then increases the
embedding dimension gradually while keeping
track of neighborhood geometry.
do ? Minimum embedding dimension for which
no additional false neighbors are found.
6
Another Rationale behind FNN
The reconstructed attractor would not show
intersect itself to ensure a validembedding.
The original attractor lies on a smooth manifold
of dimension m. Self-intersections of the
reconstructed attractor indicates that it does
not lie on a smooth manifold ? reconstruction is
not valid!
The condition for no self-intersection ? All
neighbor points in ?d should also be
neighbors in ?d1
do ? The minimum embedding dimension which
produces reconstructions without any
self-intersection
7
1. Assume a fixed time delay t.
2. Reconstruct the time delay state space with
embedding dimension d.. Compute the distance
between any reference vector x(t) and k-th
neighbor x(tk)
Rd2(t,k) x(t) x(tk)2
3. Increase the dimension to d1 and compute the
distance between same pair of vectors,
Rd12. If x(tk) is a false neighbor, then the
change in the distance would be dramatic.
Define, the relative change in distance
Dd(t,k) (Rd12(t,k) - Rd2(t,k))/Rd2(t,k)
4. By choosing certain cutoff threshold Re, false
nearest neighbor is detected by checking the
condition Dd(t,k) gt Re Note The smaller the
neighborhood, the more reliable the test. The
smallest neighborhood contains only
immediate neighbor, k 1.
5. Then the proportion of vectors in
d-dimensional state space which have a
falseneighbor will be
8
P(d) is one measure of the adequacy of the
reconstruction dimension d. That is, as d
increases towards the optimal embedding dimension
do, P(d) should saturate at some small value.
Thus, we have to choose a tolerance d 0 and
then increase d until P(d) lt d.
do ? The smallest d satisfying this condition
For small noisy data set, close neighbors may not
be that close, so Dd may be small. Define,
closeness as some fraction Ae of the attractor
size d(A). Indicate a near neighbor as false if
or
Combining both conditions,
d(A) is the measure of spatial extent of the
attractor A Some examples d(A) can be the range
of the data, or the standard deviation
9
Remarks
Results are robust against Ae, Re, d
Results are also robust to the precise value of
t.Practically, it can be set to something like
the first minimum of the mutual information or
some fraction of correlation time.
The minimum of P(d) may not be zero, and can be
significantly greater than zero for finite data
sets even when the signal comes from a fully
deterministic process. So, one should look for
the stability in P(d).
In practice, one displays P(d) vs. d and choose
do which corresponds to the smallest d that
gives a steady minimum in P(d) or P(d) ltd.
10
Lorenz II Attractor
False Nearest Neighbor
Robustness against Noise
Embedding Dimension (d)
False Nearest Neighbor
Embedding Dimension (d)
11
Multivariate EEG Signals Listening to Music
topt ? 10
(d)
Time samples
12
Another embedding parameter is embedding window
since large embedding dimension can compensate
the effects of too small time delay.
Gibson et al. (1992) Physica D
Kugiumtzis (1996) Physica D
(mean time between two successive visits to a
local neighborhood)
For low-dimensional attractor, wopt ? Mean
interval between peaks (Averaging the time
between successive maxima of the time
series)
13
Why Study Dimension?
A dynamical system with p degrees of freedom may
finally settle on an attractor with much lower
dimension m (m lt p). ? related with active
degrees of freedom ? dimension of the attractor
In practice, this information is given by
ceil(dimension) because attractor dimension can
be fractal.
Certain dimension estimates are invariant
measures ? the dimension of the original
attractor the dimension of the
reconstructed state space
Dimension is a good indicator of deterministic or
stochastic process. ? a stochastic process would
use all available dimensions of state space
whereas a deterministic process would produce m
much smaller than p
Dimension can also be used as a measure of
systems complexity
14
What Dimension Doesnt Tell?
Dimension is a geometrical quantity, thus it
conveys only static information. No dynamical
information can be extracted from dimension.
No information on stability either!
Actually, all temporal information are discarded.
Different Types of Dimensions
Topological Dimension Hausdorff
Dimension Box-counting Dimension Information
Dimension Pointwise Dimension Correlation
Dimension
15
Topologiocal Dimension
The number of coordinates needed to specify a
point in the object or set
Points ? Dimension 0 Lines ? 1 Planes ? 2 Solid
Objects ? 3
In the reconstructed state space sense,
topological dimension is the dimension of the
manifold containing the attractor ? It
quantifies the minimum degrees of freedom
required to capture the dynamics of the
system in every neighborhood of the attractor.
Also known as intrinsic dimension.
16
Box-Counting Dimension
It expressed the relationship between the number
of boxes that contain part of an object and
the size of the boxes. ? how many number of
boxes of uniform size are required to
completely cover the attractor with minimal
overlaps.
Tiling a Curve by Line Segments
Tiling a 2-D Object by Boxes

• A line can be covered by one dimensional
  boxes, which are line segments.
• For a line with length L and line segments of
  length e, the number of line segments, N(e)
  L/e L(1/e)

• A closed surface can be covered by uniform
  square boxes.
• For a closed surface with area A and boxes
  with side e, the number of boxes, N(e) A/e2
  A(1/e)2

Similarly, a 3-D volume can be covered by uniform
cubes with volume e3, and so on. In general, one
can extend the approach to any shape and equate
the exponent of the term (1/e) to the dimension
of the shape.
17
In general, for an object with unknown dimension
D0, the number of D0 dimensional hypercubes with
edge e covering the object will be
where V is proportionality constant that depends
on the shape being covered.
D0 is called box-counting or capacity dimension.
18
Correlation Dimension
Grassberger, Procaccia (1983) Physica D
It is based on the concept of how densely the
points on an attractor aggregate around one
another and its estimation is related to the
relative frequency with which the attractor
visits each covering element.
We have the time series x(k), k 1,2,, N
Reconstruct the series with time delay embedding
with embedding dimension d and time delay t.
Consider any reference vector x(i), and find how
many state vectors lie closerthan a distance of
r, a given value
Heaviside Function
How many pairs of vectors are considered?
? (Nr-1)
Repeat the whole procedure for every state space
vector as reference. ? Nr state vectors
19
Define, Correlation Sum
It counts the number of state vector pairs (xi,
xj) whose distance is smaller than r. Thus, it is
a type of spatial correlation because it
expresses the extent to which embedded state
vectors are close together. Also C(r) gives the
probability of two randomly selected vectors
which would lie within a certain distance.
In the limit of an infinite amount of data points
(Nr ? ?) and for small r, we expectC(r) to scale
like a power law
The, the correlation dimension d2 can be defined
as
For finite data, d2 is also called correlation
exponent because the condition r ? 0 could not
be met.
20
An example Lorenz Attractor
In the log-log plot, a scaling region can be
seen, where
For large radius, C(r) approaches 1 or
for unnormalized C(r) For small radius, C(r)
approaches zero or statistical fluctuations
dominate
21
Correlation Sum Two Fold Lorenz Attractor
Unnormalized C(r)
Radius r
22
Estimation of Slope, d2
The straight method Fitting a straight line to
the scaling region by least-squares
find the slope (d2)
Its simple but not problem-free!
Individual values in the scaling region have
different statistical errors. It can be corrected
by weighted LS, giving large weights on larger
values. Individual values of C(r) are not
independent of each other larger values
contain the information from all nested smaller
values. But LSE always assumes independent data
points. ? Underestimation Finding scaling
region is not obvious! Further, a curve may
have many scaling regions.
Associating a single number for each C(r) vs. r
curve may not be appropriate!
23
Estimate slope on local basis (e.g., 5
consecutive values of r).
Some suggestions ri 1.1-i ro, r0 ? the largest
inter-point distance
d2 Lorenz Attractor Noise
d 22
d 2
24

• Macroscopic Regime At large r, the limited size
  or boundedness of reconstructed attractor causes
  the slope to drop to zero scaling isdestroyed
• Scaling Range True scaling behavior is found.
  The local scalingexponent is constant and the
  same for all embedding dimensionslarger than do.
  The scaling exponent is the estimate of the
  correlation dimension of the fractal set. If
  scaling region is bounded between (ru, rl), then
  the scalingregion length L ru/rl
• Noise Regime At small r, the measurement noise
  blurs the scaling and tends to fill out all
  directions in phase space.Thus, the local
  scaling exponent increases and they reach the
  value of embedding dimension.
• Regime of Rare Neighbors On very small length
  scales (very small values of r), the lack of
  neighbors becomesdominant effect and the curves
  start to fluctuate. This scale is smaller than
  the average inter-point distance in embedding
  space

d 22
d 2
25
Choice of Metric (Norm)
Theoretically, d2 is independent of the choice of
metric, but in practice, the performances of the
numerical results may depend on this choice.
Lp metrics
p 1 ? Manhattan metric 2 ? Euclidean
metric ?? ? Maximum metric

• Advantages of Euclidean Metric
• Direct physical meaning of distance
• Distances are independent of rotations of
  coordinate system
• Robust against noise contamination

• Advantages of Maximum Metric
• Computationally faster
• Provides quicker convergence for over-sampled
  data
• Distances remain bounded even in high
  dimensional state spaces

26
Dynamical Range
The maximum value of correlation sum C(r) is 1
for r rmax The minimum value of C(r) is ?
(2/Nr2)
The interval (2/Nr2, 1) ? Dynamical range of C(r)
For C(r) ? (2/Nr2),
The smallest scale which can be studied by using
correlation sum analysis
On the other hand, by applying box-counting
dimension estimation algorithm, the size of the
smallest scales that can be probed will be
Since d2 lt d0, r lt e , smaller scales can be
probed by d2 ? One reason for the popularity of
d2 for practical applications
27
d2 Estimation for Real-life Signals
Role of Embedding Dimension
Here, the embedding dimension d is not known.
Choose a suitable time delay t for state space
reconstruction.
Perform the algorithm for d2 estimation with
increasing embedding dimension.
For too small d, the estimate approximately
equals d. For too large d, the estimate
saturates at true d2.
A necessary condition for validating the
estimation procedure
It is usually believed that dimension is
preserved iff time delay coordinate map F is a
valid embedding, but it is not necessarily true.
Ding et al. (1993) PRL
An embedding will definitely preserve certain
dimensions, but other reconstructions that are
not a valid embedding will also preserve these
dimensions.
This is practically useful because smaller values
of d can be employed.
28
In theory, d2 should become invariant as soon as
d is large enough.But for some data sets, there
may be some dependency!
Schmid, Dünki (1996) Physica D
A possible model
b0 ? True correlation dimension b1 (1/d) where
d can be regarded as the embedding dimension at
which the attractor has unfolded up to 1/e
of its full extent (unfolding dim.)
For high dimensional attractor, the model can be
modified as
Potapov, Kurths (1998) Physica D
here, the estimates are function of both d and of
r. But for deterministic system, limr?0f(r)
0 ? a useful property to discriminate high
dimensional deterministic process from
stochastic process
29
Role of Temporal Correlations
Reference vector
A
Neighbors due to temporal correlation
Neighbors due to spatial correlation
B
The successive embedding vectorsmay remain close
also in state space due to continuous time
evolution ? Temporal correlation
Correlations in time is transformed into
correlations in space ? True not only for
deterministic process but also for stochastic one
Task To exclude those pairs of points which are
close, not due to the attractor shape, but just
because they are correlated (most possibly in
time).
So, we can start our computation for C(r) only
after a typical correlation timetmin nminDt
(Dt is the sampling time)
30
Theiler (1996) Phys. Rev. A
Modified Correlation Sum
ntheil nmin, Theilers correction
What value will we choose for ntheil?
Be generous!
Because by choosing a not too small value, we
will loose a few true neighbors, but the loss
(2ntheil/Nr) is negligible.
ntheil is a must for highly sampled continuous
data but for inter event sequence ntheil is not
necessary.
31
Space Time Separation Plot
Provenzale et al. (1992) Physica D
In the presence of temporal correlations, the
probability that a given pair of pointshas a
distance smaller than r does not only depend on r
but also on the time thathas elapsed between the
two measurements.
This dependence can be estimated by plotting the
number of pairs as a function of(i) the time
separation Dt, and (ii) the spatial distance r.
By studying the point density contours or the
accumulated histogram of spatial distances r for
each time separation, one can assess up to which
time scalesdynamical correlations are present.
For uncorrelated points, the density should not
depend on temporal distances.
32
Computational Remarks
The method can be computationally very
expensive For each r, Nr(Nr-1)/2 distances need
to be evaluated.
For example, number of data points 2000
embedding dimension (d) 6 time delay
(t) 3
Number of reconstructed state vectors, Nr 2000
(6-1)3 1985
Number of distances to be calculated ?
19851985/2 ? 1.98 Mi
In the estimation of d2, large r may not be that
useful because d2 is defined for r ? 0.
So, we can give more importance on the small
distances. But it requires that theset of points
should be organized hierarchically. Once the
hierarchy is made, the computational requirement
is NrlogNr (? 0.01 Mi).
33
Multidimensional tree provides a solution.
Bingham, Kot (1989) Phys. Lett. A
Problem The process can also be very time
consuming and cumbersome for high d
Theiler (1987) Phys. Rev. A Schreiber (1995) Int.
J Bif. Chaos
Box-assisted estimation is another solution.
Divide the phase space into a grid of boxes of
side length r. Each point falls into one of
these boxes. All its neighbors nearer than r
will lie in either the same box of in one of
the adjacent boxes. So, we dont search for the
entire phase space, we only search in the
adjacent boxes a great reduction in
computational time.