Chaos Theory and Predictability

1
Chaos Theory and Predictability

• Anthony R. Lupo
• Department of Soil, Environmental, and
  Atmospheric Sciences
• 302 E ABNR Building
• University of Missouri
• Columbia, MO 65211

2
Chaos Theory and Predictability

• Some popular images..

3
Chaos Theory and Predictability

• Any attempt at weather forecasting is immoral
  and damaging to the character of a meteorologist
  - Max Margules (1904) (1856 1920)
• Margules work forms the
• foundation of modern
• Energetics analysis.

4
Chaos Theory and Predictability

• Chaotic or non-linear dynamics ? Is perhaps one
  of the most important discovery or way of
  relating to and/or describing natural systems in
  the 20th century!
• Caoz Chaos and order are opposites in the
  Greek language - like good versus evil.
• Important in the sense that well describe the
  behavior of non-linear systems!

5
Chaos Theory and Predictability

• Physical systems can be classified as 
• Deterministic ? laws of motion are known and
  orderly (future can be directly determined from
  past)
• Stochastic / random ? no laws of motion, we can
  only use probability to predict the location of
  parcels, we cannot predict future states of the
  system without statistics. Only give
  probabilities!  

6
Chaos Theory and Predictability
7
Chaos Theory and Predictability

•  
• Chaotic systems ? We know the laws of motion, but
  these systems exhibit random behavior, due to
  non-linear mechanisms. Their behavior may be
  irregular, and may be described statistically.
• E. Lorenz and B. Saltzman ? Chaos is order
  without periodicity.

8
Chaos Theory and Predictability

•  
• Classifying linear systems
• If I have a linear set of equations represented
  as
• (1)
• And b is the vector to be determined. Well
  assume the solutions are non-trivial.
• Q What does that mean again for b?
• A b is not 0!

9
Chaos Theory and Predictability

•  
• Eigenvalues are a special set of scalars
  associated with a linear system of equations
  (i.e., a matrix equation) that are sometimes also
  known as characteristic roots (source Mathworld)
  (l)
•  
• Eigenvectors are a special set of vectors
  associated with a linear system of equations
  (i.e., a matrix equation) that are sometimes also
  known as characteristic vectors (vector b)

10
Chaos Theory and Predictability

•  
• Thus we can easily solve this problem since we
  can substitute this into the equations (1) from
  before and we get
• Solve, and so, now the general solution is
• Values of c are constants of course. The
  vectors b1 and b2 are called eigenvectors of
  the eigenvalues l1 and l2.

11
Chaos Theory and Predictability

• Particular Solution

12
Chaos Theory and Predictability

•  
• One Dimensional Non-linear dynamics 
• We will examine this because it provides a nice
  basis for learning the topic and then applying to
  higher dimensional systems.
• ? However, this can provide useful analysis of
  atmospheric systems as well (time series
  analysis). Bengtssen (1985) Tellus Blocking.
  Federov et al. (2003) BAMS for El Nino. Mokhov et
  al. (1998, 2000, 2004). Mokhov et al. (2004) for
  El Nino via SSTs (see also Mokhov and Smirnov,
  2006), but also for temperatures in the
  stratosphere. Lupo et al (2006) temperature and
  precip records. Lupo and Kunz (2005), and Hussain
  et al. (2007) height fields, blocking.  

13
Chaos Theory and Predictability

•  
• First order dynamic system
• (Leibnitz notation is x dot?)
• ?If x is a real function, then the first
  derivative will represent a(n) (imaginary) flow
  or velocity along the x axis. Thus, we will
  plot x versus x dot

14
Chaos Theory and Predictability

•  
• Draw
• Then, the sign of f(x) determines the sign of
  the one dimensional phase velocity.
• Flow to the right (left) f(x) gt 0 (f(x) lt 0)

15
Chaos Theory and Predictability

•  
• Two Dimensional Non-linear dynamics 
• Note here that each equation has an x and a y
  in it. Thus, the first deriviatives of x and y,
  depend on x and y. This is an example of
  non-linearity. What if in the first equation Ax
  was a constant? What kind of function would we
  have? 
• Solutions to this are trajectories moving in the
  (x,y) phase plane.

16
Chaos Theory and Predictability

•  
• Coupled set If the set of equations above are
  functions of x and y, or f(x,y).
• Uncoupled set If the set of equations above are
  functions of x and y separately.

17
Chaos Theory and Predictability

•  
• Definitions
• Bifurcation point In a dynamical system, a
  bifurcation is a period doubling, quadrupling,
  etc., that accompanies the onset of chaos. It
  represents the sudden appearance of a
  qualitatively different solution for a nonlinear
  system as some parameter is varied.

18
Chaos Theory and Predictability

•  
• Example pitchfork bifurcation (subcritical)
• Solution has three roots, x0, x2 r

19
Chaos Theory and Predictability

•  
• The devil is in the details?............

20
Chaos Theory and Predictability

•  
• An attractor is a set of states (points in the
  phase space), invariant under the dynamics,
  towards which neighboring states in a given basin
  of attraction asymptotically approach in the
  course of dynamic evolution. An attractor is
  defined as the smallest unit which cannot be
  itself decomposed into two or more attractors
  with distinct basins of attraction

21
Chaos Theory and Predictability

• How we see it.

22
Chaos Theory and Predictability

•  Mathematics looks at Equation of Motion (NS) is
  space such that
• Closed or compact space such that ? boundaries
  are closed and that within the space divergence
  0
• Complete set ? div 0 and all the interesting
  sequences of vectors in space, the support space
  solutions are zero.

23
Chaos Theory and Predictability

•  
• Ok, lets look at a simple harmonic oscillator
  (pendulum) 
• Where m mass and k Hookes constant.
• When we divide through by mass, we get a Sturm
  Liouville type equation.

24
Chaos Theory and Predictability

•  
• One way to solve this is to make the problem
  self adjoint or to set up a couplet of first
  order equations like so let

25
Chaos Theory and Predictability

•  
• Then divide these two equations by each other to
  get
•  
•  
•  
• What kind of figure is this?

26
Chaos Theory and Predictability
27
Chaos Theory and Predictability

•  
• A set of ellipses in the phase space.

28
Chaos Theory and Predictability

•  
• Here it is convenient that the origin is the
  center!
•  
• At the center, the flow is still, and since the
  first derivative of x is positive, we consider
  the flow to be anticyclonic (NH) clockwise
  around the origin. The eigenvalues are
•  
•  
•  
• Now as the flow does not approach or repel from
  the center, we can classify this as neutrally
  stable.

29
Chaos Theory and Predictability

•  
•   Thus, the system behaves well close to certain
  fixed points, which are at least neutrally
  stable.
•  
• System is forever predictable in a dynamic sense,
  and well behaved.
• we could move to an area where the behavior
  changes, a bifurcation point which is called a
  separatrix.
•  
• Beyond this, system is unpredictable, or less so,
  and can only use statistical methods. Its
  unstable!

30
Chaos Theory and Predictability

•  
• Hopfs Bifurcation
• Hopf (1942) demonstrated that systems of
  non-linear differential equations (of higher
  order that 2) can have peculiar behavior.
•  
• These type of systems can change behavior from
  one type of behavior (e.g., stable spiral to a
  stable limit cycle), this type is a supercritical
  Hopf bifurcation.
•  
•  

31
Chaos Theory and Predictability

•  
• Hopfs Bifurcation
• Subcritical Hopf Bifurcations have a very
  different behavior and these we will explore in
  connection with Lorenzs equations, which
  describe the atmospheres behavior in a
  simplistic way. With this type of behavior, the
  trajectories can jump to another attractor
  which may be a fixed point, limit cycle, or a
  strange attractor (chaotic attractor occurs
  in 3 D only!)

32
Chaos Theory and Predictability

•  
• Example of an elliptic equation in meteorology

33
Chaos Theory and Predictability

• Taken from Lupo et al. 2001 (MWR)

34
Chaos Theory and Predictability

•  
• Ok, lets modify the equation above
•  
•  
•  
• d is now the damping constant, so lets damp
  (add energy to) this expression d gt 0 (dlt 0).
•  
• Then the oscillator loses (gains) energy and the
  determinant of the quadratic solution is also
  less (greater) than zero! So trajectories spin
  toward (away from) the center. This is a(n)
  (un)stable spiral.

35
Chaos Theory and Predictability

•  
• Example forced Pendulum (J. Hansen)

36
Chaos Theory and Predictability

•  
• Another Example behavior of a temperature series
  for Des Moines, IA (taken from Birk et. al. 2010)

37
Chaos Theory and Predictability

•  
• Another Example behavior of 500 hPa heights in
  the N. Hemi. (taken from Lupo et. al. 2007,
  Izvestia)

38
Chaos Theory and Predictability

•  
• Sensitive Dependence on Initial Conditions (SDIC
  not a federal program ? ).
• Start with the simple system
•  
• A iterative-type equation used often to
  demonstrate population dynamics
•  
• Experiment with k 0.5, 1.0, 1.5, 1.6, 1.7, 2.0
•  
•  

39
Chaos Theory and Predictability

•  
• For each, use the following xn and graph
  side-by-side to compare the behavior of the
  system.
•  
• Xn -0.5, Xn -0.50001
•  
•   
• Try to find period 2 attractor or attracting
  point behavior1 ? behavior2 ? behavior 1 ?
  behavior2, and a period 4 attractor.
•  
•  
• Period 2 behaves like the large-scale flow?

40
Chaos Theory and Predictability

•   
• Examine the initial conditions. One can be taken
  to be a measurement and the other, a
  deviation or error, whether its generated
  or real. Its a point in the ball-park of the
  original.
• Asside Heisenbergs Uncertainty Principle ? All
  measurements are subject to a certain level of
  uncertainty.

41
Chaos Theory and Predictability

•  
• X 0.5 X 0.50001

42
Chaos Theory and Predictability

•  
• Whats the diff?

43
Chaos Theory and Predictability

•  
• The differences that emerge illustrate the
  concept of Sensitive Dependence on Initial
  Conditions (SDIC). This is an important concept
  in Dynamic systems. This is also the concept
  behind ENSEMBLE FORECASTING!
•  
• Toth, Z., and E. Kalnay, 1993 Ensemble
  forecasting at NCEP The generation of
  perturbations. Bull. Amer. Meteor. Soc., 74,
  2317 2330.
•  
• Toth, Z., and E. Kalnay, 1997 Ensemble
  forecasting at NCEP and the breeding method. Mon.
  Wea. Rev., 125, 3297 3319.
•  
• Tracton, M.S., and E. Kalnay, 1993 Ensemble
  forecasting at the National Meteorological
  Center Practical Aspects. Wea. and Forecasting,
  8, 379 39

44
Chaos Theory and Predictability

•  
• The basic laws of geophysical fluid dynamics
  describe fluid motions, they are a highly
  non-linear set of differentials and/or
  differential equations.
• e.g.,
•  
• Given the proper set of initial and/or boundary
  conditions, perfect resolution, infinite computer
  power, and precise measurements, all future
  states of the atmosphere can be predicted
  forever!

45
Chaos Theory and Predictability

•  
• Given that this is not the case, these equations
  have an infinite set of solutions, thus anything
  in the phase space is possible.
• In spite of this, the same solutions appear
  time and time again!

46
Chaos Theory and Predictability

•  
• Note We will define Degrees of Freedom ? here
  this will mean the number of coordinates in the
  phase space.
•  
•  
• Advances in this area have involved taking
  expressions with an infinite number of degrees of
  freedom and replacing them with expressions of
  finite degrees of freedom.
•  
• For the equation of motion, whether we talk about
  math or meteorology, we usually examine the N-S
  equations in 2-D sense. Mathematically, this is
  one of the Million dollar problems to solve in
  3-d (no uniqueness of solutions!).

47
Chaos Theory and Predictability

•  
• Chaotic Systems
• 1. A system that displays SDIC.
• 2. Possesses Fractal dimensionality

48
Chaos Theory and Predictability

•  
• Fractal geometry self similar
• Norwegian Model L. Lemon

49
Chaos Theory and Predictability

•  
• Fractals
• Fractal geometry was developed by Benoit
  Mandelbrot (1983) in his book the Fractal
  Geometry of Nature. Fractal comes from Fractus
  broken and irregular.
•  
•  
•   
• Fractals are precisely a defining characteristic
  of the strange attractor and distinguishes these
  from familiar attractors.

50
Chaos Theory and Predictability

•  
• 3. Dissipative system
• Lyapunov Exponents - defined as the average rates
  of exponential divergence or convergence of
  nearby trajectories.
•  
•  
• They are also in a very real sense, they provide
  a quantitative measure of SDIC. Lets introduce
  the concept using the simplest type of
  differential equation.

51
Chaos Theory and Predictability

•  
• Simple differential equation
• with the solution as

52
Chaos Theory and Predictability

•  
• Thus, after some large time interval t, the
  distance e(t) between two points initially
  separated by e(0) is
• Thus, the SIGN of the exponent l here is of
  crucial importance!!!!
• A positive value for l infers that trajectories
  separate at an exponential rate, while a negative
  value implies convergence as t ? infinity!

53
Chaos Theory and Predictability

•  
• Well, we can use our simple differential equation
  to get the value of the exponent as
• So, in the general case of our differential
  equation, we can think of a (particular) solution
  as a point on the phase space, and the
  neighboring points as encompassing an
  n-dimensional ball of radius e(0)!
•   
•  
• With an increase in time, the ball will become an
  ellipsoid in non-uniform flow, and will continue
  to deform as time approaches infinity.

54
Chaos Theory and Predictability

•  
• There must be, by definition, as many Lyapunov
  exponents as there are dimensions in the phase
  space.
• Again, positive values represent divergence,
  while negative values indicate convergence of
  trajectories, which represent the exponential
  approach to the initial state of the Attractor!

55
Chaos Theory and Predictability

•  
• There must also, by definition, be one exponent
  equal to zero (which means the solution is unity)
  or corresponds to the direction along the
  trajectory, or the change in relative
  divergence/convergence is not exponential.
• Now, for a dissipative system, all the
  trajectories must add up to be negative!

56
Chaos Theory and Predictability

•  
• Lorenz (1960), Tellus
• First Low Order Model (LOM) in meteorology,
  derived using Galerkin methods, which
  approximate solutions using finite series. (e.g.
  Haltiner and Williams, 1980).

57
Chaos Theory and Predictability

•  
• Lorenz (1960)
• Tellus

58
Chaos Theory and Predictability

•  
• Lorenz (1963), J. Atmos. Sci., 20, 130 - 142
• Investigated Rayleigh Bernard (RB) convection,
  a classical problem in physics.
• We need to scale the primitive equations (use
  Boussinesq approx), then use Galerkin Techniques
  again.

59
Chaos Theory and Predictability

•  
• Lorenz (1963)
• solution

60
Chaos Theory and Predictability

•  
• Lorenz (1963) then using the initial
  conditions s 10.0 , b 8/3, r 28.0, and a
  non-dimensional time step of 0.0005.
• Then using 50 lines of FORTRAN code, and the
  leapfrog method, we can produce

61
Chaos Theory and Predictability

•  
• The Butterfly

62
Chaos Theory and Predictability

•  
• We cannot solve Lorenzs (1963) LOM unless we
  examine steady state conditions that is dx/dt,
  dy/dt, and dz/dt all equal zero.
• The trivial solution x y z 0, is the
  state of no convection.

63
Chaos Theory and Predictability

•  
• But, if we solve the equations, we get some
  interesting roots (0 lt r lt 1).

64
Chaos Theory and Predictability

•  
• But when r gt 1, we get convection and chaotic
  motions

65
Chaos Theory and Predictability

•  
• Predictability
• SDIC in the flow exists in set A if there exists
  error gt 0, such that for any and any
  neighborhood U of x, there exists and t
  gt 0. such that

66
Chaos Theory and Predictability

•  
• In plane English there will always be SDIC in
  a system (its intrinsic to many systems).
  Possible outcomes are larger than the error in
  specifying correct state!
• SDIC means that trajectories are unpredictable,
  even if the dynamics of a system are well-known
  (deterministic).
•  

67
Chaos Theory and Predictability

•  
• Thus, if you wish to compute trajectories of X in
  a system displaying SDIC, after some time ? t,
  you will accumulate error in the prediction
  regardless of increases in computing power!
• There is always resolution and measurement error
  to contend with as well. This will further muddy
  the waters.

68
Chaos Theory and Predictability

•  Singular Values and Vectors
• Is the factor by which initial error will grow
  for infinitesimal errors over a finite time at a
  particular location (singular vectors, as the
  name implies, give the direction).
• Can be numerically estimated using linear theory.
  Singular values/vectors are dependent upon the
  choice of norm they are critically state
  dependent.

69
Chaos Theory and Predictability

•  
• Thus, after some large time interval t, the
  distance e(t) between two points initially
  separated by e(0) is (from slide 48 and 49)
•  

70
Chaos Theory and Predictability

•  
• Thus, if the error doubles, or the ratio
  between one trajectory and another
•  
• and the time to accomplish this is

71
Chaos Theory and Predictability

•  
• This is the basis for stating that the
  predictability of various phenomena is about the
  size of its growth period. For extratropical
  cyclones this is approximately 0.5 3 days.
• For the planetary scale, the time period is
  roughly 10 14 days (evolution of large-scale
  troughs and ridges).

72
Chaos Theory and Predictability

•  
• This is why we say that 10 14 days is the of
  time is the limit of dynamic weather prediction.
• In atmospheric science, we know that this is the
  time period for the evolution of Rossby inertia
  waves, which are the result of the very size and
  rotation rate of the planet earth! (f 2Wsinf)

73
Chaos Theory and Predictability

•  
• Now, the question is, if we know exactly the
  initial state (is it possible to know this?) of
  the atmosphere at some time t, can we make
  perfect forecasts?
• This question is central to the contention that
  the atmosphere contains a certain amount of
  inherent unpredictability.

74
Chaos Theory and Predictability

•  
• Laplace argued that given the entire and precise
  state of the universe at any one instant, the
  entire cosmos could be predicted forever and
  uniquely, by Newtons Laws of motion. He was a
  firm believer in determinism.

75
Chaos Theory and Predictability

•  
• But, can we know the exact initial state? Lets
  revisit Heisenberg!
• Exact solutions do exist, so in theory we can
  find them.
•  
• What we can never do even in principle - is
  specify the exact initial conditions!

76
Chaos Theory and Predictability

• Measurement error and predictability
• If we solve for t (as we did earlier for
  error-doubling)
• Where h is the sum of the positive Lyapunov
  exponents.

77
Chaos Theory and Predictability

• Suppose our uncertainty is at a level of 10-5,
  then
•  
•  
• Now, lets improve the accuracy by 5 orders of
  magnitude, or 10-10
•  

78
Chaos Theory and Predictability

• Then, we should be able to infer that
• Or, this increase in precision only doubles the
  forecast time. Thus, input error, will swell
  very quickly! Should we be pessimistic? ?
•  

79
Chaos Theory and Predictability

• Not a great return on investment! Pessimistic
  about our prospects on forecasting? From a
  selfish standpoint, no because this demonstrates
  that we cannot turn over weather forecasting to
  computers.
•  From a scientific standpoint, no as well,
  because we just need to realize that forecasting
  beyond a certain limit at a certain scale is
  inevitable. As long as we realize the
  limitations, we can make good forecasts.

80
Chaos Theory and Predictability

• One beneficial issue has been stimulated for
  operational meteorology by Chaos Theory, and that
  is how do we express uncertainty in
  forecasts?
• Example

81
Chaos Theory and Predictability

• The End!

82
Chaos Theory and Predictability

• Overtime!
• Fractal Dimension
• Were used to integer whole numbers for
  dimensionality, but the Fractal can have a
  dimensionality that is not a whole number. For
  example, the Koch Snowflake (1904) dimension is
  1.26.

83
Chaos Theory and Predictability

• What? How can you have 1.25 dimensions? But the
  snowflake fills up space more efficiently than a
  smooth curve or line (1-D) and is less efficient
  than an area (2-d). So a dimension between one
  and two captures this concept.

84
Chaos Theory and Predictability

• Example (Sierpinski Gasket, 1915)
• Has a Fractal (Hausdorf) dimension of 1.59

85
Chaos Theory and Predictability

• Hausdorf dimension
• d ln(N(e)) / ln(L) ln(e)
• N(e) is the smallest number of cubes
  (Euclidian shapes) needed to cover the space.
• Here it is 3n or makes 3 copies of itself
  with each iteration.

86
Chaos Theory and Predictability

• The denominator is ln( L / e) where L 1 (full
  space) and e is copy scale factor ((1/2)n length
  of full space with each iteration).
• So we get d n ln(3) / n ln (2) 1.59

87
Chaos Theory and Predictability

• Questions?
• Comments?
• Criticisms?
• lupoa_at_missouri.edu