Complexity and robustness

1
(No Transcript)
2
Complexity and robustness
John Doyle Control and Dynamical Systems Caltech
3
Complexity?
4
Complex adaptive systems?
new science of complexity
Artificial, emergent, adaptive, etc etc
Edge-of-chaos (EOC)
Self-organized criticality (SOC)
Neuro-fuzzy-genetic expert agents
Buzzword science
5
(No Transcript)
6
Dynamical systems
Control, optimization, identification
Uncertainty and robustness
Focus
Statistical physics
7
Robustness and uncertainty
Sensitive
Error, sensitivity
Robust
Meteor impact
speech
Types of uncertainty
8
Complex systems
Extreme robustness
Sensitive
Error, sensitivity
Robust
Types of uncertainty
9
Selected examples

• Software (Y2K, embedded systems, Ariane 5, Denver
  airport baggage handling, IRS upgrade)
• Networks (internet, power, transportation, phone,
  financial, food, water, waste)
• Computer-aided design and manufacturing (VLSI,
  Mechanical, Virtual engineering, simulation based
  design)
• Fluids and continuum mechanics (Hydrodynamic
  stability, shear flows, boundary layer controls,
  flutter, weather, climate)
• Physics, chemistry, materials (Quantum systems
  and computing, laser control of molecular
  dynamic, statistical and nonequilibrium physics
  of designed systems, composite materials, smart
  materials)

10
Selected examples (cont.)

• Biology (DNA networks, signal transduction
  networks, neural networks, protein folding,
  organism behavior, macroevolution)
• Medicine (stem cells, cancer, multiorgan failure,
  autoimmune disease, AIDS, resistant parasites,
  organ and tissue regeneration)
• Ecology and environment (Specie extinction and
  biodiversity, forestry and resource management,
  sustainable agriculture and energy)
• Military "systems of systems"
• Social systems and complexity (sustainable
  societies, economics and finance, political
  systems)

11

• In these examples, robustness is more important
  than
• materials
• energy
• entropy
• information
• computation
• They have extreme robustness robust yet
  fragile.
• Developing new theories of complexity that focus
  on robustness

12
Conservation of robustness
Error, sensitivity
is balanced by
Types of uncertainty
13
Focus on

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

14
Start with some data US Power outages 1984-1997
15
1
10
Frequency (per year) of outages gt N
0
10
US Power outages 1984-1997
-1
10
-2
10
4
5
6
7
10
10
10
10
N of customers affected by outage
16
1
10
Power laws
0
10
-1
10
-2
10
4
5
6
7
10
10
10
10
17
On average, once every
1
10
0
10
-1
10
-2
10
4
5
6
7
10
10
10
10
there will be an outage of
18
Size of events vs. frequency
p ? s-a
log(probability)
1 lt a lt 3
log(size)
19
Incomplete data
Heavy tails
1
10
Frequency (per year) of outages gt N
0
10
Gaussian
-1
10
Gaussians have vanishing probability of large
events
-2
10
4
5
6
7
10
10
10
10
N of customers affected by outage
20
1
10
1984-1997
Frequency (per year) of outages gt N
0
10
-1
10
-2
10
4
5
6
7
10
10
10
10
N of customers affected by outage
21
(No Transcript)
22
Power laws are ubiquitous.

• Cascading failures in power grids
• Freeway traffic jams
• Deaths and due to disasters
• Specie extinction
• Practically every quantity in the Internet
• The world is extremely non-Gaussian.

23
Are power laws surprising?

• Central limit theorem (law of large numbers) says
  we should expect
• Gaussian distributions
• (normal, bell curves,).
• Right?

24
Actually

• The full central limit theorem says we should
  expect Gaussians or power laws.

25
However

• We still need to understand how heavy tails
  arise in the first place.

26

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

Joint work with Jean Carlson Physics UCSB
27
The simplest possible toy model of cascading
failure.
Square site percolation or simplified forest
fire model.
28
Connected clusters
connected
not connected
29
A spark that hits a cluster causes loss of that
cluster.
30
Assume one randomly located spark
yield density - loss
(average)
31
Think of (toy) forest fires.
yield density - loss
(average)
32
1
0.9
critical point
(avg.) yield
0.8
0.7
0.6
0.5
N100
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
density
33
Reductionist science.. boring.
isolated
34
(No Transcript)
35
Critical point
36
1
limit N ? ?
0.9
(avg.) yield
0.8
critical point
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
density
37
This picture is very generic.
criticality
38
Thermodynamics and statistical mechanics
Mean field theory
Renormalization group ? Universality classes
Power laws Fractals Self-similarity
hallmarksor signatures of criticality
39
Fractal and self-similar
Criticality
40
(No Transcript)
41
Criticality
Power laws
42
2
10
1
10
Power laws only at the critical point
0
10
-1
10
0
1
2
3
4
10
10
10
10
10
43
Characteristics at criticality depend only on
connectivity.
Higher dimensions
Other lattices
44
The critical density goes down with dimension.
Dimension Density 2 .592746 3 .3116 4 .197
5 .107 ... ? 0
45
This phase transition is universal.
46
Self-organized criticality (SOC) dynamics have
critical point as global attractor
Simpler explanation systems that reward yield
will naturally evolve to critical point.
47
Life, networks, the brain, the universe and
everything are at criticality or the edge of
chaos.
Does anyone really believe this?
48
Would you design a system this way?
49
Maybe random networks arent so great
50
High yields
51
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
52
tolerant
critical
isolated
53
Why power laws?
Optimize Yield
Almost any distribution of sparks
Power law distribution of events
54
Numerical Example 32x32 grid
1
0.9
optimized
0.8
0.7
grid
0.6
yield
0.5
0.4
random
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
density
55
Probability distribution (tail of normal)
x12.(- ((1 (1n)/n)/.3 ).2 ) x22.(- ((.5
(1n)/n)/.2 ).2 )
56
Probability distribution (tail of normal)
2.9529e-016
0.1902
5
10
15
20
25
30
5
10
15
20
25
30
2.8655e-011
4.4486e-026
57
Optimal evolved density 0.9678 yield
0.9625
Small events likely
Evolved add one site at a time to maximize
incremental (local) yield
At density ? explores only
choices out of a possible
Very local and limited optimization, yet still
gives very high yields.
58
High yields.
59
Optimized grid
Small events likely
density0.8496 yield 0.7752
60
Optimized grid
density0.8496 yield 0.7752
1
0.9
High yields.
0.8
0.7
grid
0.6
0.5
0.4
random
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
61
0
10
critical
Cumulative distributions
-1
10
evolved
-2
10
critical density is .55 (which maximizes yield)
grid
-3
10
-4
10
0
1
2
3
10
10
10
10
62
0
10
optimal density 0.9678 yield 0.9625
-2
10
-4
10
.9
Evolution
-6
10
.8
This shows various stages on the way to the
optimal. Density is shown.
-8
10
.7
-10
10
-12
10
0
1
2
3
10
10
10
10
63
evolving (yield ?density)
Density.8
Density.9
64
Power laws are inevitable.
Good (small events)
Bad (large events)
Gaussian
65
This source of power laws is quite universal.
Optimize Yield
Almost any distribution of sparks
Power law distribution of events
66
Tolerance is very different from criticality.

• Mechanism generating power laws.
• Higher densities.
• Higher yields, more robust to sparks.
• Highly structured, even stylized.
• Nongeneric, wont arise due to random
  fluctuations.
• Not fractal, not self-similar, not
  renormalizable.
• Extremely sensitive to small perturbations that
  were not designed for, changes in the rules.

67
Extreme robustness and extreme hypersensitivity.
Small flaws
68
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
69
1
0.9
0.8
0.7
0.6
0.5
0.4
0.3
0.2
0.1
0
0
0.2
0.4
0.6
0.8
1
70
Features of tolerance that involve totally new
questions.

• New fundamental principles and conservation
  laws.
• Dominated by robustness tradeoffs.
• Optimization of yield in the presence of
  uncertainty
• Highly Optimized Tolerance (H.O.T.)

71
These four regimes are all extremely different.
72
In this view, power laws occur only at one point.
critical
isolated
73
H.O.T.
But HOT systems have power laws at all densities.
74
These 3 transitions are all extremely different.
75
flaws
Sensitivity to
76
Important robustness conservation principles
The net amount of positive and negative
feedback in any causal system is equal.
Biological systems seem to cope with this in
especially creative ways.
77
Claim This picture is key to understanding
everything from networks to turbulence to
developmental biology.
yield
Intensity, interconnectedness, (pressure,
density, gene count)
78
Our scientific foundation is based on only this
part of the picture, and as a consequence...
Approaches out here often are mystical and
sloppy.
yield
Intensity, interconnectedness, (pressure,
density, gene count)
79
severe robustness tradeoffs
The HOT state is dominated by
and cascading failure
80
Comparison
Characteristic Critical HOT Densities
Moderate High Yields Low-moderate
High Robustness Generic, moderate High to
designed-for uncertainty Low to
flaws and unanticipated perturb. Configurat
ions Generic, fractal Structured,
stylized Large events Cascading, fractal
Cascading, structured External behavior
Complex Nominally simple Internally Simple
Complex Statistics Power laws Power laws
81
Examples of criticality?
(Systems exhibiting features of criticality,
self-organized criticality (SOC), or
edge-of-chaos (EOC))

• Second-order phase transitions
• Some kinds of sand piles and rice piles
• Earthquakes? (probably not)

82
Examples of H.O.T.?
Note The toy forest fire is emphatically not a
model for any specific system, even forests.
However, it is remarkable how many complex
systems have most or all of the HOT features.
83
Selected examples

• Software (Y2K, embedded systems, Ariane 5, Denver
  airport baggage handling, IRS upgrade)
• Networks (internet, power, transportation, phone,
  financial, food, water, waste)
• Computer-aided design and manufacturing (VLSI,
  Mechanical, Virtual engineering, simulation based
  design)
• Fluids and continuum mechanics (Hydrodynamic
  stability, shear flows, boundary layer controls,
  flutter, weather, climate)

84

• Biology (DNA networks, signal transduction
  networks, neural networks, protein folding,
  organism behavior, macroevolution)
• Medicine (stem cells, cancer, multiorgan failure,
  autoimmune disease, AIDS, resistant parasites,
  organ and tissue regeneration)
• Ecology and environment (Specie extinction and
  biodiversity, forestry and resource management,
  sustainable agriculture and energy)
• Social systems and complexity (sustainable
  societies, economics and finance, political
  systems)

85

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

86
Ecosystems and extinction

• 99.9 of all species which have ever existed are
  now extinct
• Extinction events have heavy tails.
• 5 major extinction events and numerous smaller
  ones.
• Currently in the sixth major extinction with the
  rate increasing orders of magnitude in the last
  10,000 years.

87
Ecosystems and extinction

• There is an ongoing debate about the cause of
  these extinctions.
• Biologists now agree that they are due to
  catastrophic external events
• meteor impacts
• large scale geophysical phenomena.
• Advocates of SOC/EOC argue instead that they are
  due to SOC/EOC co-evolutionary biological
  phenomena.
• But while extinctions may be triggered by
  exogenous events, the distribution of extinctions
  for a given disturbance is a fairly structured,
  deterministic, and even predictable process.

88
Habitats

• terrestrial vs. marine
• island vs. continental
• tropical vs nontropical

89
Specialization

• Within a habitat, specialization offers
  short-term benefits.
• Specialization consistently correlates with
  extinction risk in large extinctions.
• For example, large body size has been a risk
  factor in all major extinctions (although not
  always in marine animals).
• However, in the smaller late Eocene extinctions,
  large-bodied mammal species were not selected
  against.
• This highlights the role of external causes the
  late Eocene extinctions were generally related to
  global cooling, which tends to favor large body
  size.

90
Evolution and extinction
91
Ecosystems
HOT
92
Ecosystems and extinction
Characteristic Critical HOT Densities
Moderate High Yields Low-moderate
High Robustness Generic, moderate High to
designed-for uncertainty Low to
flaws. Configurations Generic, fractal
Structured, stylized Large events Cascading,
fractal Cascading, structured External behavior
Complex Nominally simple Internally Simple
Complex Statistics Power laws Power laws
93

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

94
Turbulence in shear flows
wings
channels
Turbulence is the graveyard of theories. Hans
Liepmann Caltech
pipes
95
velocity
high
low
equilibrium
periodic
chaotic
96
random pipe
97
bifurcation
laminar
flow (average speed)
turbulent
pressure (drop)
98
velocity
high
low
equilibrium
periodic
chaotic
99
Typical flow
flow
pressure
100
wings
Streamline
channels
pipes
101
theory
laminar
log(flow)
experiment
turbulent
Random pipe
log(pressure)
102
This transition is extremely delicate (and
controversial).
log(flow)
It can be promoted (or delayed!) with tiny
perturbations.
log(pressure)
103
Transition to turbulence is promoted (occurs at
lower speeds) by
Surface roughness Inlet distortions Vibrations The
rmodynamic fluctuations? Non-Newtonian effects?
104
None of which makes much difference in this case.
105
Shark skin delays transition to turbulence
106
log(flow)
It can be reduced with small amounts of polymers.
log(pressure)
107
spanwise
108
high-speed region
From Kline
109
spanwise
110
streamwise
111
vortices
flow
112
the structure results now seem to provide, at
long last, a reasonably complete picture of how
turbulence is produced and maintained in the
boundary layer and of the major eddies in the
various regions of the layer. In nearly every
other case in physics such increased knowledge
has translated into improved models for
computation. That has not been the case in
turbulent boundary layers.
S.J. Kline Stanford
113
However

• We still need to understand how the near-wall
  streamwise vorticity arises in the first place.

114
Consider infinitesimal normal velocity
fluctuations near the wall, due to

• Surface roughness
• Inlet distortions
• Vibrations
• Thermodynamic fluctuations?
• Non-Newtonian effects?
• .

v
115
(No Transcript)
116
Theorem (Bamieh and Dahleh)
The energy growth in vorticity due to velocity
perturbations scales with (mean flow speed)3
vorticity
So at high speeds and pressure drops, the energy
amplification blows up (but there is no
bifurcation.)
3-D flow
shear
Lots of previous arguments and evidence for this
from many researchers (Farrell, Trefethen,
Lorenz, )
? v
117
What does the resulting flow look like?
118
(No Transcript)
119
Turbulence
flow
HOT
pressure drop
120
Turbulence is fundamentally a robustness problem!

• Depending on who you ask, this is
• Complete nonsense
• Interesting, but irrelevant
• The most important results in the history of the
  subject

121

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

122
Building complexity computers and networks
High-level functionality
Layers of rules and protocols
Physical implementation
123
Early computing.
Machine code
High-level functionality
Layers of rules and protocols
Logic
Transistors
Physical implementation
124
User interface
Modern computation.
Applications
High-level functionality
Applications
Layers of rules and protocols
OS
Computer
Board
VLSI
Physical implementation
125
User interface
VLSI design
Instructions
Applications
Logic
Applications
Topology
OS
Geometry
Computer
Timing
Board
Fabrication
VLSI
Silicon
126
Generic versus designed
Instructions
Climate
Logic
Weather
Topology
Navier-Stokes
Keep only sets of measure zero.
Throw away sets of measure zero.
Geometry
Boltzmann dist
Timing
particle dynamics
Fabrication
Quantum mech.
Silicon
???
127
Network protocols.
Routers
128
Network protocols.
129
Networks
demand
HOT
throughput
130
Power laws in networks?
Power laws are everywhere in network. Its
controversial how important this is.
Most power laws in networks can be traced to
power laws in web sites and other files.
Why are file sizes on websites power laws?
131
A simple model for power laws?
Optimize Throughput
Power law distribution of files
The same mechanism!
132
Suppose you have documents to put on the web.
133
Suppose there is a variable probability of hits.
134
Then optimizing network throughput will lead to
power law file size distributions.
135
(No Transcript)
136
A simple model for power laws?
Optimize Throughput
Power law distribution of files
The same mechanism!
137
Networks have all the HOT features
Characteristic Critical HOT Densities
Moderate High Yields Low-moderate
High Robustness Generic, moderate High to
designed-for uncertainty Low to
flaws. Configurations Generic, fractal
Structured, stylized Large events Cascading,
fractal Cascading, structured External behavior
Complex Nominally simple Internally Simple
Complex Statistics Power laws Power laws
138

• Statistical physics
• Power systems
• Ecosystems and extinction
• Turbulence
• Computers and Internet
• Software
• Stem cells
• Bacterial chemotaxis

139
Advanced software devours complex problems
Software-intensive systems are the most extreme.
140
(No Transcript)