Complexity

1
Complexity
Advanced Technology
Math
Biology
A coherent foundation is emerging.
After many false starts.
2
Biochemical Network E. Coli Metabolism
Feedback Interactions
Robustness ? Complexity
Supplies Materials Energy
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
3
Manufacturing and metabolism
Collect and import raw materials
Common currencies and building blocks
Complex assembly
Collect and import raw materials
Common currencies and building blocks
Complex assembly
Polymerization and assembly
Taxis and transport
Core metabolism
Autocatalytic and regulatory feedback
4
Nested bowtie and hourglass
Polymerization and assembly
Core metabolism
Transport
Autocatalytic and regulatory feedback
5
Nested bowtie and hourglass
Polymerization and assembly
Core metabolism
Conservation of energy and moiety is a law.
Taxis and transport
Enzymes are modules.
Bowtie architectures is a protocol.
Autocatalytic and regulatory feedback
6
Uncertain
Uncertain
7
Uncertain
Uncertain
8
Uncertain
Uncertain
9
Modules in metabolism
Nucleotides
Catabolism
Carriers and Precursor Metabolites
Sugars
Amino Acids
Fatty acids
Energy and reducing
Cartoon metabolism
Biosynthesis
10
E. Coli all metabolism
External
metabolites
Other inputs
98
105
111
Reduced carriers
Activated carriers
174
180
Precursor
192
metabolites
275
Amino acids
295
membrane lipid
Vitamin and cofactor
Nucleotides
351
359
Lipid and LPS
397
408
Fatty acids and
CO2, Pi, H etc CC
506
514
Other outputs
537
140
282
440
553
608
739
Membrane
Amino acids
Nucleotides
Catabolism
transport
11
3
10
All metabolites
2
10
Modules ? exponential, mix to create ? power law
Cumulative distribution
1
10
Modules
0
10
0
1
2
3
10
10
10
10
Number of Reactions/substrate
12
Why ? power laws in metabolism?

• Simple and biologically meaningful explanation
• Engineering, not physics (and thus probably
  unpublishable)
• Scale-rich and self-dissimilar

13
3
10
All metabolites
2
10
Modules ? exponential, mix to create ? power law
Cumulative distribution
1
10
Modules
0
10
0
1
2
3
10
10
10
10
Number of Reactions/substrate
14
3
Cumulative distribution
10
Why?
2
10
Cumulative distribution
1
10
Modules
0
10
0
1
2
3
10
10
10
10
Number of Reactions/substrate
15
Why heavy tails in modules in metabolism?

• Simple and biologically meaningful explanation
• Engineering, not physics (and thus probably
  unpublishable)
• Scale-rich and self-dissimilar

16
3
Cumulative distribution
10
Modules
2
10
Cumulative distribution
1
10
0
10
0
1
2
3
10
10
10
10
Number of Reactions/substrate
17
3
Cumulative distribution
10
Modules
2
10
Cumulative distribution
1
10
0
10
0
1
2
3
10
10
10
10
Number of Reactions/substrate
18
Is there a nearly universal mechanism that
explains many of the specific instances of power
laws?
19
Something else gets big
Reduce waste fragility
Constraints
HOT
20
Optimize metabolic efficiency and robustness
Highly connected carriers and precursors
21

• Highly structured
• Scale-rich (not scale-free)
• Self-dissimilar (not self-similar)

22
Polymerization and assembly
Core metabolism
Transport
Autocatalytic and regulatory feedback
23
essential 230   nonessential 2373  
unknown 1804   total 4407
http//www.shigen.nig.ac.jp/ecoli/pec
24
assembly
metabolism
transport
Autocatalytic feedback
Regulatory feedback
25
assembly
metabolism
transport
Knockouts often lethal
Autocatalytic feedback
Knockouts often lose robustness, not minimal
functionality
Regulatory feedback
26
Variety of robust functions
FF
FB
Plant
Variety of physical implementations
27
assembly
metabolism
transport
Autocatalytic feedback
If feedback regulation is the dominant protocol,
what are the laws constraining whats possible?
Regulatory feedback
28
The thermostat example
29
Temperature inside buildings
Disturbances
Weather
30
Two main strategies

• Insulate
• Regulate

• Complementary strategies.
• The technical and scientific mainstream is
  surprisingly naïve about regulation.
• Regulation is far more powerful and far more
  dangerous, but not well understood outside of a
  narrow technical discipline.

Weather
31
Heating Ventilation and Air Conditioning (HVAC)
Disturbances
Weather
HVAC
People
32
Weather
Disturbances
Heat/cool
People
Energy
33
Robust yet fragile
Fragile
With heat/AC
No heat/AC
Fragile
Energy
Robust
Energy supply
Disturbances
Components
Robust
34
Thermostat
Fragility to component malfunction
HVAC
With heat/AC
Fragile
Robust
Components
35
Thermostat
Temperature
Fragility to component malfunction
HVAC
With heat/AC
Fragile
Components
36
Universal tradeoffs in complex networks
Fragile
Energy and Materials
Disturbances and Components
Disturbances and Components
Robust
37
Greater complexity
Fragile
Energy and Materials
Robust
38
Diseases of complexity
Fragile

• Parasites
• Cancer
• Epidemics
• Auto-immune disease

Complex development Regeneration/renewal Complex
societies Immune response
Uncertainty
Robust
39
logS
?
We have a proof of this. More to come.
X0
X1
Xi
Xn


Error
Xn1
40
This is a cartoon. We have no proof of this.
Yet.
Fragile

• Parasites
• Cancer
• Epidemics
• Auto-immune disease

Complex development Regeneration/renewal Complex
societies Immune response
Uncertainty
Robust
41

• Are these tradeoff necessary?
• Or are they frozen accidents?
• Why should biologists care?

Fragile
Immune response Development Regeneration renewal S
ocieties
Thermostat
Parasites Cancer Epidemics Auto-immune disease
Temperature
HVAC
Robust
Uncertainty
42
Modeling robust yet fragile systems
May need great detail here
Fragile
And much less detail here.
Uncertainty
Robust
43
Fragile
Robust (fragile) to perturbations in components
and environment ? Robust (fragile) to errors and
simplifications in modeling
More detail.
Required model complexity
Less detail.
Uncertainty
Robust
44

• Complexity of system evolution is driven by
  fragilities
• Complexity of experiments, modeling, and
  inference are also driven by fragilities

Fragile
More detail.
Required model complexity
Less detail.
Uncertainty
Robust
45
Airbags
46

• In development
• drive-by-wire
• steering/traction control
• collision avoidance

47
Cascading events in car crashes
Normal
Danger
Crash
Contact w/car
Trauma
Barriers in state space
48
Full state space
Vehicle
Airbag
49
Full state space
Vehicle
Normal
Sense/ Deploy
Contact w/bag
Desired
Trauma
Worse
Bad
Airbag
50
Full state space
Robust
Yet Fragile
51
Robust yet fragile

• Robust to uncertainties
• that are common,
• the system was designed for, or
• has evolved to handle,
• yet potentially fragile otherwise
• This is the most important feature of complex
  systems (the essence of HOT).

52
Robust yet fragile

• Complexity, when well-designed, shifts
  fragilities from outside (the environment) to the
  inside (components).
• This can be a huge win when done correctly.
• Managing this complexity and fragility is the
  central challenge for all aspects of the future
  of technology, not just the resilience of
  technological system.
• Understanding how biology does it is the key to
  the future of systems biology and medicine.

53
Robust yet fragile

• The right amount of feedback control is the
  single most powerful strategy for producing
  robust systems.
• Too much feedback control is also the most
  dangerous threat to robustness. More control than
  necessary is not better, it is disastrous.
• Arguably, this is currently both the most
  critical and the most poorly understood issue in
  science and technology.

54
Humans supply most feedback control in
transportation
Normal
Danger
Crash
Contact w/car
Trauma
Lanes Laws Lights Ramps
Collision avoidance Anti-lock brakes
Heavy metal Seat belts Airbags
Helmets
55
Imagine a fully automated system
Normal
Danger
Crash
Contact w/car
Trauma
Lanes Laws Lights Ramps
Collision avoidance

• Internally unimaginably more complex.
• Superficially much simpler?
• Biological systems are fully automated

56
Biochemical Network E. Coli Metabolism
Feedback Interactions
Robustness ? Complexity
Supplies Materials Energy
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
57
Biochemical Network E. Coli Metabolism
Enzyme
Mass Transfer in Metabolism
Metabolite
From Adam Arkin
from EcoCYC by Peter Karp
58
Biochemical Network E. Coli Metabolism
Feedback Interactions
From Adam Arkin
from EcoCYC by Peter Karp
59
Autocatalysis
Regulation
Enzyme
Metabolite
from EcoCYC by Peter Karp
60
Autocatalysis
Regulation
Enzyme
Autocatalysis
Enzyme
Regulation
61
Enzyme
Metabolite
62
Yi, Ingalls, Goncalves, Sauro
Product inhibition
perturbation
63
1.05
Step increase in demand for ATP.
ATP
1
h 3
0.95
h 2
0.9
h 1
0.85
h 0
0.8
0
5
10
15
20
Time (minutes)
h 0 1 2 3
64
1.05
1
h 3
0.95
h 2
ATP
0.9
h 1
0.85
h 0
0.8
0
5
10
15
20
h 0 1 2 3
Time (minutes)
0.8
h 3
0.6
0.4
h 2
0.2
h 0
Log(Sn/S0)
0
h 1
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
65
1.05
Ideal response
ATP
1
h 3
0.95
Time response
0.9
0.85
h 0
0.8
0
5
10
15
20
Time (minutes)
0.8
h 3
0.6
Spectrum
0.4
Normalized logarithm of spectrum of error.
0.2
h 0
Log(Sn/S0)
0
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
66
1.05
Ideal response
ATP
1
h 3
0.95
Time response
0.9
0.85
h 0
0.8
0
5
10
15
20
Time (minutes)
0.8
h 3
0.6
Spectrum
0.4
Normalized spectrum of error.
0.2
h 0
Log(Sn/S0)
0
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
67
h 3
h 2
h 1
h 0
Time
0
5
10
15
20
0.8
h 3
0.6
0.4
h 2
0.2
h 0
Log(Sn/S0)
0
h 1
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
68
Transients, Oscillations
0.8
h 3
0.6
Tighter steady-state regulation
0.4
h 2
0.2
h 0
Log(Sn/S0)
0
h 1
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
69
Transients, Oscillations
logS
?
Biological complexity is dominated by the
evolution of mechanisms to more finely tune this
robustness/fragility tradeoff.
Tighter regulation
70
h 3
h 2
h 1
h 0
Time
0
5
10
15
20
0.8
h 3
0.6
0.4
h 2
0.2
h 0
Log(Sn/S0)
0
h 1
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
71


-
72
d(k)
e(k)
u(k)
e(k) d(k) - u(k)
-
73
For simplicity, assume d, u, and e are finite
sequences.
d(k)
u(k)
k
e(k)
k
Then the discrete Fourier transform D, U, and E
are polynomials in the transform variable z.
If we set z ei? , ? ? 0,?? then X(w) measures
the frequency content of x at frequency w.
74

• Denote by zk the complex zeros for z gt 1 of
  X(z)
• Jensens theorem

Proof Contour integral
75
A useful measure of performance is in terms of
the sensitivity function S(z) defined by Bode as
If we set z ei? , ? ? 0,?? then S(w)
measures how well C does at each frequency. (If C
is linear then S is independent of d, but in
general S depends on d.) It is convenient to
study log S(w)
76

• Denote by ek and xk the complex zeros for z
  gt 1 of E(z) and D(z), respectively. Then

Proof Follows directly from Jensens formula.
If d is chosen so that D(z) has no zeros in z gt
1 (this is an open set), then
77
Assume u is a causal function of x.
x(k)
u(k-1)
k
78
Disturbance
Model
badness
-
79
Predator
trauma
-
80
Model
trauma
-

• Auxiliary measurements and a model allows the
  controller to use correlations between space and
  time to effectively create an acausal
  controller.
• Seeing a predator in the distance now suggests a
  potential for trauma here in the near future
• Provided you have a model that can connect the
  visual information with near future increased
  risks.

81
Model
trauma
-

• These tradeoffs seem to be the greatest driver
  for high complexity in biology and technology as
  well.
• It creates new fragilities as the system
  potentially becomes highly fragile to errors in
  internal models.

82
Disturbance
Unstable plants makes things worse
Model
badness
-
83
Autocatalysis
Regulation
Enzyme
Metabolite
from EcoCYC by Peter Karp
84
Autocatalysis
Regulation
Enzyme
Metabolite
85
Regulatory feedback only
h 3
h 2
h 1
h 0
Time
0
5
10
15
20
0.8
h 3
0.6
0.4
h 2
0.2
h 0
Log(Sn/S0)
0
h 1
-0.2
-0.4
-0.6
-0.8
0
2
4
6
8
10
Frequency
86
Add autocatalytic feedback
87
Add more regulator feedback
88
More instability aggravates
89
Control demo
90
Biological network mechanisms

• Negative feedback for regulation
• Feedforward control
• Protocol stacks and modularity
• Positive feedback to create switches and memory
  (digital systems)
• Autocatalysis (positive feedback in energy and
  materials)
• Oscillators for timing
• Randomized strategies
• Coding

91
Autocatalysis in biology

• Biological systems have autocatalytic feedback
  everywhere
• What are the consequences for regulatory
  feedback?
• Claim Autocatalytic systems have intrinsic and
  unavoidable excess fragilities
• The side effects of oscillations should be
  ubiquitous, though perhaps masked by lack of cell
  synchronization

92
An apparent paradox
Mutation
Selection
Darwinian evolution uses selection on random
mutations to create complexity.
93
Thus stabilizing forward flight.
At the expense of extra weight and drag.
94
For minimum weight drag, (and other performance
issues) eliminate fuselage and tail.
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
(No Transcript)
99
Boeing advance commercial aircraft design
100
Why do we love building robust systems from
highly uncertain and unstable components?
Robust control theory tells us why.
101
Sensors
Actuators
Control
Vehicle
Size ? Complexity
102

• Cheap
• Simple

• Unstable airframe
• Expensive sensors, actuators and computers
• More robust overall
• More efficient overall
• New fragilities

103
Variety of Ligands Receptors
Hourglass

• Hourglasses organize
• regulatory and signaling systems
• horizontal and vertical decompositions

Intermediates
Variety of responses
104
Beyond reductionism

• Biology must obey the laws of chemistry and
  physics
• The modules of biology are reducible to the
  modules of chemistry and physics
• The protocols and laws of biology are not
  reducible to those of chemistry and physics
• Driven by waste fragility tradeoffs

105
Is robust yet fragile necessary?
Isbowtie and hourglass necessary?
Is all of biology necessary or is something a
frozen accident?
106
Conjectures

• Much of what appears an historical accident or
  poor design is due to our lack of understanding
  of robustness requirements
• Eukaryotes are presumably more of a hack than
  prokaryotes
• But surely there are strong counterexamples to
  the notion that most of biology is extremely
  well-designed from an engineering perspective?

107
Savageau and demand theory

• A potential source of examples of frozen
  accidents might be activation versus repression
  in gene regulation
• This will give us an opportunity to look at the
  interplay between robustness and efficiency

108
High and low demand for gene products
High
Low
Time

• High demand usually high
• Low demand usually low
• Constitutive always high

109
Regulation of gene expression
Logically, they have equivalent functionality.
110
Regulation of gene expression
111
Correlations?
Is it random (accidental) or is there some
design?
Logically they are equivalent, what about
efficiency and robustness?
112
Regulation of gene expression
Since both high and low demand, and both
activators and repressors exist, we can eliminate
these cases.
113
Regulation of gene expression
Random
Which leaves three possibilities.
Can we use efficiency and robustness to predict
what we will see?
114
Regulation of gene expression
Random

• This case says that since they are logically
  equivalent, there should be no correlation.

115

• Efficiency Usually expressed in high quantities
• Robustness Small mutations have immediately bad
  phenotype

Efficiency Usually expressed in low
quantities Robustness Small mutations have less
immediate phenotype
116
Pop Quiz
Random

• To see what weve learned
• To see if I can teach
• If youve read Savageau, you are excused from the
  quiz

117

• Logically equivalent so no correlation

Random

• Efficiency Usually expressed in high quantities
• Robustness Small mutations have immediate
  phenotype

Efficiency Usually expressed in low
quantities Robustness Small mutations less
immediate phenotype
118

• Logically equivalent so no correlation

Random

• Efficiency Usually expressed in high quantities
• Robustness Small mutations have immediate
  phenotype

Efficiency Usually expressed in low
quantities Robustness Small mutations less
immediate phenotype
119

• Logically equivalent so no correlation

Random

• Efficiency Usually expressed in high quantities
• Robustness Small mutations have immediate
  phenotype

Efficiency Usually expressed in low
quantities Robustness Small mutations less
immediate phenotype
These are bogus arguments.
120
Random

• ? hundred examples
• No convincing counterexamples (yet)
• Both simple and complex examples

121
Savageau and demand theory

• This was the most trivial (and now quite old)
  example of demand theory
• A rich and deep treatment of biological necessity
• Mathematically rigorous, not just-so stories

122
Biological signal transductionhourglass
Variety of Ligands Receptors
Intermediates
Variety of responses
123
Taylor, Zhulin, Johnson
124
(No Transcript)
125
More necessity and robustness

• Integral feedback and signal transduction
  (bacterial chemotaxis, G protein) (Yi, Huang,
  Simon)

126

• Variety of receptors
• Shared cytoplasmic domains and A-W complex
• CheY universal signal to motor
• Facilitates robustness and evolvability
• Robust chemotaxis with multiple ligands
• Can easily evolve new receptors

Ligands, Receptors
CheA-CheW
CheY-Motor
Attractants, Repellants
127

• More than 50 such two component systems in E.
  Coli
• All use the same protocol
• - Histidine autokinase transmitter
• - Aspartyl phospho-acceptor receiver
• Huge variety of receptors and responses, mostly
  gene regulation

Variety of Ligands Receptors
Transmitter
Receiver
Variety of responses
128
Variety of Ligands Receptors
Transmitter
Receiver
Variety of responses
129

• Ubiquitous eukaryote protocol
• Huge variety of receptors
• Large number of G-proteins grouped into similar
  groups
• Handful of primary targets
• Huge variety of downstream responses

130
Various functionality
Digital
Analog substrate
131
Applications
Software Hardware
Modern Computing
Operating System
Hardware
132
Applications
Software Hardware
Modern Computing
Operating System
Hardware
133
Applications
Operating System
Hardware
134
The Internet Protocol Stack Hourglass
135
The Internet hourglass
Applications
Web
FTP
Mail
News
Video
Audio
ping
napster
Transport protocols
TCP
SCTP
UDP
ICMP
IP
Ethernet
802.11
Satellite
Optical
Power lines
Bluetooth
ATM
Link technologies
136
The Internet hourglass
Everything on IP
IP
From Hari Balakrishnan
137
Routers
Hosts
138
Routers
Links
Sources
Hosts
139
Links
Sources
140
Links
Sources
141
Network protocols.
Files
HTTP
TCP
IP
packets
packets
packets
packets
packets
packets
Links
Sources
142
Network protocols.
HTTP
Hidden from the user
Sources
143
Network protocols.
HTTP
TCP
Vertical decomposition Protocol Stack
IP
Routing Provisioning
144
Network protocols.
HTTP
TCP
IP
Horizontal decomposition Each level is
decentralized and asynchronous
Routing Provisioning
145

• Entirely different from the telephone system,
  although the parts are identical (VLSI, copper,
  and fiber)
• The Internet is much more like biology and
  relies on feedback regulation at every level.
• Only recently has a coherent theory of the
  Internet started to emerge.
• Steven Low is leading a project at Caltech to
  develop new theory and redesign and deploy new
  protocols.

HTTP
TCP
Vertical decomposition
IP
Horizontal decomposition
Routing Provisioning
146
Clothing
147
The clothing hourglass
Garments
Dress
Shirt
Slacks
Lingerie
Coat
Scarf
Tie
Wool
Cotton
Nylon
Silk
Polyester
Rayon
Material technologies
148
T-Shirt
Jacket
Shirt
Tie
Boxers
Shoes
Coat
Slacks
Socks
149
T-Shirt
Jacket
Shirt
Tie
Boxers
Shoes
Coat
Slacks
Socks
Silk
Wool
Nylon
Rayon
Cotton
Polyester
150
Horizontal decomposition
Garments
Dress
Shirt
Slacks
Lingerie
Coat
Scarf
Tie
Sewing
Vertical decomposition
The hourglass
Cloth
Wool
Cotton
Nylon
Silk
Polyester
Rayon
Horizontal decomposition
Material technologies
151
Money
152
Consumers
Barter
Commodities
153
Consumers
Barter
Commodities
154
(No Transcript)
155
New fragilities

• Theft, counterfeiting, fraud, and creative
  accounting are now possible
• Need complex legal infrastructure to protect
• The beginning of a growing complexity-fragility
  spiral

156
(No Transcript)
157
Consumers
Investors
It is now possible to have a global financial
crisis which is completely independent of the
original motivation for money.
Money
Markets, Insitutions
Commodities
Investments
158
Variety of Ligands Receptors
Bowties and hourglasses are universal
architectures (or protocols suites).
Intermediates
Variety of responses
Nutrients
Products
159
Bowtie

• Bowties organize
• efficient and robust flows
• of materials and energy
• in production and manufacturing
• of complex systems and their components.

Nutrients
Products
160
Variety of Ligands Receptors
Hourglass

• Hourglasses organize
• regulatory and signaling systems
• horizontal and vertical decompositions

Intermediates
Variety of responses
161
Uncertain
Uncertain
162
Nested bowtie and hourglass
Polymerization and assembly
Core metabolism
Transport
Autocatalytic and regulatory feedback
163
Uncertain
Uncertain
164
Uncertain
Variety of Ligands Receptors
Robust and highly evolvable
Fragile and hard to change
Intermediates
Variety of responses
Uncertain