John Doyle

1
Theory and Biological Networks

• John Doyle
• Control and Dynamical Systems
• Caltech

2
Collaboratorsand contributors(partial list)

• AfCS Simon, Sternberg, Arkin,
• Biology Csete,Yi, Borisuk, Bolouri, Kitano,
  Kurata, Khammash, El-Samad, Gross,
• Theory Parrilo, Paganini, Carlson, Lall,
  Barahona, DAndrea,
• Web/Internet Low, Effros, Zhu,Yu, Chandy,
  Willinger,
• Turbulence Bamieh, Dahleh, Gharib, Marsden,
  Bobba,
• Physics Mabuchi, Doherty, Marsden,
  Asimakapoulos,
• Engineering CAD Ortiz, Murray, Schroder,
  Burdick, Barr,
• Disturbance ecology Moritz, Carlson, Robert,
• Power systems Verghese, Lesieutre,
• Finance Primbs, Yamada, Giannelli,
• and casts of thousands

Caltech faculty
Other Caltech
Other
3
Biochemical Network E. Coli Metabolism
Regulatory Interactions
Complexity ? Robustness
Supplies Materials Energy
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
4
Biochemical Network E. Coli Metabolism

• Constraints
• Mass balance
• Energy balance
• Entropy

from EcoCYC by Peter Karp
5
(No Transcript)
6
500Kv
350Kv
250Kv
7

• Constraints
• Mass balance
• Energy balance
• Entropy

8
Biochemical Network E. Coli Metabolism
Regulatory Interactions
Constraints?
Supplies Robustness
From Adam Arkin
from EcoCYC by Peter Karp
9
Robustness
Complexity
10
Complexity and robustness

• Complexity phenotype robust, yet fragile
• Complexity genotype internally complicated
• New theoretical framework HOT (Highly optimized
  tolerance)
• Applies to biological and technological systems
• Pre-technology simple tools
• Primitive technologies use simple strategies to
  build fragile machines from precision parts.
• Advanced technologies use complicated
  architectures to create robust systems from
  sloppy components
• but are also vulnerable to cascading failures

11
There are about 13,000 commercial aircraft
worldwide.
A Boeing 777 has 150,000 different components and
a total of 3,000,000, some with millions of
subparts.
During flight test, a partial system state is
saved at the rate of 1e8 bits (100 Mbits) per
second.
The human genome can be stored with 1e10 bits (lt
2 CDs).
12

• Forward engineering of 777 cost gt1B in software
  infrastructure alone
• Systems cost more than structures or
  aerodynamics
• Aeronautical, mechanical and electrical
  engineering and computer science

13
Robustness

• Robust to large scale atmospheric disturbances,
  variations in cargo loads and fuels, turbulent
  boundary layers, inhomogeneities and aging of
  materials, etc
• ...but could be catastrophically disabled by
  microscopic alterations in a handful of
  components (eg. 4 carefully chosen transistors).
• This is, fortunately, very unlikely. (Less likely
  than, say, large meteor impacts.)

14
Robust, yet fragile phenotype

• Robust to large variations in environment and
  component parts (reliable, insensitive,
  resilient, evolvable, simple, scaleable,
  verifiable, ...)
• Fragile, often catastrophically so, to cascading
  failures events (sensitive, brittle,...)
• Cascading failures can be initiated by small
  perturbations (Cryptic mutations,viruses and
  other infectious agents, exotic species, )
• Greater pheno-complexity more extreme robust,
  yet fragile

15
Robust, yet fragile phenotype

• Cascading failures can be initiated by small
  perturbations (Cryptic mutations,viruses and
  other infectious agents, exotic species, )
• In many complex systems, the size of cascading
  failure events are often unrelated to the size of
  the initiating perturbations
• Fragility is interesting when it does not arise
  because of large perturbations, but catastrophic
  responses to small variations

16
Complicated genotype

• Robustness is achieved by building barriers to
  cascading failures
• This often requires complicated internal
  structure, hierarchies, self-dissimilarity,
  layers of feedback, signaling, regulation,
  computation, protocols, ...
• Greater geno-complexity more parts, more
  structure

17
Robustness of HOT systems
Fragile
Fragile (to unknown or rare perturbations)
Robust (to known and designed-for uncertainties)
Uncertainties
Robust
18
Robustness of HOT systems
Fragile
Humans
Chess
Meteors
Robust
19
Robustness is a conserved quantity
Fragile
Chess
Meteors
Robust
20
Robustness of HOT systems
Fragile
Humans
Archaea
Chess
Meteors
Machines
Robust
21
Diseases of complexity
Fragile

• Cancer
• Epidemics
• Viral infections
• Auto-immune disease

Uncertainty
Robust
22
Modeling complex systems
May need great detail here
Fragile
And much less detail here.
Uncertainty
Robust
23
Fragile
Robust (fragile) to perturbations in components
and environment ? Robust (fragile) to errors and
simplifications in modeling
More detail.
Required model complexity
Less detail.
Uncertainty
Robust
24
An apparent paradox
Component behavior seems to be gratuitously
uncertain, yet the systems have robust
performance.
Mutation
Selection
Darwinian evolution uses selection on random
mutations to create complexity.
25
Component behavior seems to be gratuitously
uncertain, yet the systems have robust
performance.

• Such feedback strategies appear throughout
  biology (and advanced technology).
• Gerhart and Kirschner (correctly) emphasis that
  this exploratory behavior is ubiquitous in
  biology
• but claim it is rare in our machines.
• This is true of primitive, but not advanced,
  technologies.
• Robust control theory provides a clear
  explanation.

Transcription/ translation Microtubules Neurogenes
is Angiogenesis Immune/pathogen Chemotaxis .
Regulatory feedback control
26
Motivation

• Without extensive engineering theory and math,
  even reverse engineering complex engineering
  systems would be hopeless.
• Modeling and simulation alone is inadequate
• Why should biology be much easier?
• We would not expect to have much success reverse
  engineering this laptop with
• Reductionism try to find the single transistor
  or group of transistors responsible for this
  slide
• Emergence emerging a random collection of
  silicon and metal

27
Engineering theory

• In some ways, with respect to robustness and
  complexity, there is too much theory, not too
  little.
• Two great abstractions of the 20th Century
• Separate systems engineering into control,
  communications, and computing
• Theory
• Applications
• Separate systems from physical substrate
• Facilitated massive, wildly successful, and
  explosive growth in both mathematical theory and
  technology
• but creating a new Tower of Babel where even the
  experts do not read papers or understand systems
  outside their subspecialty.

28
Tower of Babel

• Issues for theory
• Rigor
• Relevance
• Accessibility
• Spectacular success on the first two
• Little success on the last one, which is critical
  for a multidisciplinary approach to systems
  biology
• Perhaps all three is impossible?
• (In contrast, there are whole research programs
  in complex systems devoted exclusively to
  accessibility. They have been relatively
  popular, but can be safely ignored in biology.)

29
Biology and advanced technology

• Biology
• Integrates control, communications, computing
• Into distributed control systems
• Built at the molecular level
• Advanced technologies will do the same
• We need new theory and math, plus unprecedented
  connection between systems and devices
• Two challenges for greater integration
• Unified theory of systems
• Multiscale from devices to systems

30
Todays goal (Doyles part)

• Why theory is important, what it looks like, what
  specifically it can tell us about biological
  systems.
• Aim to tell you something not easily obtained
  elsewhere (papers online, other talks, texts,
  etc)
• Introduce basic ideas about robustness and
  complexity
• Minimize math details, but still suggest what is
  possible
• Hopefully familiar (but unconventional) example
  systems, not requiring specialized expertise
• Biology E. Coli Heat shock, chemotaxis
• Engineering Laptops, cars, and planes

31
Todays agenda (Doyles part)

• Part one qualitative description of issues
• Nature of complexity Robust, yet fragile
• Implications for modeling and analysis
• Need for theory
• Cartoons and pictures, no math
• Biology vs. engineering
• Part two (after break) quantitative
• Minimal math (but hopefully big returns?)
• Selected biology details to illustrate main themes

32
E. Coli Heat Shock (with Kurata, El-Samad,
Khammash, Yi)
33
Cell
Temp cell
Temp environ
34
Cell
How does the cell build barriers (in state
space) to stop this cascading failure event?
Temp cell
Temp environ
35
Temp cell
Folded Proteins
Temp environ
36
Temp cell
Folded Proteins
Temp environ
37
More robust ( Temp stable) proteins
Unfolded Proteins
Aggregates
Temp cell
Folded Proteins
Temp environ
38

• Key proteins can have multiple (allelic or
  paralogous) variants
• Allelic variants allow populations to adapt
• Regulated multiple gene loci allow individuals
  to adapt

Unfolded Proteins
Aggregates
Temp cell
Folded Proteins
Temp environ
39
37o
42o
Log of E. Coli Growth Rate
46o
21o
-1/T
40
Robustness/performance tradeoff?
37o
42o
Log of E. Coli Growth Rate
46o
21o
-1/T
41
Heat shock response involves complex feedback and
feedforward control.
Unfolded Proteins
Temp cell
Folded Proteins
Temp environ
42
Alternative strategies
Why does biology (and advanced technology)
overwhelmingly opt for the complex control
systems instead of just robust components?

• Robust proteins
• Temperature stability
• Allelic variants
• Paralogous isozymes
• Regulate temperature
• Thermotax
• Heat shock response
• Up regulate chaperones and proteases
• Refold or degraded denatured proteins

43
Why does biology (and advanced technology)
overwhelmingly opt for the complex control
systems instead of just robust components?
44

• In development
• drive-by-wire
• steering/traction control
• collision avoidance

45
Cascading events in car crashes
Normal
Danger
Crash
Contact w/car
Trauma
Barriers in state space
46
Normal
Danger
Crash
Contact w/car
Trauma
Normal
Sense/ Deploy
Contact w/bag
Trauma
47
Full state space
Desired
Worse
Bad
48
Full state space
Robust
Yet Fragile
49
Robust, yet fragile

• Robust to uncertainties
• that are common,
• the system was designed for, or
• has evolved to handle,
• yet fragile otherwise
• This is the most important feature of complex
  systems (the essence of HOT).

50
Humans supply most feedback control
Normal
Danger
Crash
Contact w/car
Trauma
Lanes Laws Lights Ramps
Collision avoidance Anti-lock brakes
Heavy metal Seat belts Airbags
Helmets
51
Fully automated systems?
Normal
Danger
Crash
Contact w/car
Trauma
Lanes Laws Lights Ramps
Collision avoidance

• Internally unimaginably more complex.
• Superficially much simpler?
• Biological systems are fully automated

52
Uncertainty
Basic functionality
Sensors
Robustness
53
Uncertainty
Sensors
Actuators
Actuators
Basic functionality
Sensors
Complexity is dominated by Robustness (through
regulatory feedback networks)
54
Uncertainty
Sensors
Actuators
Actuators
Basic functionality
Sensors
But scientific research has ignored almost all
real complexity.
55
Uncertainty in
Nutrients, O2, T, ions,
-taxis
Stress Response (e.g. heat)
Metabolism
Control
Cell cycle
Control
56
Uncertainty in

Nutrients, O2, T, ions,


Control
Metabolism
Cell cycle



57
Uncertainty
Sensors
Actuators
ic functiona ces, compone
materials
Actuators
Basic functionality
Sensors
But scientific research has ignored almost all
real complexity.
58
Uncertainty
Sensors
Actuators
Basic functionality Devices, components, material
s
Ators
Actuators
Basic functionality
Sensors
Sensors
But scientific research has ignored almost all
real complexity.
59
Control, communications, computing
Uncertainty
Sensors
Actuators
Actuators
Basic functionality
Sensors
Control networks

• Sense data
• Communications
• Information Focus on what is surprising in data
• Reliably store or transmit information
• Control
• Extract what is useful (not merely surprising)
• Compute decisions from useful information
• Take appropriate action

60
Theoretical foundations

• Control theory feedback, optimization, games
• Information theory source and channel coding
• Computational complexity decidability,
  P-NP-coNP-
• Dynamical systems dynamics, bifurcation, chaos
• Statistical physics phase transitions, critical
  phenomena, multiscale physics
• These are largely fragmented within isolated
  technical disciplines.
• Unified theory would be both intellectually
  satisfying and of enormous practical value.

61
Control Theory
Information Theory
Computational
Theory of Complex systems?
Complexity
Statistical Physics
Dynamical Systems
1 dimension ?
62
Uncertainty in

• Natural focus
• Chemo/Aero/Thermo-taxis
• Stress response
• Metabolic control
• Cell cycle control

Nutrients, O2, T, ions,
-taxis
Stress Response (e.g. heat)
Metabolism
Control
Cell cycle
Control
63
Uncertainty in

• Natural focus
• Chemo/Aero/Thermo-taxis
• Stress response
• Metabolic control
• Cell cycle control

Nutrients, O2, T, ions,
-taxis

• Highly conserved
• Design principles
• Molecular level

redox
Stress Response (e.g. heat)
Metabolism
Control
Cell cycle
Control

• Modular
• Integrated at higher levels

64
PMF
Metabolic control
-taxis
redox
T, O2, nutrients

• Modular
• Integrated at higher levels

65
Taylor, Zhulin, Johnson
66
(No Transcript)
67
Bacterial chemotaxis
68
Bacterial chemotaxis (Yi, Huang, Simon, Doyle)
Random walk
Ligand
Motion
Motor
69
Biased random walk
gradient
Ligand
Motion
Motor
Signal Transduction
70
Parallels between G-protein and Two-component
Signaling
71
(No Transcript)
72
Protocol stacks

• Key to modular architectures
• Various protocol stacks cut in various ways
  through complex networks

73
Protocols
74
Consumers
Barter
Commodities
75
(No Transcript)
76
Uncertainty
Robust Mesoscale
Robust, yet fragile
Uncertainty
77
Biological common currencies

• CheY
• Membrane potentials, ion channels
• Chemical energy ATP, glucose, lipids
• Transcriptional regulation
• G-protein signaling
• .

78
Energy

• 110 V, 60 Hz AC
• Gasoline
• ATP, glucose, etc
• Proton motive force

79
Ligands, Receptors
CheA-CheW
CheY-Motor
Attractants, Repellants
80
Molecular Machines
RNA Polymerase Ribosomes
Proteins
RNA
DNA
Atoms
DNA Polymerase
81
(No Transcript)
82
The hourglass
Garments
Dress
Shirt
Slacks
Lingerie
Coat
Scarf
Tie
Wool
Cotton
Nylon
Rayon
Polyester
Material technologies
83
The Internet hourglass
IP
From Hari Balakrishnan
84
The Internet hourglass
Everything on IP
IP
From Hari Balakrishnan
85
Applications
Robust Mesoscale
TCP/ IP
Robust, yet fragile
Hardware
86
Various functionality
Digital
Analog substrate
87
Applications
Software Hardware
Modern Computing
Operating System
Hardware
88
Robust, yet fragile
89
Applications

• Decentralized
• Asynchronous
• Robust to
• Network topology
• Application traffic
• Delays, link speeds

TCP/ IP
Hardware
High performance
Necessity Essentially only one design is possible
90
The existing design is incredible, but
Applications

• Decentralized
• Asynchronous
• Robust to
• Network topology
• Application traffic
• Delays, link speeds

TCP/ IP
Hardware
Its a product of evolution, and is not optimal.
High performance
Necessity Essentially only one design is possible
91

• Decentralized
• Asynchronous
• Robust to
• Environment
• Components

Molecular dynamics
High performance Evolvable
Necessity Essentially only one design is
possible??
92
An apparent paradox
Gratuitously uncertain components and complex
networks, but robust system performance.
Mutation
Selection
Darwinian evolution uses selection on random
mutations to create complexity.
93
Thus stabilizing forward flight.
At the expense of extra weight and drag.
94
For minimum weight drag, (and other performance
issues) eliminate fuselage and tail.
95
(No Transcript)
96
(No Transcript)
97
(No Transcript)
98
(No Transcript)
99
Why do we love building robust systems from
highly uncertain and unstable components?
Robust control theory tells us why.
100
Sensors
Actuators
Control
Vehicle
Size ? Complexity
101
Heater
Airplane
feedback
Thermostat
Sensors
Airbag
feedforward
Size ? Gain
Accelerometer
102
Airplane
Heater

• Sensors
• Critical component
• Low signal levels
• Highly stochastic
• Precise regulation

Sensors
Thermostat
Airbag
Sound familiar?
Accelerometer
103
Summary of part I

• Robustness/complexity spiral
• Robust, yet fragile complexity
• Gratuitously uncertain components
• Robust systems
• Emergent fragility
• Must be exploited for modeling and analysis of
  complex biological systems
• Part II Quantitative treatment