Chemical oscillator

1
Chemical oscillator
Nondimensional state equations
2
1
Can be computed analytically, which is not
scalable.
0.8
0.6
a
0.4
0.2
0
0
0.2
0.4
0.6
0.8
1
1.2
1.4
b
3
Robust hybrid/nonlinear systems theory (of
embedded networks)?
Linear theory plus bounds, with scalable
algorithms.
Theory without scalable algorithms.
Hacking. (Scalable algorithms without theory.)
Theory with scalable algorithms?
Most research Not scalable, no theory.
4
3
2.5
2
1.5
a 0.1, b 0.13
1
0
0.1
0.2
0.3
0.4
0.5
0.6
Numerical simulation.
5
Chemical oscillator
Reaction rate equation
6
a 1, b 2
a 0.6, b 1.1
(1.1, 0.6)
(2, 1)
7
Exponential scaling.
8
(No Transcript)
9
equilibrium
10
(No Transcript)
11
(No Transcript)
12
a 0.6, b 1.1
1
0.8
y
0.6
0.4
0.2
0
x
1
1.5
2
2.5
13
a 1, b 2
1
0.8
0.6
0.4
0.2
0
2.2
2.6
3
3.4
14
equilibrium
15
Yi, Ingalls, Goncalves, Sauro
Product inhibition
perturbation
16
The big picture

• How do we study large networks with many
  components? Scientific iteration between
• Data
• Modeling
• Inference
• Inference is the hardest part to scale to large
  problems, and is largely ad hoc.
• Science as an enterprise has largely given up on
  inference, focusing on data, modeling, and
  simulation instead. This is not enough and it
  does not scale.
• Progress and problems in addressing this issue.

17
Acknowledgements

• This workshop NEDO and Kitano ERATO
• Funding Kitano ERATO, DARPA, AFOSR, AfCS
• Robustness analysis examples Prajna,
  Papachristodoulou, Parrilo, Goncalves
• Heat Shock examples El-Samad, Khammash, Kurata,
  Gross, Grigorova,
• Metabolic control examples Yi, Ingalls, Sauro,
  Goncalves,
• Multiscale theory and algorithms Petzold,
  Gillespie, Rathanam, Bamieh, Mabuchi, cast of
  thousands
• Theory Parrilo, Carlson, cast of thousands

18
Key ideas

• There are fundamental laws governing the
  organization of biological networks.
• Simulation alone is not scalable, because
  complex, uncertain systems need an exponentially
  large number of simulations to answer
  biologically meaningful questions.
• Multiscale and large-scale stochastic simulation
  is an essential technology.

19
Hypotheses

• There are fundamental laws governing the
  organization of biological networks. Without
  exploiting these, the complexity is
  overwhelming.
• Simulation alone is not scalable. Automated
  scalable inference using semi-algebraic geometry
  and semi-definite programming.
• Multiscale and large-scale stochastic simulation
  is an essential technology. We need Gillespie
  Petzold integrated into a single methodology.

20
The scientific iteration

• Do experiments and gather data.
• Make assertions about the data (modeling).
• Reason about the assertions to make inferences
  about the systems under study.
• Current scientific infrastructure
• scales ok for 1, less well for 2
• is completely intractably for 3
• We are focusing on 3. An example.

21
The challenge of inference

• What are the properties of a model? What about it
  is robust and fragile?
• Is a model consistent with a set of data and with
  the constraints of biochemistry?
• Is a model structure fundamentally incompatible
  with data, no matter what parameters are
  adjusted?
• What are the critical parameters? Is something
  missing?
• What is the best experiment to do next?
• Simulation is a useful but ultimately limited
  tool in answering these questions.
• But, if we are going to do simulations, we need
  to at least do them correctly and fast.

22
Engineering design objectives

• Robustness to uncertainty in environment and
  components
• Scarce resources are used efficiently
• Scalable to large numbers
• (in order to do 1-2, it may be necessary to have
  high internal complexity (complicated), but we
  want simple, robust, verifiable external
  behavior, so)
• Verifiable with short proofs

23
Robustness, evolvability/scalability,
verifiability
Robustness
Ideal performance
Verifiability?
24
Robustness, evolvability/scalability,
verifiability
Robustness
Ideal performance

• Verifiability in forward engineering translates
  into comprehensibility in reverse engineering of
  biological systems
• This research direction may be good news for
  understanding complex biological processes

25
Robustness and verifiability
Robustness
Ideal performance

• How do you prove nothing bad can happen?
• Need both new modeling techniques and new proof
  techniques.

26
Complexity lessons review

• Highly evolved systems are robust yet fragile
• Complexity implies fragility
• Orthodoxy of order-disorder transition is a red
  herring

27
Complexity lesson 2

• Complexity implies fragility
• Dual complexity implies primal fragility
• Dual complexity ? proof length
• Primal fragility ? ill-conditioning
• Fragile The answer changes a lot if the question
  changes a little.
• Complex The shortest explanation is long.

28
Set of bad system behaviors
Set of possible system behaviors
Proof of robustness
Modeling
Analysis
29
Set of experimental behaviors
Set of possible model behaviors
Proof that model doesnt work
Modeling
Analysis
30
Set of bad system behaviors

• Sources of uncertainty
• Variations in components and environment
• Modeling assumptions
• Computational approximations
• Incomplete or inadequate description of
  objectives
• Want to manage these in a systematic and
  integrated way.

Set of possible system behaviors
Proof of robustness
31
Set of bad system behaviors
Set of possible system behaviors
NP exhibit a point
Modeling
Analysis
Problem Not robust
32
Set of experimental behaviors
Set of possible model behaviors
NP exhibit a point
Exists a model consistent with data.
Modeling
Analysis
33
Set of bad system behaviors
Set of possible system behaviors
coNP Give a proof
Modeling
Analysis
34
Set of experimental behaviors
Set of possible model behaviors
NP exhibit a point
Exists a model consistent with data.
Modeling
Analysis
35
Set of bad system behaviors
Key idea Complexity implies fragility of model
Set of possible system behaviors
Modeling
Analysis
Problem Robust but no short proof
36
Set of bad system behaviors

• Needs
• Rich modeling methods (hybrid, nonlinear, DAE,
  PDE)
• Systematic uncertainty management
• Scalable, automated proof system
• Feedback from proof complexity to model fragility
• Enormous progress on 1-3, promising new insights
  for 4.

Set of possible system behaviors
Proof of robustness
37
Set of experimental behaviors
Set of possible model behaviors
NP exhibit a point
coNP exclude large regions that need not be
searched.
38
Assume that this is easy. Call that P.
39
?
Search can be harder than functional
evaluation. Call this NP.
40
Exponential scaling.
41
?
If true, then there is a short proof. But
finding this may be hard.
42
Set of bad system behaviors
Set of possible system behaviors
NP exhibit a point
Modeling
Analysis
Problem Not robust
43
Typically coNP hard.

• Fundamental asymmetries
• Between P and NP
• Between NP and coNP

• More important problem.
• Short proofs may not exist.

?
Unless theyre the same
44
Set of bad system behaviors
Set of possible system behaviors
coNP Give a proof
Modeling
Analysis
45
?
46
?
No.
47
?
Yes.
48
What makes a problem harder?
?
49
Set of bad system behaviors
Key idea Complexity implies fragility of model
Set of possible system behaviors
Modeling
Analysis
Problem Robust but no short proof
50
Easy to find solutions?
?
Satisfiable or feasible
51
Set of bad system behaviors
Set of possible system behaviors
NP easy to find a point
Modeling
Analysis
Problem Not robust
52
Easy to find proofs?
Unsatisfiable or infeasible
?
53
Set of bad system behaviors
coNP easy to find a proof
Set of possible system behaviors
Modeling
Analysis
54
Set of bad system behaviors
Key idea Complexity implies fragility of model
Set of possible system behaviors
Modeling
Analysis
Problem Robust but no short proof
55
Hard Problems
coNP
Economics
Algorithms
Controls
NP
Communications
P
Dynamical Systems
Physics
56
Hard Problems
coNP
Economics
Algorithms
Controls
Biology?
NP
Internet
Communications
P
Dynamical Systems
Physics
57
Hard Problems
coNP
Unified Theory Status
Goal
Economics
Algorithms
Controls
Biology?
NP
Goal
Internet
Communications
P
Dynamical Systems
Physics
58
Lattice models?
What can we do with lattices that will be easy to
understand, yet relevant to the real
computational complexity problems that we most
care about?

• Key abstractions
• Robustness/Fragility
• Verifiability
• Complexity
• Resource scarcity

59
.2
.4
.6
.8
Density fraction of occupied sites (black)
Focus on horizontal paths.
60
Vertical paths in empty sites are allowed to
connect through corners or edges. (8 neighbors)
Horizontal paths connect only on edges. (4
neighbors.Ordinary square site percolation.)
Focus on horizontal paths.
Some (nonstandard) definitions
61
Critical phase transition at density .59
62
.2
.4
.6
.8
Density fraction of occupied sites (black)
Focus on horizontal paths.
63

• Robustness is provided by barriers in some state
  space. These prevent cascading failure events.
• Lattices offer a crude abstraction, in that paths
  can be thought of as barriers, with robustness to
  perturbations in the lattice.

• Verifiability complexity is measured in the
  length of the proof required to verify
  robustness.
• Lattices can offer a variety of crude
  abstractions to this as well. The length of
  minimal paths would be a simple measure of proof
  length.

64

• Very special features
• Dual and primal problems are essentially the
  same.
• There is no duality gap.

Caution potential source of confusion.
65
Barriers in 3d lattices are 2d cuts.
Barriers in 1d lattices are 0d cuts.
path fragments
barrier
In general, barriers are d-1 dimensional (dual)
cuts stopping 1-dim (primal) paths in a d-dim
lattice.
66
Short proofs may not exist.
?
67
Critical phase transition at density .59
68

• Lattices offer pedagogically useful but
  potentially dangerously misleading
  simplifications, which are thus both strengths
  and weaknesses
• Internal complexity
• Computational complexity
• Duality

Focus on horizontal paths.
69

• Internal vs external complexity Real biology
  and technology uses extremely complex
  (complicated) hierarchical organization in order
  to create robust and verifiably (simple)
  behavior. Lattices allow no distinction between
  complex organization and complex behavior. This
  can be very misleading.
• Computational complexity Most lattice
  computational problems are in P and thus easily
  explored, but fail to illustrate the P/NP
  asymmetry. We will rely on notions of complexity
  that are good analogies, but not precisely
  comparable.
• Duality Duality is greatly simplified and
  transparent. This makes exposition easy but hides
  the NP/coNP asymmetry which is central to the
  general problem.

70

• Lattices offer enormous (and potentially
  dangerous) simplifications
• Robustness problem existence of horizontal path
• Verification prove existence of horizontal
  path
• Complexity minimum horizontal path length (of
  proof)
• Model fragility minimum number of site changes
  to break all horizontal paths ( create a
  vertical path)

Focus on horizontal paths.
71
Note Im going to draw small lattices and rely
on your imagination for what large lattices would
look like.
72

• Alternative definition of complexity
• The computer is you, looking at the lattice
  and determining by inspection whether there is a
  path or not.
• This can be easy or hard, depending on the
  density.
• This is not exactly the same as minimal path
  length, but close enough for now.
• Do a very informal story, and then make it
  rigorous.

.2
.4
.6
.8
Density fraction of occupied sites (black)
73
No
Yes
Easy
Exist horizontal path?
Hard
For random lattices, there are 4 regimes, with
all combinations of Easy/Hard and Yes/No. The
hard cases correspond to lattices that are of
intermediate density, near the critical point.
Easy cases are either high or low densities,
which always correspond to Yes or No,
respectively.
74
No
Yes
No
Yes
Easy
Easy
Hard
Hard

• It is much easier to see with all the clusters
  colored. But thats cheating, because determining
  the clusters is essentially the computational
  problem.
• Note that the complexity (minimum proof) can be
  much simpler than the complicated description of
  the lattice

75
The orthodox story
No
Yes
Easy
Hard problems are associated in some way with the
phase transition.
Hard
76
The counter-examples
Exactly the opposite of criticality
No
Yes

• Yes or no
• Easy or hard
• High or low density
• Robust or fragile (to perturbations)

Easy
Hard
77
The counter-examples
Exactly the opposite of criticality

• Yes or no
• Easy or hard
• Low or high density
• Robust or fragile (to perturbations)

16 different possible combinations
78
The counter-examples
Exactly the opposite of criticality
8

• Yes or no
• Easy or hard
• Low or high density
• Robust or fragile (to perturbations)

16 different possible combinations
79
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density (but connected)
High density
Hard implies fragile (well prove this later). So
only 6 of the 8 possibilities exist, and the
critical density is nothing special. We will
prove that these and only these implications hold.
80
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density
High density
Robust
Fragile
Random
Easy
Hard
81
Robust
Fragile
All interesting real world problems are in this
regime, with efficient, highly structured, rare
configurations, using scarce (limited) resources.

Easy
Hard
Low Density
Robust
Fragile
Random
Easy
Hard
82
Robust
Fragile
Easy
Impossible.
Hard
Low Density
Robust
Fragile
Improbable in random lattices.
Random
Easy
Hard
83
Robust
Fragile
Robust
Fragile
Easy
Easy
Hard
Hard
Low Density
High density
Proof to follow.
84
Random lattices are complex (and fragile) only
at critical phase transition.
Low Density
High density
Robust
Fragile
Easy
Hard
85
Definitions. Assume there is a connected
(horizontal) path of minimal length l .
n length of side r density l MinPath length
Occupied
Empty
MinPath
Typical minimal path
86
Definitions. Assume there is a connected path of
minimal length l .
n length of side r density l MinPath
length b MinCut barrier length
Typical minimal cut
Occupied
Empty
MinPath
Typical minimal path
b
87
Definitions. Assume there is a connected path of
minimal length l .
n length of side r density l MinPath
length b MinCut barrier length
Vertical path
b
88
n length of side r density l MinPath
length b MinCut barrier length
Assume a path exists. (Otherwise LF?.)
Necessarily ? ? 1/n, n2 ? l ? n and define
89
l MinPath length b MinCut barrier length
90
l MinPath length b MinCut barrier length
Proof (Vinnicombesushi) To provide robustness
to b changes, there must be at least b
independent paths, which by assumption have
minimum length l. Necessarily ? n2 ? lb, or ?
n/b ? l/n. Take log of both sides.
91
This is maximally tight in the sense that

• Lattices and paths can be
• Resources Scarce or rich
• Existence of path Yes or no
• Complexity Hard or easy
• Perturbations Fragile or robust

92

• Lattices and paths can be
• Existence Yes or no
• Resources Scarce or rich
• Perturbations Fragile or robust
• Complexity Hard or easy

Anything is possible, consistent with the theorem.
Well just consider the 8 cases with paths.
93
Fragile
Hard
Robust
Easy
Rich
Scarce
-Slog(?)
94
Fragile
Hard
Robust
Easy
Rich
Scarce
95
Fragile
Hard
Robust
Easy
Rich
Scarce
96
Hard
Fragile
Scarce
Rich
Easy
Robust
97
Easy
Occupied
Empty
MinPath
FS, L0
98
Easy
Most robust possible.
FS, L0
99
Easy and Fragile
Flog(n)gtS, L0
100
Hard
Fragile
FSL
Scarce
Rich
Easy
m
d
Robust
b
Occupied
Empty
MinPath
101
r density b MinCut barrier length l MinPath
length n length of side m of cells d
width of open regions
To construct asymptotically tight cases where ?n2
lb, consider the lattice below.
m
d
b
b
d
102
Now take limits
By constructing lattices as below, with ngtgtmgtgt1,
it is possible to find lattices such that any ?n2
? lb, with ?lt1 is achievable.
103
FSL
Hard
Fragile
Scarce
Rich
Easy
Robust
104
The Fragile Face
Hard
Fragile
Scarce
Rich
Easy
Robust
105
The Four Corners
Hard
Fragile
Scarce
Rich
Easy
Robust
106
FSL
Fragile
Most Fragile FgtgtS
Scarce
Most Robust FS
Easy
Robust
107
Random
Hard
Fragile
Scarce
Rich
Easy
Robust
108
Efficient and robust is far from random
109
Efficient, robust, verifiable
110

• How general is this?
• Seems to hold in all theory where it has been
  investigated.
• Extensive literature on ill-conditioning in LPs
  and numerical linear algebra.
• Anecdotally, seems to capture essence of many
  complexity problems.
• Needs to be combine with laws constraining net
  system fragility.

111
Hard Problems
coNP
Economics
Algorithms
Controls
Biology?
NP
Internet
Communications
P
Dynamical Systems
Physics
112
Hard
Assume optimal and worst-case
Fragile
Scarce
Economics
Rich
Algorithms
Easy
Controls
Biology?
Internet
Robust
Assume random or generic
Communications
P
Dynamical Systems
Physics
113
Hard
Assume optimal and worst-case
Fragile
Scarce
Economics
Rich
Algorithms
Easy
Controls
Biology?
Internet
Robust
Assume random or generic
Communications
P
Dynamical Systems
Physics
114
Hard
Fragile
Scarce
Rich
Easy
Robust
Efficient, robust, verifiable
115
Bad news and good news

• Bad news? Some hoped-for connections between
  phase transitions and complexity are not there.
• Good news? Ideas still interesting.
• Lots more really good news!
• The alternative is much richer and useful, and
  connects in interesting ways with phase
  transitions
• New algorithms, new mathematics, new practical
  applications,
• And deep implications for physics.

116
Physics and emergilence at the edge of
chaocritiplexity
Phase transitions

• Internet traffic and topology
• Biological and ecological networks
• Evolution and extinction
• Earthquakes and forest fires
• Finance and economics
• Social and political systems

?
Complexity
117
(No Transcript)
118
Physics and the edge of chaocritiplexity
Phase transitions

• Internet traffic and topology
• Biological and ecological networks
• Evolution and extinction
• Earthquakes and forest fires
• Finance and economics
• Social and political systems

?
Rich new unifying theory of complex control,
communication, and computing systems
Complexity
119

• Ubiquity of power laws (Carlson, UCSB)
• Coherent structures in shear flow turbulence
  (Bamieh, UCSB)
• Stat mech for nonequilibrium systems. (Caltech
  and UCSB)
• Quantum entanglement and complexity in quantum
  information technology (Caltech)

Physics applications
Rich new unifying theory of complex control,
communication, and computing systems
120

• Ubiquity of power laws (Carlson, UCSB)
• Coherent structures in shear flow turbulence
  (Bamieh, UCSB)
• Stat mech for nonequilibrium systems. (Caltech
  and UCSB)
• Quantum entanglement and complexity in quantum
  information technology (Doherty, Caltech)

Last talk on Friday.
121
Complexity lessons review

• Highly evolved systems are robust yet fragile
• Complexity implies fragility
• Orthodoxy of order-disorder transition is a red
  herring

122
Complexity lesson 2

• Complexity implies fragility
• Dual complexity implies primal fragility
• Dual complexity ? proof length
• Primal fragility ? ill-conditioning
• Fragile The answer changes a lot if the question
  changes a little.
• Complex The shortest explanation is long.

123
Modeling Robustness barriers Polynomial
inequalities
Analysis Real algebraic geometry Duality SDP/SOS

• Has already proven to be astonishingly effective
  in a wide variety of areas
• Math and applications largely unfamiliar to many
  control theorists, so
• sketch broadly the themes and applications

124
Why it all works.
Modeling Robustness barriers Polynomial
inequalities
Analysis Real algebraic geometry Duality SDP/SOS
125
Why its hard.
Modeling Robust control theory Operator Banach
Algebras
Analysis Real algebraic geometry Duality Optimizat
ion
Model fragility
Proof complexity
Theoretical CS NP-coNP
126
The challenge of inference

• Example Is a model structure consistent with a
  set of data?
• Much harder than (but as important as) taking
  data or extracting information or forming models
  or running simulations.
• Modeling and simulation is a small part of the
  solution, but receives most of the attention (you
  do what you can).
• Will attempt to describe the bigger challenge
• which is critical to empowering biologists to
  tackle large network problems.

127

• Technology has become dominated by the challenge
  of systematic inference, whether we like it or
  not.
• Example Verifying (proving) that software works,
  as opposed to running programs, searching for
  bugs, and hoping for the best. (We dont do this
  very well.)
• The cost and challenges of scaling the problem of
  inference (not data, modeling, or simulation)
  will dominate future biology, whether we like it
  or not.
• (Just as the cost of embedded software has come
  to dominate all other costs in large engineering
  projects. We have to do this better too.)
• New research breakthroughs offer unprecedented
  promise for inference in biology (and
  verification of embedded protocols).Will sketch
  the implications now, and can describe the
  methods this evening.