Multiscale

1
My interests John Doyle CDS Caltech
Multiscale Physics
Core theory challenges
Network Centric, Pervasive, Embedded, Ubiquitous
Systems Biology, Medicine, Ecology
2
Architecture in engineering

• Design emphasis is shifting from
• Components to
• Platforms to
• Systems to
• Systems of systems to
• Architectures

• Increasingly dominated by
• network fragilities
• cascading failures
• infectious hijacking
• unintended consequences

3
Substantial challenges

• From networks for computing and communication
• To networks for control
• Fragmented (stove-piped) theories and disciplines
• Techno/eco success (i.e. sustainability) depends
  on social networks (in the broadest sense)
• Need theories and research as networked as our
  bio/eco/techno networks are

4
Promising progress

• Case studies Internet, bacterial biosphere,
  medicine, ecosystems, manufacturing,
• Theory integrating distributed control,
  communications, computing, thermodynamics
• Stories simple toy examples that illustrate
  essential ideas without math details
• Yet substantial challenges
• High-impact literature fraught with basic errors

5
Essential ideas

• Robust yet fragile
• Network architecture constraints
• System function and robustness
• Components
• Protocols
• Layering and control
• Hourglasses and Bowties

6
Robust
Human complexity
Yet Fragile

• Efficient, flexible metabolism
• Complex development and
• Immune systems
• Regeneration renewal
• Complex societies
• Advanced technologies

• Obesity and diabetes
• Rich microbe ecosystem
• Inflammation, Auto-Im.
• Cancer
• Epidemics, war,
• Catastrophic failures

• Evolved mechanisms for robustness allow for, even
  facilitate, novel, severe fragilities elsewhere
• often involving hijacking/exploiting the same
  mechanism
• There are hard constraints (i.e. theorems with
  proofs)

7
Systems Hard limits and tradeoffs

• On systems and their components
• Thermodynamics (Carnot)
• Communications (Shannon)
• Control (Bode)
• Computation (Turing/Gödel)

No dynamics, feedback
No networks
8
Hard limits and tradeoffs

• On systems and their components
• Thermodynamics (Carnot)
• Communications (Shannon)
• Control (Bode)
• Computation (Turing/Gödel)
• Fragmented and incompatible
• Cannot be used as a basis for comparing
  architectures
• New unifications are encouraging

Assume different architectures a priori.
9
Architecture examples

• There are universal architectural features
  ubiquitous in complex technological and
  biological networks
• Examples include
• Bowties for flows of materials, energy, redox,
  information, etc (stoichiometry)
• Hourglasses for layering and distribution of
  regulation and control (fluxes, kinetics,
  dynamics)
• Nascent theory confirms (reverse engineers)
  success stories but has (so far) limited forward
  engineering applications (e.g. FAST TCP/AQM)

10
Universal architectures

• Hourglasses for control
• Bowties for flows
• Robust and evolvable
• Architecture protocols constraints

11
The Internet hourglass

• Brief aside for (inter)networking experts
• The hourglass label inherited from networking
• Whats new (last decade) is theory (that largely
  confirms the engineers intuition)
• Layering
• Dynamics
• Feedback control

12
The Internet hourglass
Applications
Web
FTP
Mail
News
Video
Audio
ping
napster
Ethernet
802.11
Satellite
Optical
Power lines
Bluetooth
ATM
Link technologies
13
The Internet hourglass
Applications
Web
FTP
Mail
News
Video
Audio
ping
napster
TCP
IP
Ethernet
802.11
Satellite
Optical
Power lines
Bluetooth
ATM
Link technologies
14
The Internet hourglass
Applications
IP under everything
Web
FTP
Mail
News
Video
Audio
ping
napster
TCP
IP
Ethernet
802.11
Satellite
Optical
Power lines
Bluetooth
ATM
Link technologies
15
Applications
Top of waist provides robustness to variety
and uncertainty above
TCP/ AQM
Bottom of waist provides robustness to
variety and uncertainty below
IP
16
Layering as optimization decomposition

• Each layer is abstracted as an optimization
  problem
• Operation of a layer is a distributed solution
• Results of one problem (layer) are parameters of
  others
• Operate at different timescales

17
Examples
Optimal web layer Zhu, Yu, Doyle 01
HTTP/TCP Chang, Liu 04
TCP Kelly, Maulloo, Tan 98,
TCP/IP Wang et al 05,
TCP/MAC Chen et al 05,
TCP/power control Xiao et al 01,
Chiang 04,
Rate control/routing/scheduling Eryilmax et al
05, Lin et al 05, Neely, et al 05, Stolyar 05
detailed survey in Proc. of IEEE, 2007
18
Main bowtie in Internet S


Variety of files
Variety of files
packets

• All sender files transported as packets
• All files are reconstructed from packets by
  receiver
• All advanced technologies have protocols
  specifying knot of carriers, building blocks,
  interfaces, etc
• This architecture facilitates control, enabling
  robustness and evolvability
• It also creates fragilities to hijacking and
  cascading failure

19
Many bowties in Internet


Variety of files
Variety of files
packets


Applications




TCP


IP


20
FAST TCP/AQM theory

• Arbitrarily complex network
• Topology
• Number of routers and hosts
• Nonlinear dynamics
• Delays

• Short proof
• Global stability
• Equilibrium optimizes aggregate user utility

Papachristodoulou, Li
21
Examples of
knot
universal carriers

• Knots
• Packets in the Internet
• 60 Hz AC in the power grid
• Lego snap
• Money in markets and economics?
• Lots of biology examples

22
Nested bowties advanced technologies
Everything is made this way cars, planes,
buildings, laptops,
23
Electric power
Variety of producers
Variety of consumers
24
Standard
interface
Variety of consumers
Variety of producers
Energy carriers

• 110 V, 60 Hz AC
• (230V, 50 Hz AC)
• Gasoline
• ATP, glucose, etc
• Proton motive force

25
fan-out of diverse outputs
universal carriers
fan-in of diverse inputs
Other examples
Diverse function

• Lego
• Biology
• Independent design/evolution of these examples

Universal Control
Diverse components
26

• Carriers
• Precursors
• Trans
• 2CST

Bacterial bowties
Sugars

Fatty acids
Precursors
Co-factors
Catabolism
Amino Acids
Nucleotides
Genes
Proteins
Carriers
Trans
DNA replication
27
Lego hourglass
Diverse function
control
Universal Control
assembly
Diverse components
28
Lego hourglass
Diverse components
29
Lego hourglass
Diverse function
30
Lego hourglass

control
Universal Control
assembly
31
Lego system requirements
32
Lego hourglass
Huge variety
Standardized mechanisms Highly conserved
control
assembly
Huge variety
33
Lego
Huge variety
Limited environmental uncertainty needs minimal
control
Standard assembly
Huge variety
34
Question what is the difference between
hourglass and bowtie here?
Diverse function
Standard assembly
Diverse components
A minimal setting to address this issue.
Variety of systems
Variety of bricks
Snap
35
The snap is a static interface specification.
Variety of systems
Variety of bricks
Snap
36
Diverse function
Assembly is a control process that evolves in
time, and respects the snap, but adds to it.
Standard assembly
Diverse components
It inputs instructions and components and outputs
assembled systems.
37
Diverse function
Standard assembly
Diverse components
Variety of systems
Variety of bricks
Snap
38
(No Transcript)
39
(No Transcript)
40
Random, uncontrolled, snap connection of Lego
parts yields nonfunctional toys.
Diverse function
Standard assembly
Diverse components
41
Diverse function
Standard assembly
Loss of reuse, gain in robustness.
Diverse components
42
Diverse function
Standard assembly
Diverse components
43
Robustness/ Evolvability

• A huge variety of new and different toys can be
  built
• From a huge variety of different components
• Both toys and components can be rearranged and
  added in new ways
• Yet fragile?

44
Yet fragile

• Add or remove a tiny, indistinguishable amount of
  material from either side of a key interface.
• ? Complete failure.
• Other parts of the bricks may be nicked or cut
  with minimal impact
• This robust, yet fragile (RYF) feature of
  protocols is a candidate for a universal law
• What robustness/fragility properties do
  alternative protocols have?

45
Standard interface. (Wild type.)
46
? better ? worse
47

• Lego is optimally robust (Pareto) not
  optimal.
• Similar to complex engineering systems and
  biology.

? better ? worse
48
Fragility Perturbing the snap connector?
WT is very fragile
49
Robust or fine-tuned?

• Set of all possible interconnections is a
  (combinatorial) huge set.
• Set of interesting toys is also large, but an
  infinitesimally small subset. Very special and
  finely tuned.
• Similarly, among the potential toy systems using
  the same basic plastic material, Lego is highly
  structured and finely tuned.
• At the component level, the stud-and-tube
  coupling is very finely machined.
• Robust yet fragile (RYF) is universal in complex
  engineering and biology

50
The evolution of complexity
51
Suppose you want to put a structure on wheels?
Easy Find Lego parts with wheels.
52
Suppose you want to motorize a vehicle with
wheels?
Easy Add Lego motors,gears and battery.
53
Suppose you want to motorize a vehicle with
wheels?
Easy Add Lego motors,gears and battery.
Additional protocols and modules.
54
This cart/motor/gear configuration could be a
module that is reused in many toys.
55
Evolvability

• The snap/brick can be augmented with additional
  parts and interfaces
• Assembly remains essentially the same

Variety of systems
Variety of parts
Snap
56
Complex toys can be created, and require
additional layers of control.
57
Random
Not random
Variety of systems
Variety of bricks
Snap
58
Lego hourglass
Uncertain environments
Require additional layers
control
assembly
Huge variety
59
NXT controller
Variety of actuators
Variety of sensors
Real- time control
60
NXT controller
Real-time control
Variety of actuators
Variety of sensors
Real- time control
Control software Hardware (CPU, memory,...)
Sensor interface
Actuator interface
61
fan-out of diverse outputs
universal carriers
fan-in of diverse inputs
Diverse function
Bowties andHourglasses
Universal Control
Diverse components
62
Variety of actuators
Variety of sensors
Control
Lego Bowties
Variety of systems
Variety of bricks
Snap
63
Most complexity is in digital hardware and
software.
structure
wheels
motor
controls
64
Variety
control
Lego hourglass
assembly
Variety
65
Variety
Yet fragile?
Robust
Variety
66
Yet fragile?
Real-time control
Variety of actuators
Variety of sensors
Control software Hardware (CPU, memory,...)
Sensor interface
Actuator interface
67
System-level
Architecture Constraints
Aim a universal taxonomy of complex systems and
theories
Emergent
Protocols
Component

• Describe systems/components in terms of
  constraints on what is possible
• Decompose constraints into component,
  system-level, protocols, and emergent
• Not necessarily unique, but hopefully
  illuminating nonetheless

68
Systems requirements functional,
efficient, robust, evolvable
Hard constraints Thermo (Carnot) Info
(Shannon) Control (Bode) Compute (Turing)
Protocols
Constraints
Components and materials Energy, moieties
69
Robust
Human complexity
Yet Fragile

• Efficient, flexible metabolism
• Complex development and
• Immune systems
• Regeneration renewal
• Complex societies
• Advanced technologies

• Obesity and diabetes
• Rich microbe ecosystem
• Inflammation, Auto-Im.
• Cancer
• Epidemics, war,
• Catastrophic failures

• Evolved mechanisms for robustness allow for, even
  facilitate, novel, severe fragilities elsewhere
• often involving hijacking/exploiting the same
  mechanism
• There are hard constraints (i.e. theorems with
  proofs)

70
(No Transcript)
71
(No Transcript)
72
Figure 7. How market-centered social
organization contributes to mortality from
hypervigilance and hyposatisfaction.
73
food intake
Blood
Organs Tissues Cells Molecules
Glucose Amino acids Fatty acids
Universal metabolic system
74
Universal reward systems
sports music dance crafts art toolmaking
sex food
Prefrontal cortex Accumbens
dopamine
VTA
Dopamine, Ghrelin, Leptin,
75
Universal reward systems
sports music dance crafts art toolmaking
sex food
Prefrontal cortex Accumbens
dopamine
VTA
food
Blood
Organs Tissues Cells Molecules
Glucose Amino Acids Fatty acids
Universal metabolic system
76
Universal reward systems
work family community nature
Prefrontal cortex Accumbens
dopamine
VTA
food sex toolmaking sports music dance crafts
art
Robust and adaptive, yet
77
Universal reward systems
work family community nature
Prefrontal cortex Accumbens
dopamine
VTA
food sex toolmaking sports music dance crafts
art
Robust and adaptive, yet
78
work family community nature
Prefrontal cortex Accumbens
dopamine
VTA
sex food toolmaking sports music dance crafts art
79
work family community nature
market/ consumer culture
money
salt sugar/fat nicotine alcohol
Prefrontal cortex Accumbens
dopamine
VTA
Vicarious
sex food toolmaking sports music dance crafts art
industrial agriculture
80
work family community nature
money
salt sugar/fat nicotine alcohol
Prefrontal cortex Accumbens
dopamine
VTA
Vicarious
sex toolmaking sports music dance crafts art
cocaine amphetamine
81
high sodium
money
obesity
salt sugar/fat nicotine alcohol
overwork
smoking
Vicarious
alcoholism
drug abuse
82
high sodium
money
dopamine
VTA
obesity
salt sugar/fat nicotine alcohol
overwork
smoking
Glucose Amino Acids Fatty acids
Vicarious
alcoholism
drug abuse
83
Robust
Human complexity
Yet Fragile

• Efficient, flexible metabolism
• Complex development and
• Immune systems
• Regeneration renewal
• Complex societies
• Advanced technologies

• Obesity and diabetes
• Rich microbe ecosystem
• Inflammation, Auto-Im.
• Cancer
• Epidemics, war,
• Catastrophic failures

• Evolved mechanisms for robustness allow for, even
  facilitate, novel, severe fragilities elsewhere
• often involving hijacking/exploiting the same
  mechanism
• There are hard constraints (i.e. theorems with
  proofs)

84
Molecular levels
85
Essential ideas

• Robust yet fragile
• Network architecture constraints
• System function and robustness
• Components
• Protocols
• Layering and control
• Hourglasses and Bowties