John Doyle

1
Theory of Complex Networks

• John Doyle
• Control and Dynamical Systems
• Caltech

2
Transportation
Finance
Health
Energy
Commerce
Our lives are run by/with networks
Consumer
Emergency
Information
Manufacturing
Utilities
3
Environment
Transportation
Health
Finance
Commerce
Energy
Convergent Networks
Emergency
Consumer
Manufacturing
Information
Utilities
4
Convergent networking the promise

• Resulting in
• Seamless integration and automation of everything
• Efficient and economic operation
• Robust and reliable services

• Ubiquitous computing, communications, and control
• that is embedded and intertwined
• via sensors and actuators
• in complex networks of networks, with layers of
  protocols and feedback.

We can demo anything
Environment
Transportation
Health
Finance
Commerce
Energy
Convergent Networks
Emergency
Consumer
Manufacturing
Information
Utilities
5
Elements of systems

• Sense the environment and internal state
• Extract whats novel
• Communicate or store whats novel
• Extract whats useful
• Compute decisions based on whats useful
• Take action
• Evaluate consequences
• Repeat

We want results
H A R D E R

• Data
• Is not novel information
• Is not useful Information
• Is not knowledge
• Is not understanding
• Is not wisdom
• Is not action
• Is not results

6
Theoretical foundations

• Control theory
• Dynamical systems
• Computational complexity
• Information theory
• Statistical physics

Unified theory would be both intellectually
satisfying and of enormous practical value.
Largely fragmented within isolated technical
disciplines.
7
Two great abstractions in science and technology

• Separate systems from physical substrate
• Systems The physical details dont matter
• Dont sweat the small stuff
• Reductionists The physics are all that matters
• Its all small stuff

• Separate systems engineering into control,
  communications, and computing
• Theory
• Applications

8
Zen Dont sweat the small stuff, and its
all small stuff
9
Two great abstractions

• Separate systems
• from physical substrate
• into control, communications, and computing
• Facilitated massive, wildly successful, and
  explosive growth in both mathematical theory and
  technology
• but creating a new Tower of Babel where even the
  experts do not read papers or understand systems
  outside their subspecialty.

10
Biology and advanced technology

• Biology
• Integrates control, communications, computing
• Into distributed control systems
• Built at the molecular level
• Advanced technologies will do the same
• We need new theory and math, plus unprecedented
  connection between systems and devices
• Two challenges for greater integration
• Unified theory of systems
• Multiscale from devices to systems

11
Bonus!

• Unified systems theory helps resolve
  fundamental unresolved problems at the
  foundations of physics
• Ubiquity of power laws (statistical mechanics)
• Shear flow turbulence (fluid dynamics)
• Macro dissipation and thermodynamics from micro
  reversible dynamics (statistical mechanics)
• Quantum-classical transition
• Quantum entanglement, measurement
• Thus the new mathematics for a unified theory of
  systems is directly relevant to multiscale
  physics
• The two challenges are connected.

12
Motivating scenarios

• Multiscale physics (turbulence, statistical
  mechanics, quantum mechanics, solid mechanics)
• Biological complexity (intra and inter cellular
  signaling and control)
• Ubiquitous, embedded, networked sensing,
  computing, and control
• Seek multiscale theory of protocols/modules/laws
• What follows focuses on engineering networks
• Unifying theory is emerging, but the math is
  hard, so we need to have lots of accessible
  examples

13
Ubiquitous embedded networking and control

• Until recently, even the most basic special cases
  had little theory
• Verification of embedded software for control
• Theory of protocol stacks such as TCP/IP
• Performance and robustness of distributed control
• Dramatic progress on many fronts
• Coherent theory of Applications/TCP/IP protocols
• Unified theory for robustness analysis of
  discrete and continuous and nonlinear systems
• Many promising implications but largely
  unexplored
• New theory has been somewhat inaccessible to
  nonexperts

14
Communications and computing
Store
Communicate
Compute
Communicate
Communicate
15
Store
Communicate
Compute
Communicate
Communicate
Act
Environment
16
Computation
Communication
Communication
Devices
Devices
Dynamical Systems
17

• From
• Software to/from human
• Human in the loop

• To
• Software to Software
• Full automation
• Integrated control, comms, computing
• Closer to physical substrate

Computation

• New capabilities robustness
• New fragilities vulnerabilities

Communication
Communication
Devices
Devices
Control
Dynamical Systems
18

• Until recently, there were no promising methods
  for addressing this full problem
• Even very special cases have had limited
  theoretical support for systematic verification
  of robustness

• This represents an enormous change, the impact of
  which is not fully appreciated
• Robustness and verifiability of highly autonomous
  control systems with embedded software is the
  central challenge

Computation

• New capabilities robustness
• New fragilities vulnerabilities

Communication
Communication
Devices
Devices
Control
Dynamical Systems
19
Multiscale physics

• Persistent mysteries at the foundations of
  physics where interconnection and robustness play
  a role.
• New theory yielding systematic resolution
• This in turn will provide a needed foundation for
  multiscale biochemistry and nanotechnology
• New mathematics addresses both
• Horizontal integration (control, comms,
  computing)
• Vertical coordination (protocols, multiscale
  physics)

20
Multiscale network biochemistry

• Biology
• Integrates control, communications, computing
• Into distributed control systems
• Built at the molecular level
• Will push all aspects of new theory, including
  horizontal and vertical integration
• Additional challenge of interaction with highly
  constrained (relative to engineering)
  experimental capabilities

21
Current research efforts

• Complex, ubiquitous, convergentnetworks
• Integrating controls, communications, and
  controls
• System robustness and software reliability
• Cascading failures in interacting networks
• Multiscale physics (turbulence, dissipation,
  power laws, quantum theory)
• Biological regulatory networks and ecosystems
• Finance and economics

22
Progress Highlights

• Integrated theory of networking protocols
• Source/channel characterization
• Robustness of TCP/IP
• Unified computational theory of robustness and
  reliability (continuous and discrete)
• Theoretical and software infrastructure for
  reverse engineering biological networks
• New view of shear flow turbulence
• Origin of power laws in complex systems
• Many results in statistical and quantum physics

23
Robustness of HOT systems
Fragile
Humans
Chess
Meteors
Robust
24
Robustness is a conserved quantity
Fragile
Chess
Meteors
Robust
25
Robustness of HOT systems
Fragile
Humans
Archaea
Chess
Meteors
Machines
Robust
26
Diseases of complexity
Fragile

• Parasites
• Cancer
• Epidemics
• Auto-immune disease

Complex development Regeneration/renewal Complex
societies Immune response
Uncertainty
Robust
27
Robustness of HOT systems
Fragile
Fragile (to unknown or rare perturbations)
Robust (to known and designed-for uncertainties)
Uncertainties
Robust
28
Network protocols.
Files
HTTP
TCP
IP
packets
packets
packets
packets
packets
packets
Routers
29
Web/internet traffic
web traffic
Is streamed out on the net.
Web client
Creating internet traffic
Web servers
30
web traffic
Lets look at some web traffic
Is streamed out on the net.
Web client
Creating internet traffic
Web servers
31
6
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
4
Cumulative
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
32
6
Web files
5
Codewords
4
Cumulative
3
Frequency
Fires
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
Log (base 10)
33
6
gt1e5 files
Data compression (Huffman)
WWW files Mbytes (Crovella)
5
4
gt4e3 fires
Cumulative
3
Frequency
Forest fires 1000 km2 (Malamud)
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
34
6
Data compression
WWW files Mbytes
5
4
Cumulative
3
Frequency
Forest fires 1000 km2
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Decimated data Log (base 10)
Size of events
35
6
Data compression
WWW files Mbytes
5
exponential
4
-1
Cumulative
3
Frequency
Forest fires 1000 km2
2
-1/2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
36
6
Data compression
WWW files Mbytes
5
exponential
4
Cumulative
All events are close in size.
3
Frequency
Forest fires 1000 km2
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
37
6
Most files are small
Data compression
WWW files Mbytes
5
4
-1
Cumulative
3
Frequency
Forest fires 1000 km2
Most fires are small
2
-1/2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
38
6
Data compression
WWW files Mbytes
Robust
5
4
-1
Cumulative
3
Frequency
Forest fires 1000 km2
2
-1/2
1
Yet Fragile
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
Size of events
39
Generalized coding problems

• Optimizing d-1 dimensional cuts in d dimensional
  spaces
• To minimize average size of files or fires,
  subject to resource constraint.
• Models of greatly varying detail all give a
  consistent story.

Data compression
Web
40
Theory
d 0 data compression d 1 web layout d
2 forest fires
41
Data
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
42
Data Model/Theory
6
DC
5
WWW
4
3
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
43
Data Orthodox Theory (Self-organized
criticality)
6
DC
5
WWW
4
WWW
3
FF
FF
2
1
0
-1
-6
-5
-4
-3
-2
-1
0
1
2
44
Network protocols.
Files
HTTP
TCP
IP
packets
packets
packets
packets
packets
packets
Routers
45
Mice
Network
Sources
Elephants
46
Router queues
Mice
Delay sensitive
Network
Sources
Bandwidth sensitive
Elephants
47
Router queues
Mice
Delay sensitive
Network
Control
Sources
Bandwidth sensitive
Elephants