The Internet as a computing surface

1
The Internet as a computing surface

• Avogadro-scale Computing
• MIT, April 17, 2008
• Scott Kirkpatrick
• Hebrew University of Jerusalem and CBA, MIT

2
Questions

• Based on 4 yrs of Internet measurement activity
• DIMES, ETOMIC, EVERGROW and Internet Archive
• Is the Internet Avogadro-scale?
• Routing IS asynchronous conformal computing
  analyze it.
• But the Internet is multi-layered
• Data is centrally managed and closely-held
• The current direction of growth is towards
  greater local independence
• Independence and asynchrony have their costs
• The details (data model, computational model,
  geographic distribution) are critical

3
Is the Internet really Avogadro-scale?

• Avogadros number is really 278.99 80 bits
• Internet addresses
• IPv4 232, IPv6 2128, IPv4 with NAT 264
• China takes NAT-ing the furthest, with only a
  very few entry points, huge internal address
  space.
• Web content modern search engines crawl 106
  web pages/sec, 2.5x1012 pages each month before
  discarding. Deep web maybe 100x larger.
• IPv4 operates in layers, divided by subnetworks
  (ASes).
• AS-AS uses BGP routing, inside AS uses
  shortest-path with link-state, may conceal all
  under MPLS

4
Next discuss how the Internet is connected

• K-shell analysis shows an interesting kind of
  hierarchical structure on the largest scale,
  gives unambiguous identification of the
  nucleus.
• Its fractal, so the structure if it has
  actually evolved to be a solution to more general
  problems can be applied on many scales.
• Percolation properties show that both local and
  long-ranged connectivity coexist

5
K-shell for network visualization
DIMES monitoring project www.netdimes.org
6
K-shell picture gives unique nucleus fractal
Carmi et al., PNAS 2007
7
Meduza (?????) model
This picture has been stable from January 04
(kmax 30) to present day, with little change in
the nucleus composition. The precise definition
of the tendrils sites isolated from the largest
cluster in all the crusts connect only to the
core.
8
Disruptive Alternatives to todays Protocols

• The Medusa structure has consequences routing,
  viewed as a computation, is changing
• Monitoring becomes a function in which every
  router participates.
• Information shared beyond todays
  customer-provider privilege even one step
  helps a lot
• Regional networking reduces dependence on
  long-haul carriers in the nucleus.
• Each trend is towards more local interchange in a
  flatter surface

9
Synchronization and in-band Control

• Roughening is an issue with distributed
  data-intensive CA-style computation. Can graded
  longer-range communications eliminate it? Are
  occasional small-world links enough? Which is
  more effective for the headaches required?

SK, Science Perspective, 2004
10
Compute-intensive CAs are different beasts

• Single data source, serial execution algorithms
  (like sort) suffer no synchronization overheads.
• Load and unload are critical
• Ultimate wallpaper computing will develop its
  own models
• Hierarchy aint going away anytime soon!