MEDUSA New Model of Internet Topology Using kshell Decomposition

1
MEDUSA New Model of Internet Topology Using
k-shell Decomposition

• Shai Carmi Shlomo Havlin

Bloomington 05/24/2005
2
Who we are

• Talk prepared by Shai Carmi.
• Graduate student in the Department of Physics,
  Bar-Ilan University, Israel.
• Supervised by Prof. Shlomo Havlin, who gives
  the talk.

3
Who we are

• Collaborators
• Prof. Scott Kirkpatrick, Hebrew University of
  Jerusalem, Israel.
• Dr. Yuval Shavitt, Tel-Aviv University, Israel.
• Eran Shir, Ph.D. Student, Tel-Aviv University,
  Israel.

Scott
4
Measuring the Internet

• Previous efforts to measure the Internet have
  used
• One machine Traceroute to many destinations.
• Many machines, specially deployed to traceroute
  to many destinations
• Sites restricted to academic or govt labs, on
  network backbone
• General perception was that Law of Diminishing
  Returns has set in.

5
Measuring the Internet

• DIMES Distributed Internet MEasurement and
  Simulations (http//www.netdimes.org), seems to
  have made a breakthrough.
• Dont manage machines, offer a very lightweight,
  limited purpose client, and collect its
  measurements centrally.
• 100 1000 clients via word-of-mouth (Sep04 to
  Apr05). gt5000 clients now, achieved via Science
  article, slashdot. 82 countries represented. 2-3
  M measurements per day.

6
The network we analyze

• We consider the Internet at the level of its
  autonomous systems (ASes), with roughly 20,000
  nodes and 70,000 links.
• We use data gathered between March and June 2005.
  In the future, can study network dynamics, using
  intervals of months, weeks or even days.

7
k-shell method

• Use recursive pruning to peel network layers.
• To remove the 1-shell, keep removing all nodes
  with one link (degree1) until only nodes with
  degree 2 or more remain.
• To remove the 2-shell, keep removing nodes with 2
  links, until all degrees are gt 3.
• Keep going until all nodes are removed.

8
k-shell method - example
Original Graph
9
k-shell method - example
Pruning Degree 1
10
k-shell method - example
Keep Pruning Degree 1
11
k-shell method - example
Keep Pruning Degree 1
12
k-shell method - example
Pruning Degree 2
13
k-shell method - example
Keep Pruning Degree 2
14
k-shell method - example
Pruning Degree 3
15
k-shell method

• Definitions
• k-Core union of all shells with indices gt k.
• k-Crust union of all shells withindices lt k.

16
Applications

• Can use k-shell method to analyze the AS network.
• For example, color each node by its shell index
  to visualize the network.
• Next, plot quantities as a function of the shell
  index.
• Gain understanding of the network structure.
• More useful indicator then the degree.

17
AS graph colored by shells
18
Identification of a nucleus

• k-shell method enables us to identify the heart,
  or nucleus of the network as nodes in the last
  core.
• No parameters need to be fixed. (Topology
  dependent only).
• Stable over time.
• Significant ASes (tier-1) were verified to be in
  the nucleus.
• Most quantities show singular behavior at the
  last shell. Some examples -

19
Number of nodes and degrees in the shells
Slope 2.6
20
Centrality vs. shell
21
Where links go
22
Distances (vs. crusts)
Distances measured between all pairs in the
largest cluster of the crust
23
Number of site-distinct paths in the nucleus
At least 41 distinct paths between each pair
41 is the k-shell index of the nucleus
The nucleus is k-connected!
24
Beyond the nucleus

• It is left to understand the role of the other
  nodes in the network.
• We look at the connectivity properties of the
  crusts.
• Incorporate this with observations from
  previously shown plots.

25
Clusters in the crusts
Percolation Threshold
26
Structure of the AS network

• Nodes outside of the nucleus can be categorized
  into
• The fractal part nodes in the largest cluster
  of the one-before-last crust contains 70 of
  the nodes in the network.
• The rest of the nodes become then the isolated
  part.

27
Properties of the fractal part

• Connected (by construction), so that routing is
  possible without traversing and congesting the
  nucleus.
• Connections to the nucleus decrease path lengths
  significantly.
• Show fractal properties and power-laws.

28
Properties of the fractal part

• Fractal dimension calculated using the box cover
  method (SHM 2005).
• Crossover behavior between non-fractal and
  completely fractal at the percolation critical
  point.
• Percolation theory arguments predict

29
Properties of the fractal part
Percolation theory prediction slope 2.5
The 6-crust is renormalized with box of size 4
30
Properties of the isolated part

• Contains 30 of the ASes, not reachable without
  the nucleus.
• Low degree nodes, high clustering.
• Many small clusters.
• Contributions found in up to k10 shell.
• Many nodes are connected directly to highly
  connected nodes in the nucleus.

31
AS network model

• We summarize the AS network is composed of 3
  main sub-components
• Nucleus Nodes in last shell.
• Fractal Part Nodes in the largest cluster of
  the one-before-last crust.
• Isolated Part Nodes in all-but-largest clusters
  of the one-before-last crust.
• We name this model Medusa because of its
  jellyfish like structure.
• Some similarities to Faloutsous Jellyfish model
  but important differences.

32
Medusa model of the AS network
Our view of the Internet
33
The End.Thank you for your attention.
34
Comments

• Some properties (such as percolation) are found
  in the Random-Scale-Free-Model
• Internet might not be so special.
• To have more insight must investigate navigation
  with commercial restrictions Many properties
  change.

35
Clustering coefficient vs. shell
36
Nearest neighbor degree vs. shell
37
Derivation of the fractal dimension

• At the threshold, almost all the high degree
  nodes are removed, such that the network becomes
  similar to a random (Erdos-Renyi) network.
• Percolation in random networks is equivalent to
  percolation in an infinite dimensional lattice,
  in which we know the fractal dimension of the
  largest component is 4, .
• For infinite dimensional lattices, .
• Thus we conclude, ,or the
  ''shortest-path'' fractal dimension is 2.

38
Faloutsos Internet jellyfish model

• The Jellyfish Model
• Identify core of network as maximal clique. (?
  not a very robust or reproducible approach)
• Shells around network labeled by hop count from
  core (a small world)
• Find large portion of peripheral sites connect to
  core.

39
Faloutsos Internet jellyfish model

• Faloutsos view of the internet