Internet Tomography: Seeing the Network through the Trees

1
Internet Tomography Seeing the Network through
the Trees

• D. Towsley
• Dept. of Computer Science
• UMass, Amherst
• towsley_at_cs.umass.edu

joint work with M. Adler, T. Bu, N. Duffield,
F. LoPresti
Revised Presented By Chun Zhang
2
Network Tomography

• Goal obtain detailed picture of a
  network/internet from end-to-end views

• infer topology /connectivity

3
Network Tomography

• Goal obtain detailed picture of a
  network/internet from end-to-end views

• infer link-level
• loss
• delay
• utilization

4
Outline

• introduction/motivation
• first attempt
• use of multicast
• forests (collections of trees)
• summary

5
Why End-to-End

• no participation by network needed
• measurement probes regular packets
• no administrative access needed
• inference across multiple domains
• no cooperation required
• monitor service level agreements
• reconfigurable applications
• video, audio, reliable multicast

6
Naive Approach I

• D0 D1 M1
• D0 D2 M2

2 equations, 3 unknowns
?
M1
M2
Di not identifiable
7
Naive Approach II

• bidirectional tree

8
Naive Approach II

• bidirectional tree

D2 D1
9
Naive Approach II

• bidirectional tree

D1D2
D2 D1
10
Naive Approach II
D0 D1 D0 D2

• bidirectional tree

D0
D2
D1
D1D2
D2 D1
11
Naive Approach II

• bidirectional tree

• 6 equations, 6 unknowns

• not linearly independent! (not
  identifiable)

12
Naive Approach III

• Round trip link delays

RAB R0 R1
RAC R0 R2

• RBC R1 R2
• Linear independence! (identifiable)
• true for general trees
• can infer some link delays within general graph
• (Shavitt, etal, INFOCOM 2001)
• measurements over cycles
• (Sidi, etal, INFOCOM 2001)

13
Bottom Line

• similar approach for losses
• yields round trip and one way metrics for subset
  of links
• approximations for other links

14
Can we do better?
15

• Correlation!
• Correlation!

16
MINC (Multicast Inference of Network
Characteristics)

• http//www-net.cs.umass.edu/minc/

17
MINC (Multicast Inference of Network
Characteristics)
source

• multicast probes
• copies made as needed within network

• receivers observe correlated performance
• exploit correlation to get link behavior
• loss rates
• delays

receivers
18
MINC (Multicast Inference of Network
Characteristics)

• multicast probes
• copies made as needed within network

• receivers observe correlated performance
• exploit correlation to get link behavior
• loss rates
• delays

?
?
19
MINC (Multicast Inference of Network
Characteristics)

• multicast probes
• copies made as needed within network

• receivers observe correlated performance
• exploit correlation to get link behavior
• loss rates
• delays

?
?
?
?
20
MINC (Multicast Inference of Network
Characteristics)

• multicast probes
• copies made as needed within network

• receivers observe correlated performance
• exploit correlation to get link behavior
• loss rates
• delays

?
?
? ?
? ?
21
MINC (Multicast Inference of Network
Characteristics)

• multicast probes
• copies made as needed within network

• receivers observe correlated performance
• exploit correlation to get link behavior
• loss rates
• delays

estimates of a1, a2, a3
22
Question How to exploit correlation given one
multicast tree

• Multicast-Based Inference of Network-Internal
  Characteristics Accuracy of Packet Loss
  Estimation
• R.Caceres N.G. Duffield J.Horowitz D.Towsley
  T.Bu
• IEEE Transactions on Information Theory (Nov 1999)

23
Modeling Loss on Multicast Trees

• tree model
• known logical mcast topology

source

• probes multicast from source node

• set of receivers R

receivers
24
Modeling Loss on Multicast Trees

• loss model
• Bernoulli losses, ak on link k
• independent between links

• data
• n probes
• Xn per rcvr record of probes

• goal
• estimate link probabilities a
  ak k ?R from X

25
Loss Estimator

• inference
• given a ak , construct probability Prob( Xn
  a) of observed data
• Maximum Likelihood Estimator
• a(n) arg maxa Prob( Xn a )
• Maximum Likelihood Estimator (MLE)
• estimator has minimal variance
• strongly consistent
• converges to true value as n ? ?
• asymptotically normal
• can find confidence intervals

26
Question How it works in real world?

• Inference of Internal Loss Rates in the Mbone
• R.Caceres N.G. Duffield S.B. Moon D.Towsley
• Proc. IEEE Global Internet'99

27
Internet Measurements

• experiments with 2- 8 receivers (100ms probes)
  summer 98
• topology determined using mtrace
• 2 minute inferences
• validation against mtrace

28
Internet Measurements
29
One Multicast Tree Limitations

• may not characterize links of interest

need two trees to characterize links of interest
Tree Layout for Internal Network
Characterizations in Multicast
Networks' M. Adler, T. Bu, R.
Sitaraman, and D. Towsley Proceedings
of Third International Workshop on
Networked Group Communication, 2001

• network tomography on a general topology
• tree layout
• inference

30
Question How to exploit correlation given
multiple multicast trees

• Network Tomography on General Topologies
• T. Bu, N. Duffield, F.L. Presti, D. Towsley
• Proceedings of ACM SIGMETRICS 2002

31
Identifiability
Green tree identifies (A,C), (C,E), (C,F)
Red tree identifies (B,D), (D,F), (D,C,E)

• identifiability problem formulated as a linear
  algebra problem
• solved by Gaussian elimination in polynomial time

32
Multiple tree inference

• Goal obtain MLE of internal behavior from
  observations from a collection of trees

A
B
aAC
aBC
C

• naïve approach
• combine results of single tree inferences

aCD
aCE
aCF
F
D
E
not MLE !
33
Maximize likelihood estimate

• NM,S No. of probes reaching M sent by source S.

A
B
aAC
aBC

• focus on aCE
• MLE for aCE
• 1 - (NE,A NE,B)/(NC,A NC,B)

C
aCD
aCE
aCF

• NC,A and NC,B not observable !

F
D
E

• apply Expectation Maximization algorithm

34
EM algorithm for loss inference

• initialize link loss rates a(0),
• a(0) (aAC(0), aBC(0), )

A
B
C
F
E
D
0 1 0
0 1 0
1 0 0
0 1 0
35
EM algorithm for loss inference

• initialize link loss rates a(0),
• a(0) (aAC(0), aBC(0), )

A
B

• at i-th iteration
• Expectation step
• Maximization step

aAC(i)
aBC(i)
C
estimate NM,S by conditional expectation
given observation data under the probability
law a(i)
aCD(i)
aCF(i)
aCE(i)
F
compute new estimates a(i1) by maximizing
likelihood
E
D
0 1 0
0 1 0
1 0 0
0 1 0

• iterate EM steps until convergence

36
Evaluating tailored EM algorithm

• simulation using Abilene mcast topology
• 11 trees chosen by fast greedy heuristics
• probes compete bandwidth with TCP/UDP traffic

EM provide good estimate to probe loss
37
Multicast-based Delay Inference

• same tree model
• delay model
• probe encounters (queueing) delay Dk on link k
• Dk ? if probe lost on link k
• Dk independent random variables

• data
• n probes
• Xn per rcvr record of delays
• goal
• estimate distribution of Dk from receiver
  traces

38
Delay Distribution Estimator

• extension of loss MLE
• estimator performance (6000 probes)

• estimating mean delay less work

39
Deployment

• multicast applications
• embed reports in Real-time Transport signaling
  Protocol (RTCP)
• web servers
• apply unicast methods to TCP packets
• string - striped ping

40
Issues and Challenges

• relationship between logical and physical
  topology
• relation to unicast
• tree layout/composition
• combining with network-aided measurements
• scalability

41
Related Work

• Multicast-based loss inference with missing
  data
• N. Duffield, J. Horowitz, D. Towsley, W. Wei, T.
  Friedman
• IEEE Journal of Selected Areas of
  Communications, 2002
• Multicast Topology Inference from Measured
  End-to-End Loss
• N. Duffield, J. Horowitz, F. LoPresti, D.
  Towsley,
• IEEE Transactions on Information Theory,
  2002
• Inferring link loss using striped unicast
  probes
• N.G. Duffield, F. Lo Presti, V. Paxson, D.
  Towsley
• IEEE Infocom 2001

42
Summary

• network tomography new, exciting research area
• intellectually challenging
• huge potential applicability
• essential ingredient
• quantifiable correlation
• elements
• statistical, algorithmic techniques

43
Thanks !