Passive Network Tomography Using Bayesian Inference

1
Passive Network Tomography Using Bayesian
Inference

• Lili Qiu
• Joint work with
• Venkata N. Padmanabhan and Helen J. Wang
• Microsoft Research
• Internet Measurement Workshop 2002
• Marseille, France

2
Motivation
Web Server
Ethernet
ATT
CW
UUNet
Sprint
AOL
Qwest
Earthlink
3
Network Diagnosis
4
Network Diagnosis (Cont.)

• Goal Determine internal network characteristics
  using passive end-to-end measurements
• Primary focus identifying lossy links
• Applications
• Trouble shooting
• Server selection
• Server placement
• Overlay network path construction

5
Previous Work

• Active probing to infer link loss rate
• multicast probes
• striped unicast probes
• Pros cons
• accurate since individual loss events identified
• expensive because of extra probe traffic

S
A
B
6
Problem Formulation

• (1-l1)(1-l2)(1-l4) (1-p1)
• (1-l1)(1-l2)(1-l5) (1-p2)
• (1-l1)(1-l3)(1-l8) (1-p5)
• Challenges
• Under-constrained system of equations
• Measurement errors

server
l1
l3
l2
l8
l7
l6
l4
l5
clients
p1
p2
p3
p4
p5
7
Gibbs Sampling

• D
• observed packet transmission and loss at the
  clients
• ?
• ensemble of loss rates of links in the network
• Goal
• determine the posterior distribution P(?D)
• Approach
• Use Markov Chain Monte Carlo with Gibbs sampling
  to obtain samples from P(?D)
• Draw conclusions based on the samples

8
Gibbs Sampling (Cont.)

• Applying Gibbs sampling to network tomography
• 1) Initialize link loss rates arbitrarily
• 2) For j 1 warmup for each link i
  compute P(liD, li) where li is
  loss rate of link i, and li ?k?I lk
• 3) For j 1 realSamples for each link
  i compute P(liD, li)
• Use all the samples obtained at step 3 to
  approximate P(?D)

9
Performance Evaluation

• Simulation experiments
• Trace-driven validation

10
Simulation Experiments

• Advantage no uncertainty about link loss rate!
• Methodology
• Topologies used
• randomly-generated 20 - 3000 nodes, max degree
  5-50
• real topology obtained by tracing paths to
  microsoft.com clients
• randomly-generated packet loss events at each
  link
• A fraction f of the links are good, and the rest
  are bad
• LM1 good links 0 1, bad links 5 10
• LM2 good links 0 1, bad links 1 100
• Link loss processes Bernoulli and Gilbert
• Goodness metrics
• Coverage correctly inferred lossy links
• False positive incorrectly inferred lossy
  links

11
Random topologies
Confidence estimate for gibbs sampling works
well and can be used to rank order the inferred
lossy links.
12
Trace-driven Validation

• Validation approach
• Divide client traces into two tomography and
  validation
• Tomography data set ? loss inference
• Validation set ? check if clients downstream of
  the inferred lossy links experience high loss
• Experimental setup
• Real topologies and loss traces collected from
  traceroute and tcpdump at microsoft.com during
  Dec. 20, 2000 and Jan. 11, 2002
• Results
• For the small subset of inferences that could be
  validated, all the inferences are correct
• Likely candidates for lossy links
• links crossing an inter-AS boundary
• links having a large delay (e.g. transcontinental
  links)
• links that terminate at clients

13
Summary

• Passive network tomography is feasible
• Gibbs sampling yields a high coverage (over 80),
  and a low false positive rate (below 5-10)
• We have developed two other inference techniques
  which trade-off accuracy for speed (more details
  in Server-based Inference of Internet
  Performance, to appear in INFOCOM03)
• Future work make loss inference in real time
• Acknowledgements
• Chris Meek, David Wilson, Christian Borgs,
  Jennifer Chayes, David Heckerman