collaborators: Mark Coates, Rui Castro, Ryan King,

1

Network Bandwidth Estimation and Tomography
Rob Nowak Rich Baraniuk UW-Madison
Rice University
collaborators Mark Coates, Rui Castro, Ryan
King, Mike Rabbat, Yolanda Tsang, Vinay
Ribeiro, Shri Sarvotham, Rolf Reidi
spin.rice.edu
2
Internet Boom
1993
1969

• too complex to measure everywhere, all the time
• traffic measurements expensive (hardware,
  bandwidth)

3
Proprietary Concerns

• companies do not share data or performance
  information

4
Networking 101

• bits are bundled into packets
• packets go through routers
• queues absorb bursts of packets
• congestion queues fill up, large delays, packet
  drops

5
Network Measurement Inference
Path Modeling and Bandwidth Estimation
equivalent model
Internet
Network Tomography
6
Brain Tomography
counting projection
MRF model
Poisson
7
Network Tomography
routing counting
binomial / multinomial
queuing behavior
8
Network Tomography
Vardi 96, Tebaldi West 98 Cao, Davis,
Vander Wiel, Yu 00
From link-level traffic measurements, infer
end-to-end traffic flow rates
9
Network Tomography (MINC Project,
Towsley-Duffield)

• y packet losses or delays
• measured at the edge
• A routing matrix (graph)
• ? packet loss probabilities
• or queuing delays
• for each link
• randomness inherent
• traffic measurements

likelihood function
10
Probing the Network
cross-traffic
delay
probe packet stripe
Probe packets experience similar queuing effects
and may interact with each other
11
Network Tomography The Basic Idea
sender
receivers
12
Network Tomography The Basic Idea
sender
receivers
13
Maximum Likelihood Estimation via EM
Problem How to compute maximum likelihood
estimates of link-by-link loss/delay
distributions from end-to-end measurements ?
14
Topology ID via Probe Interactions
15
Topology ID via Probe Interactions
16
Finding the Maximum Likelihood Tree
Stochastic search through forest via
Metropolis-Hastings
17
Internet measurement experiments
True topology
estimated topology
18
What have we learned?

• Clever probing and sampling schemes
• reveal hidden network structure and behavior
• Simple inference algorithms are
• effective, intuitive, easy to implement, scale
  nicely
• MLE criteria are easily modified to include
  prior information
• Bayesian or regularized MLE methods are
  straightforward

Complex interplay between measurement/probing
techniques, statistical modeling, and
computational methods for optimization
19
Open Problems Placement/Coverage
How should measurement devices be deployed
? Logical graph coverage of physical topology
? Can random graph models shed some light ?
20
Open Problems Spatio-temporal Correlation
competing traffic
Long-range dependence of network traffic
Correlations due to competing traffic flows
Can we detect correlations? Can we exploit them
in measurement and mapping applications? Fuse
tomography and bandwidth estimation
21
Open Problems Detection and Localization
How can we capitalize on conventional wisdom
most links are good and only a few are
bad ?
Detecting and locating anomalous behavior rather
than estimating everything
Estimation
Hypothesis Testing
22
Open Problems Timing and Synchronization
sender
receiver
network
sender monitor
receiver monitor
How to accurately measure time ?

• Hardware solutions (expensive)
• Software solutions (more practical)
• - sophisticated software clocks (Veitch 02)
• - crude software clocks (ICMP timestamping)
• and statistical averaging

23
Open Problems Network Security
How can measurement and monitoring across the
Internet help detect and prevent malicious
activities ?