A Non-intrusive, Wavelet-based Approach To Detecting Network Performance Problems

1
A Non-intrusive, Wavelet-based Approach To
Detecting Network Performance Problems

• Polly Huang ETH Zurich
• Anja Feldmann U. Saarbruecken
• Walter Willinger ATT Labs-Research

2
Road Map

• Motivation and rationale
• Mechanism details
• Conclusion and outlook

3
Performance Problem
Web
Web
TCP
TCP
Google.com
Internet
Network
Network
Link/Physical
Link/Physical
server
proxy
congestion
congestion
routing
routing
else
else
4
Current State

• Active probing
• Ex traceroute, ping
• Disturbing - injecting unnecessary traffic
• Biasing - distort metrics of interest
• Heisenberg effects
• Passive measurements
• Ex Cisco NetFlow, IP Accounting, other
  packet-level measurment
• give much information
• Do not infer problems inside the network

5
What Would Be Cool

• Passive
• Trigger alerts in real time
• For problems due to
• Server load
• Congestion
• Routing error
• Common Symptoms
• Delay and drop

6
TCPs Closed-loop Control

• Delays/drops reflected in RTT/RTO estimations
• RTT round trip time
• RTO retransmission timeout
• Quality of Network Path
• Values of RTT/RTO estimations
• Amounts of RTT/RTO samples
• Can be measured passively

7
Detailed Estimation

• Methodology
• A hash table of all data packets observed
• One RTT sample per data-ack pair
• One RTO sample per data-data pair
• Slow
• packets/observation period
• especially with high date rate connections (the
  likely trouble makers)

8
Objectives

• Passive measurement
• Non-intrusive
• Infer quality of network paths
• Detecting network performance problem
• Efficiently (so can be done in real time)
• Wavelet-based technique

9
Road Map

• Motivation and rationale
• Mechanism details
• Conclusion and outlook

10
Wavelet-based Technique

• Theoretical ground
• Wavelet transform
• Energy plots (or scaling plots)
• Interpreting energy plots
• WIND, the problem detection tool
• Features examples
• Detection methodology
• Validation effort

11
Theoretical Ground

• FFT
• Frequency decomposition
• fj, Fourier coefficient
• Amount of the signal in frequency j
• WT wavelet transform
• Frequency (scale) and time decomposition
• dj,k, wavelet coefficient
• Amount of the signal in frequency j, time k

12
Wavelet Example
1
0
-1
00 00 00 00 11 11 11 11
s1 s2 s3 s4
d1 d2 d3 d4
13
Self-similarity

• Energy function
• Ej S(dj,k)2/Nj
• Self-similar process
• Ej 2j(2H-1) C lt- the magic!!
• log2 Ej (2H-1) j log2C
• linear relationship between log2 Ej and j

14
Self-similar Traffic
15
Effect of Periodicity
self-similar
Internet Traffic
16
Adding Periodicity

• packets arrive periodically, 1 pkt/23 msec
• coefficients cancel out at scale 4

17
Simulation TrafficSingle RTT
18
Simulation TrafficCongestion
19
Interpreting Energy Functions

• Abrupt knees at
• RTT time scale
• RTO time scale
• Knee shifts
• RTT/RTO time changes
• Low energy level (after normalization)
• congestion
• low traffic volume

20
WIND - The Detection Tool

• Wavelet-based
• Inference for
• Network
• Detection

• Based on libpcap and tcpdump
• On-line mode (efficient)
• Per packet compute dj,k
• Per observation period output Ej
• On a subnet basis
• Off-line mode
• Detailed RTT/RTO estimation

21
Real TrafficBy Subnets
22
Real TrafficBy Periods
23
Real TrafficBy Periods
24
Detecting Methodology

• Reference function
• Smoothed average
• Difference
• Area below the reference function
• Weighted sum by scale
• Flagged interesting
• Top 10 deviations

25
Pick Out Interesting Ones26, 30, 31
26
Validation By

• WIND off-line mode
• Detailed RTT/RTO estimations
• Volume
• Similar heuristics (area difference)
• CCDF of RTT/RTO
• Ratio of RTO/RTT
• Volume

27
Validate period 26, 30, 31
CCDF of RTO pick out period 23, 26, 31
CCDF of RTT pick out period 29, 30, 31
80-90 are validated interesting
28
Road Map

• Motivation and rationale
• Mechanism details
• Conclusion and outlook

29
Summary

• Detect problems using energy plots
• If self-similar, clean linear relationship
• If periodic, getting knees
• If problems, knee shifts or low energy level
• WIND the online/offline analysis tool
• Passive
• Efficient

30
Outlook

• Full-fledged diagnosing tool
• More sophisticated heuristics
• Use of traceroute data
• Illustrative examples
• Using the tool (beta release)
• Using the methodology

31
Questions?

• http//www.tik.ee.ethz.ch/huang