A Signal Analysis of Network Traffic Anomalies

1
A Signal Analysis of Network Traffic Anomalies

• Paul Barford, Jeffrey Kline, David Plonka, and
  Amos Ron

2
Network Traffic Anomalies

• Failures and attacks
• Detection part of everyday work for
  administrators
• Data derived mainly from two sources
• SNMP
• Queries to nodes mostly counts of activity
• IP flows
• More specific than SNMP

3
Related Work

• Statistical detection of anomalies
• Past work on malicious (DoS, port scan) behavior
  detection
• Flash crowd studies

4
Data

• Analysis based on SNMP and IP data
• Taken from a border router at University of
  Wisconsin-Madison
• Flows sampled 1 in 96 packets
• Journal of known anomalies and events was kept
• Network
• Attack
• Flash
• Measurement

5
Current Practices

• Network operators use ad hoc methods
• Rely on operators personal experience
• Handling SNMP data
• Graph network data
• Alarms for certain events
• Flow data handling less mature
• Popular tool converts into time-series data

6
Method

• Wavelet analysis
• Divides the data into strata
• Low-frequency strata slow-varying trends
• High-frequency strata spontaneous variations

7
Wavelet Processing

• Analysis/Decomposition
• Break down the signal into the strata
• Run different filters for the different
  frequencies
• Synthesis
• Inverse of decomposition
• Wavelet algorithms
• Recombine strata, but filtering out unwanted data

8
Cont.

• The technique used by the authors synthesizes 3
  separate parts of the signal
• Total amount within the parts will be longer than
  the actual signal
• L Captures long term patterns ideal for weekly
  trends
• M Captures midrange patterns ideal for daily
  trends
• H High frequency data capture

9
Anomaly Detection

• Normalize H- and M- to a variance of 1
• Compute local variability of data within a moving
  window (3 hours)
• Combine variability of H- and M-
• Apply thresholding

10
IMAPIT

• Development environment for anomaly detection
• Used the H-, M-, and weights for both to
  determine deviation scores
• Anomalies tend to have deviation over 2.0

11
Characteristics of Ambient Traffic

• Need data free of anomalies as a calibration

12
Flash Crowds

• Test data New Linux release on ftp mirror

13
Short-lived Anomalies
14
Discriminator for Short-term Anomalies
15
Two DoS Events
16
Analysis of Network Outage
17
Deviation Score Evaluation

• Used logged anomalies as baseline for evaluation
• Of 39 logged anomalies, detected 38

18
Comparison to Holt-Winters

• Holt-Winters is an exponential smoothing
  algorithm
• Uses baseline (intercept), linear trend (slope),
  and seasonal trend
• Aberrations are detected by detecting a certain
  amount of data outside the threshold range within
  a window
• Different from wavelet in that the different
  strata are processed separately whereas
  Holt-Winters is one prediction function
• Compared to an alternative using Holt-Winters
  algorithm
• Holt-Winters detected 37 anomalies
• Both missed anomalies would have been detected
  with a larger window
• Holt-Winters more sensitive

19
Conclusion

• Performs comparably to Holt-Winters
• Deviation score detection can be effective
• Learning methods potentially used in the future
• Study ways of classification