InOut Traffic Proportion Based Analyses for Network Anomaly Detection

1
In/Out Traffic Proportion Based Analyses for
Network Anomaly Detection

• By Zhang FengXiang
• 2006-07-17

2
Outline

• Research background
• Traffic analyses for anomaly detection
• Based on input/output proportion of traffic
• Applying GLR test and Bin-test
• Numerical examples and discussions
• Conclusions further works

3
What is the network anomaly

• Anomaly Operations deviate from normal behavior.
  
• What could cause anomaly?
• Malfunction of network devices
• Network overload
• Malicious attacks, like DoS/DDoS attacks
• Other network intrusions
• Two main kinds of network anomalies.
• 1. Related to network failures and performance
  problems.
• 2. Security-related problems
• (1) Resource depletion
• (2) Bandwidth depletion

4
Anomaly detection meets troubles

• There are many schemes based on checking abrupt
  traffic changes.
• E.g. apply signal processing technique to detect
  out traffics abrupt change
• However, this kind of anomaly does not always
  mean illegitimate.
• Abrupt change of traffic does not mean an attack
  has exactly happened
• We call this case as

Legitimately-abrupt-change (LAC)
5
Legitimately abrupt changes

• Example 1
• Famous information gateway websites, e.g. Yahoo.
• When bombastic news is announced, it would
  appear.
• Example 2
• Special information announce center, e.g. the
  website of national meteorological agency
• When a nature disaster is said to be coming, it
  would occur.
• Typhoon, Earthquake, Tsunami
• Important outdoor holidays

6
Existing anomaly detection schemes trouble

• For those detection schemes based on abrupt
  changes of the unidirectional traffic
• When legitimately abrupt changes appear, false
  alarms might appear.
• However, the bidirectional traffic would have
  some kinds of symmetry
• Check the Input/Output traffic proportion.
• Test their Generalized Likelihood Ratio (GLR).
• Test expected proportion number in each special
  value range (Bin).

7
Network Model of Analyzing
Input/Output Proportion
In
Out
Near the protected object
8
In/Out Traffic Proportion Based Analyses

• In/Out proportion, GLR and Bin test

9
Detect abnormal changes of proportion

• For existing LACs, we consider bidirectional
  traffics.
• For this case, the Input/Output proportion would
  not change abruptly as well
• It seems be in a relatively narrow range.
• Due to the nature of the TCP protocol there is a
  loose symmetry on the In/Out packet rates.
• In the legitimate use of networks, more are the
  request packets, more are the response packets.
• Almost all bandwidth attacks destroy this
  attribute.

10
Generalized Likelihood Ratio test

• In statistical analysis, network anomalies are
  modeled as correlated abrupt changes in time
  series of network data.
• GLR shows the likelihood of the residuals in two
  adjacent windows.
• Abrupt changes are detected by comparing the
  variance in two windows.
• When GLR is closer to 1, the data distribution in
  test window is more likely to happen after the
  learn window
• It is more likely to be anomaly when GLR is
  smaller then a preset threshold.

11
How to do GLR test

• Get the In/Out proportion sequence
• Apply GLR scheme between two adjacent windows

12
Calculation of GLR

• Abrupt changes in time series data can be modeled
  using an auto-regressive (AR) process.
• Abrupt changes are correlated in time, yet are
  short-range dependent.
• As some other detection schemes, we use an AR
  process of order 1 here to model the data in a
  80-sec window.

13
W the length of each window
SL, SS the sample variance of the residual in
the learn and test window SP the pooled
sample variance of two adjacent windows
the GLR with the value range (0,1
14
The analyzed traffic data

• Use 4 traffic sets between the Science
  Information Network (SINET) and other two
  commercial Internet exchange service networks,
  JaPan Internet eXchange (JPIX) and JPNAP. They
  are bit rates in
• 1. 24 hours on 10 Gigabit Ethernet line of JPIX
  from 1744 on May 03, 2005.
• 2. 24 hours on 10 Gigabit Ethernet line of JPIX
  from 1306 on March 25, 2004.
• 3. 4 hours on 10 Gigabit Ethernet line of JPIX
  from 1401 to 1801on March 24, 2004.
• 4. 24 hours on 10 Gigabit Ethernet line of
  JPNAP from 1744 on May 03, 2005.

15
SINET ? ? JPIX ( 1 day traffic )
16
The GLR sequence of the bit rate proportion time
series between JPIX and SINET
17
SINET ? ? JPNAP ( 1 day traffic )
18
The GLR sequence of the bit rate proportion time
series between JPNAP and SINET
19
The percentage distribution of the GLR value

• Most GLR values are close to 1, and mostly above
  0.8.
• This means the distribution of Input/Output
  traffic proportion is most likely to its former
  one.

20
Bin-test scheme

• According to proportion data, we can decide
  several value ranges (bins).
• From most frequently appearing value range to the
  seldom appearing value range
• Give the expected number of proportions in each
  bin under the normal and legitimate case.
• Test the data points in the observing window
• If not match the expected distribution of the
  bins, alert.

21
Proportion of Gigabit Ethernet line of JPIX to
SINET
March 24/2004(1401 -gt 1801)
22
An illustration of Bin-test
5
Get the expected number Ni in the ith bin In
higher level bin the Ni should be larger.
1st Most common
4
2nd Most common
3
Count data number ni in each bin Compare ni with
Ni. If the deviation exceeds some confidence
interval, an anomaly is declared.
Normal
2
seldom
1others
never
23
Bin-test based on Input/Output proportion

• Four data sets number distribution in 4 bins

24
Conclusions and future works

• Weve noticed the effects of the legitimately
  abrupt changes for anomaly detections.
• Showed the bidirectional In/Out traffic monitored
  for the same networks is close to a constant.
• A valuable reference for reduction of false
  positive alarms in the detection of bandwidth
  attacks.
• Proposed a Bin-test detection method based on
  traffic analysis.
• In the future,
• Further study the In/Out traffic proportion
  constancy.
• Simulate DoS/DDoS attacks and apply the detection
  scheme.

25
Thank You!
Advices?