A Dos Resilient Flowlevel Intrusion Detection Approach for Highspeed Networks

1
A Dos Resilient Flow-level Intrusion Detection
Approach for High-speed Networks

• Yan Gao, Zhichun Li, Yan Chen
• Department of EECS, Northwestern University
• Presented By
• Sudarsan Vinay Maddi
• Christopher Brandon Barkley

2
Outline

• Motivation
• Background on Sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

3
The Problem

• The increasing frequency, severity, and
  sophistication of viruses makes it critical to
  detect outbursts at routers and gateways instead
  of end hosts.

4
Current Intrusion Detection Systems

• Signature-based Detection
• Anomaly-based Detection

5
Signature-based Intrustion Detection

• Examples BRO, Snort
• Perform pattern-matching and report situations
  that match known attack types.
• Advantage Accurately detects known attack types.
• Disadvantage Attackers can modify or create
  attacks that avoid detection until a software
  update.

6
Anomaly-based Intrusion Detection

• Example Manhunt
• Build a model of acceptable behavior and flag
  exceptions using heuristics.
• Advantage Model is built according to actual use
  and can detect previously unknown attacks.
• Disadvantage Heuristic model can lead to false
  positives, system is inaccurate in the beginning
  (when it has little information).

7
Existing Network IDSes Insufficient

• Signature based IDS cannot recognize unknown or
  polymorphic intrusions
  
• Statistical IDSes for rescue, but
• Flow-level detection unscalable
• Vulnerable to DoS attacks
• e.g. TRW IEEE SSP 04, TRW-AC USENIX Security
• Symposium 04, Superspreader NDSS 05 for
  port scan detection
• Overall traffic based detection inaccurate, high
  false positives
• e.g. Change Point Monitoring for flooding
  attack
• detection IEEE Trans. on DSC 04

8
Existing Network IDSes Insufficient

• Key features missing
• Distinguish SYN flooding and various port scans
  for effective mitigation
• Aggregated detection over multiple vantage points

9
Other Limitations

• Another limitation of existing IDSes is that they
  are implemented in software.
• Software-based data recording have trouble
  keeping up with link speeds of high-speed
  routers.
• To solve this data recording must be hardware
  implementable.

10
HiFIND System

• The main goal is to develop an accurate
  High-speed Flow-level Intrusion Detection
  (HiFIND) system
• Leverage the data streaming techniques
  reversible sketches
• Select an optimal small set of metrics from
  TCP/IP headers for monitoring and detection
• Aggregate compact sketches from multiple routers
  for distributed detection

11
Goals of HiFIND

• Scalable to flow-level detection on high speed
  networks
• DoS resilient
• Distinguish SYN flooding from port scans
• Enable aggregate detection over multiple
  gateways.
• Seperate anomalies to limit false positives.

12
Deployment of HiFIND

• Attached to a router/switch as a black box
• Edge network detection particularly powerful

Monitor each port separately
Monitor aggregated traffic from all ports
Original configuration
13
Outline

• Motivation
• Background on Sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

14
Reversible Sketches

• Traditional sketches do not store key information
  making it hard to infer a culprit flow.
• Reversible sketches use a reversible hashing
  function to infer keys of culprits without
  storing explicit key information.
• More info Reversible Sketches for Efficient and
  Accurate Change Detection over Network Data
  Streams by Schweller, Gupta, Parsons, and Chen of
  Northwestern University.

15
Two Dimensional k-ary Sketch

• Instead of using one-dimensional hash table, use
  a 2D hash table matrix.
• Allows to distinguish between types of attacks by
  keeping track of more information.
• Ex. Columns are a hash of SIP,DIP, rows are a
  hash of Dport.

16
Outline

• Motivation
• Background on Sketches
• Design of the HiFIND system
• Architecture
• Sketch-based intrusion detection
• Intrusion classification with 2D sketches
• Feature analysis
• Evaluation
• Conclusion

17
Architecture of the HiFIND system
18
Architecture of the HiFIND system

• Threat model
• TCP SYN flooding (DoS attack)
• Port scan
• Horizontal scan
• Vertical scan
• Block scan
• Forecast methods
• EWMA

19
Sketch-based Detection Algorithm
20
Sketch-based Detection Algorithm

• RS(DIP, Dport, SYN - SYN/ACK)
• Detect SYN flooding attacks
• RS(SIP, DIP, SYN - SYN/ACK)
• Detect any intruder trying to attack a particular
  IP address
• RS(SIP, Dport, SYN - SYN/ACK)
• Detect any source IP which causes a large number
  of uncompleted connections to a particular
  destination port

21
Intrusion Classification

• Major challenge
• Can not completely differentiate different types
  of attacks
• E.g., if destination port distribution unknown,
  it is hard to distinguish non-Spoofing SYN
  flooding attacks from vertical scans by
• RS(SIP, DIP, SYN - SYN/ACK)

22
Intrusion Classification

• Bi-modal distribution

SYN floodings
SYN floodings
Vertical scans
Vertical scans
23
Two-dimensional (2D) Sketch

• For example differentiate vertical scan from
  SYN flooding attack
• The two-dimensional k-ary sketches
• An example of UPDATE operation

24
DoS Resilience Analysis

• HiFIND system is resilient to various DoS
  attacks as follows
• Send source spoofed SYN packets to a fixed
  destination
• Detected as SYN flooding attack
• Send source spoofed packet to random destinations
  
• Evenly distributed in the buckets of each hash
  table, no false positives
• Reverse-engineer the hash functions to create
  collisions
• Difficult to reverse engineering of hash
  functions
• Unknown hash output of each hash function
• Multiple hash tables and different hash functions
• Even know the hash functions of sketches
• Very hard to find collisions through exhaustive
  search

25
Distributed Intrusion Detection
SYN/ACK2
SYN2
SYN1
SYN/ACK1

• Naive solution
• Transport all the packet traces or connection
  states to the central site
• HiFIND
• Summarize the traffic with compact sketches at
  each edge router, and deliver them to the central
  site

26
Outline

• Motivation
• Background on Sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

27
Evaluation Methodology

• Router traffic traces
• Lawrence Berkeley National Laboratory
• One-day trace with 900M netflow records
• Northwestern University
• One day experiment in May 2005 with 239M netflow
  records, 1.8TB traffic and 11 packet samples
• Evaluation metrics
• Detection accuracy
• Online performance
• Speed
• Memory consumption
• Memory access per packet

28
Highly Accurate
29
(No Transcript)
30
Detection Validation

• SYN flooding
• Backscatter Hscans and Vscans
• The knowledge of port number
• e.g. 5 major scenarios of the top 10 Hscans

31
Detection Validation
e.g. 5 major scenarios of the bottom 10 Hscans
32
Online performance evaluation

• Small memory access per packet
• 16 memory accesses per packet with parallel
  recording
• Small memory consumption

33
Online performance evaluation

• Recording speed
• Worst case recording 239M items in 20.6 seconds
• i.e., 11M insertions/sec
• Detection speed
• Detection on 1430 minute intervals
• Average detection time 0.34 seconds
• Maximum detection time 12.91 seconds
• Stress experiments in each hour interval
• Detecting top 100 anomalies with average 35.61
  seconds and maximum 46.90 seconds

34
Outline

• Motivation
• Background on Sketches
• Design of the HiFIND system
• Evaluation
• Conclusion

35
Conclusion - Advantages

• Achieves proposed goals including scalability and
  distinguishing attack types.
• Highly accurate on test data.
• Reduction in False Positives
• Very low memory usage (13.2 MB)

36
Conclusion - Disadvantages

• HiFIND did not detect some small horizontal port
  scans that TRW detected.
• Authors said these were a combination of multiple
  small scans too stealthy for their thresholds
• Future work to further investigate this and find
  a way to account for it.

37
Conclusion Paper Disadvantages

• Authors vague on implementation, only mentioning
  it used a single FPGA board.
• Authors not explicitly define terms (e.g.
  Sketches).
• Authors do not explain or cite heuristics used to
  reduce false positives.

38
Thank You !

• Questions?