Networkbased Intrusion Detection, Prevention and Forensics System

1
Network-based Intrusion Detection, Prevention and
Forensics System

• Yan Chen
• Department of Electrical Engineering and Computer
  Science
• Northwestern University
• Lab for Internet Security Technology (LIST)
• http//list.cs.northwestern.edu

2
The Spread of Sapphire/Slammer Worms
3
Current Intrusion Detection Systems (IDS)

• Mostly host-based and not scalable to high-speed
  networks
• Slammer worm infected 75,000 machines in
• Host-based schemes inefficient and user dependent
• Have to install IDS on all user machines !
• Mostly simple signature-based
• Cannot recognize unknown anomalies/intrusions
• New viruses/worms, polymorphism

4
Current Intrusion Detection Systems (II)

• Cannot provide quality info for forensics or
  situational-aware analysis
• Hard to differentiate malicious events with
  unintentional anomalies
• Anomalies can be caused by network element
  faults, e.g., router misconfiguration, link
  failures, etc., or application (such as P2P)
  misconfiguration
• Cannot tell the situational-aware info attack
  scope/target/strategy, attacker (botnet) size,
  etc.

5
Network-based Intrusion Detection, Prevention,
and Forensics System

• Online traffic recording
• SIGCOMM IMC 2004, INFOCOM 2006, ToN 2007,
  INFOCOM 2008
• Reversible sketch for data streaming computation
• Record millions of flows (GB traffic) in a few
  hundred KB
• Small of memory access per packet
• Scalable to large key space size (232 or 264)
• Online sketch-based flow-level anomaly detection
• IEEE ICDCS 2006 IEEE CGA, Security
  Visualization 2006
• Adaptively learn the traffic pattern changes
• As a first step, detect TCP SYN flooding,
  horizontal and vertical scans even when mixed
• Online stealthy spreader (botnet scan) detection
• IWQoS 2007

6
Network-based Intrusion Detection, Prevention,
and Forensics System (II)

• Polymorphic worm signature generation detection
• IEEE Symposium on Security and Privacy 2006,
  IEEE ICNP 2007
• Accurate network diagnostics
• ACM SIGCOMM 2006 IEEE INFOCOM 2007 (2)
• Scalable distributed intrusion alert fusion w/
  DHT
• SIGCOMM Workshop on Large Scale Attack Defense
  2006
• Large-scale botnet and P2P misconfiguration event
  forensics work in progress

7
System Deployment

• Attached to a router/switch as a black box
• Edge network detection particularly powerful

Monitor each port separately
Monitor aggregated traffic from all ports
Original configuration
8
P2P Doctor Measurement and Diagnosis of
Misconfigured Peer-to-Peer Traffic
Anup Goyal, Zhichun Li, Yan Chen and Aleksandar
Kuzmanovic Lab for Internet and Security
Technology (LIST) Northwestern Univ.
9
What is P2P Misconfiguration

• P2P file sharing accounted for 60 of traffic
  in USA and 80 in Asia
• Thousands of peers send P2P file downloading
  requests to a random target on the Internet
• possibly triggered by bugs or by malicious
  reasons
• generates large amount of unwanted traffic
• It contributes on an average of about 30 of the
  Internet background radiation

10
Motivations

• On Dec. 6th, 2006, 5,047 sources generated
  31,000 packets/sec and 11MB/s of traffic to a
  single unused IP in Northwestern University
• P2P software DC has already been exploited by
  attackers for DoS
• direct gigabit junk data per second to a victim
  host from more than 150,000 peers
• Currently, little is known about the
  characteristics or root causes of P2P
  misconfiguration events

11MB/s
11
Outline

• Motivation
• Passive measurement results
• P2P Doctor system design
• Root cause diagnosis and analysis
• Conclusion

12
Peer Classification
Poisoned Peers (Intentional)
Unintentionally Misconfigured peers
All the peers
Normal Peers
Bogus Peers
Anti-P2P Peers
Not in the P2P Network
In the P2P Network
13
Passive Measurement

• Honeynet/honeyfarm datasets
• Events of unique sources 100 in 6 hours
• After filtering scan traffic
• Event characteristics
• Mostly target a single IP
• Duration A few hours to up to a month

14
Popularity
30!

• Growth Trend
• IP space
• Observed in three sensors in five different /8 IP
  prefixes

15
Further Diagnosis

• Problems with passive measurement on archived
  data
• Events have gone
• Hard to backtrack the propagation
• Root cause?
• Need a real-time backtracking and diagnosis
  system!

16
Outline

• Motivation
• Passive measurement results
• P2P Doctor system design
• Root cause diagnosis and analysis
• Conclusion

17
Design of P2P Doctor System
Backtracking system
P2P-enabled Honeynet
Root cause inference
P2P payload signature based responder
Event identification
Protocol parsing for metadata
18
Design of P2P Doctor System
Backtracking system
P2P-enabled Honeynet
Root cause inference
Index Server (tracker) Crawling BT top 100,
eMule 185
DHT Crawling
Peer Exchange Protocol Crawling
19
Design of P2P Doctor System
Backtracking system
P2P-enabled Honeynet
Root cause inference

• What is the root cause?
• Which peers spread misconfigurtion?
• How is misconfiguration disseminated?
• What is the percentage of bogus peers in the
  misconfigured P2P networks?

20
Deployment and Data Collection

• Deployed the P2P doctor system on NU honeynet (10
  /24 networks in three /8)
• Real-time events
• Previous passive measurement data referred as
  historical events

21
Outline

• Motivation
• Passive measurement results
• P2P Doctor system design
• Root cause diagnosis and analysis
• Conclusion

22
Root Cause Analysis

• Methodology
• Track how honeynet IPs propagated in P2P systems
• Use unroutable IP space as a big honeynet (66.8
  of IPv4 Space)
• Hypothesis formulation and testing
• Classification of measured peers
• Misconfigured peers Passively observed from
  honeynet
• Backtracked peers actively observed through
  backtracking
• Reverse honeynet peers the IP obtained by
  reversing the target IP from the honeynets
• Results
• Data plane traffic radiation
• Detailed results focus on eMule and BitTorrent

23
Data Plane Traffic Radiation
1.2.3.4
Resource mapping
Who has Beowulf.avi?
1.2.3.4
24
eMule Root Cause

• Byte ordering is the problem!

4.3.2.1
1.2.3.4
1.2.3.4
4.3.2.1
4.3.2.1
4.3.2.1
4.3.2.1
25
eMule Root Cause

• Byte ordering is the problem!
• Hypothesis from the historical data
• In 80 of events, the reverse target IPs are
  alive
• Verified with real-time events
• 61 of the reverse honeynet peers indeed running
  eMule with the port number reported
• For the backtracked peers which is in the
  unroutable IP space, 69.6 of them having reverse
  IPs run eMule

26
eMule Peers Dissemination

• Which peers spread misconfiguration?
• 99.24 of misconfigured peers are normal peers
• How is the misconfiguration disseminated?
• Index Server? No
• Peer exchange? Yes
• Percentage of bogus peers in eMule network?
• 12.7, 25.0 w/ a total of 37,079 backtracked
  peers

27
BitTorrent Responsible Peers

• Both anti-P2P and normal peers are responsible
• Events classified to two types with diagonally
  different sets of characteristics
• For anti-P2P peers events
• All the sources are from the IP range owned by
  anti-p2p companies like Media Defender, Media
  Sentry, Net Sentry etc.
• Seen 6 out of 7 major anti-P2P companies sources
  in our honeynet.

28
BitTorrent Root Cause

• Refuted Byte Ordering Hypothesis
• For 20 real-time events, no reverse honeynet
  peers runs BitTorrent
• For normal peer events, culprit is Peer Exchange
  (PEX) protocol implemented by uTorrent-compatible
  clients
• For anti-P2P peer events
• Possibly related to Azureus system
• Still an open question (No real-time events)

29
BitTorrent Dissemination

• How is misconfiguration disseminated?
• Index server? - No
• Peer exchange? - Yes
• Percentage of bogus peers in BitTorrent network?
• Out of a total of 9,000 backtracked peers, only
  13 IPs are unroutable and 3,150 IPs gave
  connection timeout
• 0.14

30
Conclusions

• The first study to measure and diagnose
  large-scale P2P misconfiguration events
• Found 30 Internet background radiation is caused
  by P2P misconfiguration
• Popular in various P2P systems, exponential
  growth trend, and scattered in the IPv4 space
• For eMule, we found it is caused by network byte
  order problem
• For BitTorrent, classified to anti-P2P peer
  events and normal peer events with diagonally
  different sets of characteristics
• Found the uTorrent PEX causes the problem in
  normal peer events

31

• ? ? ?

32
Backup Slides
33
Motivation

• Given unprecedented amount of traffic, even a
  slight mis-configuration of the P2P system can
  result in a DDoS kind of situation
• Prevalence in time, space, and across a number of
  distinct P2P systems with a temporal increasing
  trend is alarming.
• P2P miscongurations can cause innocent people to
  get involved in the above war between P2P and
  anti-P2P systems.
• Presently, nothing is known about the causes or
  overall effects of P2P mis-configurations
• Our goal is to determine the root cause(s) of
  each type of mis-configuration

34
Related Work

• Misconguration is widely spread across different
  networked and distributed systems like BGP
  Labovitz et al. and firewalls Cuppens et al.
  .
• Measurement studies of normal P2P traffic ACM
  SOSP (2003), MCN (2002), while we measure the
  abnormal P2P traffic observed in honeynets.
• In INFOCOM (2005), Content pollution including
  intentional and unintentional pollution is
  widespread for popular titles.
• P2P systems like Fasttrack and Overnet are
  vulnerable to the index poisoning attack INFOCOM
  (2006)
• All of the above studies focus on the content
  pollution or index poisoning while our focus is
  the index misconfiguration.
• First large-scale measurement study on the root
  causes for both intentional/unintentional index
  misconfiguration.

35
What is P2P Misconfiguration

• More than 50 of the traffic in the Internet
  today is P2P traffic
• By Symantec Corporations recent report
• P2P file sharing accounted for 60 of traffic
  in USA and 80 in Asia