Yan%20Chen

1
Intrusion Detection and Forensics for
Self-defending Wireless Networks

• Yan Chen
• Lab for Internet and Security Technology
• EECS Department
• Northwestern University

2
Security Challenges in GIG Wireless Networks

• In addition to sharing similar challenge of wired
  net
• High speed traffic (e.g., WiMAX)
• Zero-day threats
• Lack of quality info for situational-aware
  analysis attack target/strategy, attacker
  (botnet) size, etc.
• Wireless networks are more vulnerable
• Many emerging wireless network protocols WiMAX,
  mobile IP v4/6, EAP authentication protocols

3
Self-Defending Wireless Networks

• Proactive vulnerability analysis of wireless
  network protocols (done in year 1)
• Found a class of exception triggered DoS attacks
• Net-based adaptive anomaly diagnosis, intrusion
  detection and mitigation
• Polymorphic zero-day worm signature generation
  (done in year 2)
• Automated analysis of large-scale botnet probing
  events for situation aware info (done in year 3)
• Scalable signature matching w/ massive
  vulnerability signatures (done in year 4)
• Projects done in pipeline, not serialized.

4
Accomplishments in AY 06-07

• Five conference papers
• Detecting Stealthy Spreaders Using Online
  Outdegree Histograms, in the Proc. of the 15th
  IEEE International Workshop on Quality of Service
  (IWQoS), 2007 (26.6).
• Hamsa Fast Signature Generation for Zero-day
  Polymorphic Worms with Provable Attack
  Resilience, to appear in IEEE Symposium on
  Security and Privacy, 2006 (9).
• Towards Scalable and Robust Distributed Intrusion
  Alert Fusion with Good Load Balancing, in Proc.
  of ACM SIGCOMM Workshop on Large-Scale Attack
  Defense 2006(33).
• Automatic Vulnerability Checking of IEEE 802.16
  WiMAX Protocols through TLA, in Proc. of the
  Second Workshop on Secure Network Protocols
  (NPSec) (33).
• A DoS Resilient Flow-level Intrusion Detection
  Approach for High-speed Networks, in IEEE
  International Conference on Distributed Computing
  Systems (ICDCS), 2006 (14).

5
Accomplishments in AY 07-08

• Three conference, one journal papers and two
  book chapters, and one patent filed
• Accurate and Efficient Traffic Monitoring Using
  Adaptive Non-linear Sampling Method", in the
  Proc. of IEEE INFOCOM, 2008
• A Survey of Existing Botnet Defenses , in Proc.
  of IEEE IWSSE 2008.
• Honeynet-based Botnet Scan Traffic Analysis",
  invited book chapter for Botnet Detection
  Countering the Largest Security Threat,
  Springer, 2007.
• Integrated Fault and Security Management,
  invited book chapter for Information Assurance
  Dependability and Security in Networked Systems,
  Morgan Kaufmann Publishers, 2007.
• Reversible Sketches Enabling Monitoring and
  Analysis over High-speed Data Streams, in
  ACM/IEEE Transaction on Networking, Volume 15,
  Issue 5, Oct. 2007.
• Network-based and Attack-resilient Length
  Signature Generation for Zero-day Polymorphic
  Worms, in the Proc. of the IEEE ICNP, 2007.

• Collaborated publication with Dr. Keesook Han
  from AFRL

6
Accomplishments in AY 08-09

• Five conference and one journal papers
• Using Failure Information Analysis to Detect
  Enterprise Zombies", in the Proc. of SecureComm
  2009.
• Exception Triggered DoS Attacks on Wireless
  Networks, the 39th IEEE/IFIP International
  Conference on Dependable Systems and Networks
  (DSN), 2009.
• "BotGraph Large Scale Spamming Botnet
  Detection", USENIX Symposium on Networked Systems
  Design and Implementation (NSDI) 2009.
• "Towards Efficient Large-Scale VPN Monitoring and
  Diagnosis under Operational Constraints", IEEE
  INFOCOM (main conference), 2009.
• Automating Analysis of Large-Scale Botnet
  Probing Events, ACM Symposium on Information,
  Computer and Communications Security (ASIACCS),
  2009.
• Pollution Attacks and Defenses for Internet
  Caching Systems, in Journal of Computer
  Networks, 2008.

7
Accomplishments in AY 09-10

• Two conference and four journal papers and one
  patent filed
• NetShield Massive Semantics-based Vulnerability
  Signature Matching for High-speed Networks, in
  the Proc. of ACM SIGCOMM 2010
• HiFIND A high-speed flow-level intrusion
  detection approach with DoS resiliency, Journal
  of Computer Networks, Volume 54, Issue 8, June
  2010..
• Measurement and Diagnosis of Address
  Misconfigured P2P Traffic, in the Proc. of IEEE
  INFOCOM, 2010
• Thwarting Zero-Day Polymorphic Worms With
  Network-Level Length-Based Signature Generation,
  in ACM/IEEE Transaction on Networking (ToN),
  Volume 18, Issue 1, 2010.
• POPI A User-level Tool for Inferring Router
  Packet Forwarding Priority", ACM/IEEE Transaction
  on Networking (ToN), Volume 18, Issue 1, 2010.
• "Towards Unbiased End-to-End Network Diagnosis",
  in ACM/IEEE Transaction on Networking (ToN),
  Volume 17, Number 6, Dec. 2009.

8
Overall Achievement

• 15 conference papers
• 6 journal papers
• 2 book chapters
• 2 patents filed
• Several software released to the community
• E.g. www.nshield.org
• Invited talks to Cisco, Juniper, etc. for
  potential tech transfer

8
9
NetShield Matching a Large Vulnerability
Signature Ruleset for High Performance Network
Defense
10
Outline

• Motivation
• High Speed Matching for Large Rulesets
• High Speed Parsing
• Evaluation
• Research Contributions

10
10
11
NetShield Overview

• NIDS/NIPS (Network Intrusion
  Detection/Prevention System) operation

NIDS/NIPS
Packets

• Accuracy
• Speed
• Attack Coverage

Security alerts
12
Network Level Defense

• Network gateways/routers are the vantage points
  for detecting large scale attacks
• Only host based detection/prevention is not
  enough
• Some users do not apply the host-based schemes
  due to the reliability, overhead, and conflicts
• Many users do not update or patch their system on
  time
• E.g., Conficker worm in the end of 2008 infected
  915 millions of hosts
• Cannot only reply on end users for security
  protection

13
State Of The Art
Regular expression (regex) based approaches
Used by Cisco IPS, Juniper IPS, open source Bro
Example .Abc.\x90de\r\n30

• Pros
• Can efficiently match multiple sigs
  simultaneously, through DFA
• Can describe the syntactic context

14
Cons of Regex
Limited expressive power, cannot describe
semantic context, thus inaccurate
Theoretical prospective
Protocol grammar
Practical prospective

• HTTP chunk encoding
• DNS label pointers

15
State Of The Art
Vulnerability Signature Wang et al. 04
Blaster Worm (WINRPC) Example BIND rpc_vers5
rpc_vers_minor1 packed_drep\x10\x00\x00\
x00 context0.abstract_syntax.uuidUUID_Remote
Activation BIND-ACK rpc_vers5
rpc_vers_minor1 CALL rpc_vers5
rpc_vers_minors1 packed_drep\x10\x00\x00\x0
0 opnum0x00 stub.RemoteActivationBody.actu
al_lengthgt40 matchRE(stub.buffer,
/\x5c\x00\x5c\x00/)

• Pros
• Directly describe semantic context
• Very expressive, can express the vulnerability
  condition exactly
• Accurate

• Cons
• Slow!
• Existing approaches all use sequential matching
• Require protocol parsing

16
Motivation of NetShield
16
17
Motivation

• Desired Features for Signature-based NIDS/NIPS
• Accuracy (especially for IPS)
• Speed
• Coverage Large ruleset

Cannot capture vulnerability condition well!
Shield sigcomm04
Regular Expression Vulnerability
Accuracy Relative Poor Much Better
Speed Good ??
Memory OK ??
Coverage Good ??
17
18
Research Challenges and Solutions

• Challenges
• Matching thousands of vulnerability signatures
  simultaneously
• Sequential matching ?match multiple sigs.
  simultaneously
• High speed protocol parsing
• Solutions
• An efficient algorithm which matches multiple
  sigs simultaneously
• A tailored parsing design for high-speed
  signature matching

18
19
Background

• Vulnerability signature basic
• Use protocol semantics to express vulnerabilities
• Defined on a sequence of PDUs one predicate for
  each PDU
• Example ver1 methodput len(buf)gt300
• Data representations
• For all the vulnerability signatures we studied,
  we only need numbers and strings
• number operators , gt, lt, gt, lt
• String operators , match_re(.,.), len(.).

Blaster Worm (WINRPC) Example BIND rpc_vers5
rpc_vers_minor1 packed_drep\x10\x00\x00\
x00 context0.abstract_syntax.uuidUUID_Remote
Activation BIND-ACK rpc_vers5
rpc_vers_minor1 CALL rpc_vers5
rpc_vers_minors1 packed_drep\x10\x00\x00\x0
0 opnum0x00 stub.RemoteActivationBody.actu
al_lengthgt40 matchRE(stub.buffer,
/\x5c\x00\x5c\x00/)
19
20
Outline

• Motivation
• High Speed Matching for Large Rulesets
• High Speed Parsing
• Evaluation
• Research Contributions

20
21
Matching Problem Formulation

• Suppose we have n signatures, defined on k
  matching dimensions (matchers)
• A matcher is a two-tuple (field, operation) or a
  four-tuple for the associative array elements
• Translate the n signatures to a n by k table
• This translation unlocks the potential of
  matching multiple signatures simultaneously

Rule 4 URI.Filenamefp40reg.dll
len(Headershost)gt300
RuleID Method Filename Header LEN
1 DELETE
2 POST Header.php
3 awstats.pl
4 fp40reg.dll namehost len(value)gt300
5 nameUser-Agent len(value)gt544
22
Matching Problem Formulation

• Challenges for Single PDU matching problem (SPM)
• Large number of signatures n
• Large number of matchers k
• Large number of dont cares
• Cannot reorder matchers arbitrarily -- buffering
  constraint
• Field dependency
• Arrays, associative arrays
• Mutually exclusive fields.

22
23
Difficulty of the SPM

• Bad News
• A well-known computational geometric problem can
  be reduced to this problem.
• And that problem has bad worst case bound O((log
  N)K-1) time or O(NK) space (worst case ruleset)
• Good News
• Measurement study on Snort and Cisco ruleset
• The real-world rulesets are good the matchers
  are selective.
• With our design O(K)

24
Matching Algorithms

• Candidate Selection Algorithm
• Pre-computation decides the rule order and
  matcher order
• Decomposition. Match each matcher separately and
  iteratively combine the results efficiently

• Integer range checking ? balanced binary search
  tree
• String exact matching ? Trie
• Regex ? DFA (XFA)

24
25
Outline

• Motivation
• High Speed Matching for Large Rulesets.
• High Speed Parsing
• Evaluation
• Research Contribution

25
26
High Speed Parsing
General V.S. Special Purpose
Keep the whole parse tree in memory
Parsing and matching on the fly
V.S.
Parse all the nodes in the tree
Only signature related fields (leaf nodes)
V.S.

• Design a parsing state machine
• Build an automated parsing state machine generator

27
Outline

• Motivation
• High Speed Matching for Large Rulesets.
• High Speed Parsing
• Evaluation
• Research Contributions

27
28
Evaluation Methodology

• Fully implemented prototype
• 12,000 lines of C and 3,000 lines of Python
• Release at
• www.nshield.org
• Deployed at a university DC
• with up to 106Mbps

• 26GB Traces from Tsinghua Univ. (TH),
  Northwestern (NU) and DARPA
• Run on a P4 3.8Ghz single core PC w/ 4GB memory
• After TCP reassembly and preload the PDUs in
  memory
• For HTTP we have 794 vulnerability signatures
  which cover 973 Snort rules.
• For WINRPC we have 45 vulnerability signatures
  which cover 3,519 Snort rules

28
29
Parsing Results
Trace TH DNS TH WINRPC NU WINRPC TH HTTP NU HTTP DARPA HTTP
Avg flow len (B) 77 879 596 6.6K 55K 2.1K
Throughput (Gbps) Binpac Our parser 0.31 3.43 1.41 16.2 1.11 12.9 2.10 7.46 14.2 44.4 1.69 6.67
Speed up ratio 11.2 11.5 11.6 3.6 3.1 3.9
Max. memory per connection (bytes) 15 15 15 14 14 14
29
30
Matching Results
11.0
8-core
Trace TH WINRPC NU WINRPC TH HTTP NU HTTP DARPA HTTP
Avg flow length (B) 879 596 6.6K 55K 2.1K
Throughput (Gbps) Sequential CS Matching 10.68 14.37 9.23 10.61 0.34 2.63 2.37 17.63 0.28 1.85
Matching only time speed up ratio 4 1.8 11.3 11.7 8.8
Avg of Candidates 1.16 1.48 0.033 0.038 0.0023
Max. memory per connection (bytes) 27 27 20 20 20
30
31
Scalability and Accuracy Results
Rule scaling results
Accuracy

• Create two polymorphic WINRPC exploits which
  bypass the original Snort rules but detect
  accurately by our scheme.
• For 10-minute clean HTTP trace, Snort reported
  42 alerts, NetShield reported 0 alerts. Manually
  verify the 42 alerts are false positives

Performance decrease gracefully
32
Research Contribution
Make vulnerability signature a practical
solution for NIDS/NIPS
Regular Expression Exists Vul. IDS NetShield
Accuracy Poor Good Good
Speed Good Poor Good
Memory Good ?? Good
Coverage Good ?? Good

• Multiple sig. matching ? candidate selection
  algorithm
• Parsing ? parsing state machine

Build a better Snort alternative!
32
33

• Q A
• Thanks!

34
Observations

• PDU ? parse tree
• Leaf nodes are numbers or strings

PDU
array
General V.S. Special Purpose
Keep the whole parse tree in memory
Parsing and matching on the fly
V.S.
Parse all the nodes in the tree
Only signature related fields (leaf nodes)
V.S.
34
35
Efficient Parsing with State Machines

• Studied eight protocols HTTP, FTP, SMTP, eMule,
  BitTorrent, WINRPC, SNMP and DNS as well as their
  vulnerability signatures
• Common relationships among leaf nodes
• Pre-construct parsing state machines based on
  parse trees and vulnerability signatures

Automated parsing state machine generator
UltraPAC
35
36
Example for WINRPC

• Rectangles are states
• Parsing variables R0 .. R4
• 0.61 instruction/byte for BIND PDU

36
37
Step 1 Pre-Computation

• Optimize the matcher order based on buffering
  constraint field arrival order
• Rule reorder

1
Require Matcher 1
Require Matcher 1
Require Matcher 2
Dont care Matcher 1
Dont care Matcher 1 2
n
38
Step 2 Iterative Matching
PDUMethodPOST, Filenamefp40reg.dll, Header
namehost, len(value)450
S12 Candidates after match Column 1 (method)
S2
B2
2
444
RuleID Method Filename Header LEN
1 DELETE
2 POST Header.php
3 awstats.pl
4 fp40reg.dll namehost len(value)gt300
5 nameUser-Agent len(value)gt544
R1
R2
R3
38
39
Complexity Analysis
Three HTTP traces avg(Si)lt0.04 Two WINRPC
traces avg(Si)lt1.5

• Merging complexity
• Need k-1 merging iterations
• For each iteration
• Merge complexity O(n) the worst case, since Si
  can have O(n) candidates in the worst case
  rulesets
• For real-world rulesets, of candidates is a
  small constant. Therefore, O(1)
• For real-world rulesets O(k) which is the
  optimal we can get

40
Refinement and Extension

• SPM improvement
• Allow negative conditions
• Handle array cases
• Handle associative array cases
• Handle mutual exclusive cases
• Extend to Multiple PDU Matching (MPM)
• Allow checkpoints.

40
41
Experiences

• Working in process
• In collaboration with MSR, apply the semantic
  rich analysis for cloud Web service profiling. To
  understand why slow and how to improve.
• Interdisciplinary research
• Student mentoring (three undergraduates, six
  junior graduates)

42
Future Work

• Near term
• Web security (browser security, web server
  security)
• Data center security
• High speed network intrusion prevention system
  with hardware support
• Long term research interests
• Combating professional profit-driven attackers
  will be a continuous arm race
• Online applications (including Web 2.0
  applications) become more complex and vulnerable.
  
• Network speed keeps increasing, which demands
  highly scalable approaches.

43
Research Contributions

• Demonstrate vulnerability signatures can be
  applied to NIDS/NIPS, which can significantly
  improve the accuracy of current NIDS/NIPS
• Propose the candidate selection algorithm for
  matching a large number of vulnerability
  signatures efficiently
• Propose parsing state machine for fast protocol
  parsing
• Implement the NetShield

43
44
Comparing With Regex

• Memory for 973 Snort rules DFA 5.29GB (XFA 863
  rules1.08MB), NetShield 2.3MB
• Per flow memory XFA 36 bytes, NetShield 20
  bytes.
• Throughput XFA 756Mbps, NetShield 1.9Gbps
• (XFA SIGCOMM08Oakland08)

45
Measure Snort Rules

• Semi-manually classify the rules.
• Group by CVE-ID
• Manually look at each vulnerability
• Results
• 86.7 of rules can be improved by protocol
  semantic vulnerability signatures.
• Most of remaining rules (9.9) are web DHTML and
  scripts related which are not suitable for
  signature based approach.
• On average 4.5 Snort rules are reduced to one
  vulnerability signature.
• For binary protocol the reduction ratio is much
  higher than that of text based ones.
• For netbios.rules the ratio is 67.6.

45
46
Matcher order
Reduce Si1
Enlarge Si1
Merging Overhead Si (use hash table to
calculate in Ai1, O(1))
fixed, put the matcher later, reduce Bi1
47
Matcher order optimization

• Worth buffering only if estmaxB(Mj)ltMaxB
• For Mi in AllMatchers
• Try to clear all the Mj in the buffer which
  estmaxB(Mj)ltMaxB
• Buffer Mi if (estmaxB(Mi)gtMaxB)
• When len(Buf)gtBuflen, remove the Mj with minimum
  estmaxB(Mj)