Distributed Data Parallel Techniques for ContentMatching Intrusion Detection Systems

1
Distributed Data Parallel Techniques for
Content-Matching Intrusion Detection Systems

• Christopher V. Kopek, Errin W. Fulp, and Patrick
  S. Wheeler

Department of Computer Science Wake Forest
University Winston-Salem, NC
Department of Computer Science University of
California at Davis Davis, CA
2
Signature-Based IDS

• Network Intrusion Detection Systems (IDS)
• Single entity located at the network edge
• Scans packet payloads for malicious content
• IDS requires more processing than filtering
• IDS is a bottleneck and vulnerable to DoS
• Need techniques to improve IDS performance

3
Snort

• A multi-mode packet analysis tool'' by Martin
  Roesch
• Snort has the following primary modules
• Packet capture
• Preprocessing performs various operations
• Content normalization
• Detection engine applies rules (content matching)
• Alert engine performs the matching rule
  action

4
Snort Rules

• Snort rules have two parts, rule header and rule
  options
• Header describes the action and packets to
  consider
• Options provides more details packet attributes
  (if needed)
• For example, consider the snort rule
• Rule header alerts when TCP traffic is observed
  originating from any network with any source
  port, destined for network 10.1.1.x to
  destination port 222
• Keyword content in option field requires the
  payload to be searched for the pattern
• Multiple content fields may exist, longest used
  for initial match

5
What is the Problem?

• Content searching is very time consuming
• Initial match and verification
• Others have reported from 40 to 75

6
Content Matching Algorithms

• Essential for any signature-based IDS
• Algorithms were not necessarily motivated by IDS
• It is just string searching
• Multi-pattern search has increased performance
• But it is not enough for future networks
• Parallel IDS provides a scalable solution

7
Parallel IDS

• Perform bottleneck operation (match) in parallel
• Data Parallel (DP)
• Distribute packets across processing elements
• Reduces the arrival to any one processing element
• Requires load balancing
• New approach, Divided Data Parallel (DDP)
• Divide data of one packet across processing
  elements
• Each processing element has a smaller part
  (fragment)

8
Dividing the Payload

• DDP divides the packet payload into fragments
• Fragments given to processing elements, complete
  payload scanned
• Signature (malicious content) may span fragment
• Single processor may not see complete signature
• Must overlap fragments to prevent false negatives
• Overlap dependent on largest signature, p,
  (example above p 3)
• Overlap is (p-1) with leftmost fragment

Fragment 0
Fragment 2
(p-1)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
17
16
15
(p-1)
(p-1)
Fragment 1
Fragment 3
9
Match Bit

• Only considering the initial match
• Once a match made, can skip remaining packet
• Original DDP does not
• Once a match is made, set match bit for the
  packet
• Match bit indicates do not process remaining
  fragments
• Allows processors to skip fragments
• Processors can then process different packets
• Another form of parallelism achieved

Packet Divider
Packet 3 Frag 0
Packet 2 Frag 1
Packet 2 Frag 0
Packet 2 Match Bit
Packet 3 Match Bit
Packet 1 Match Bit
Packet 1 Frag 0
Packet 1 Frag 1
Packet 1 Frag 2
Packet 0 Match Bit
Packet 0 Frag 0
Packet 0 Frag 1
Packet 0 Frag 2
Processor 2
Processor 0
Processor 1
10
Performance

• Compare different parallel match performance
• Data parallel and forms of distributed data
  parallel
• Created a multi-threaded match for Snort 2.6
• Traffic traces from a university
• Rules from govt' agency (345 total web-rules)
• Recorded processing time and speed-up

11
Results

• DDP performed better than DP
• DDP with match bit performed the best
• Speed-up was often greater than number of
  processors

12
Future Work

• IDS is increasingly important
• Searches packet payload for threats
• Unfortunately search is time consuming
• Distributed Data Parallel IDS
• Must overlap fragments
• Match-bit improves performance
• Only considered initial match
• Verification must be done
• Hierarchical system can be used
• Proper fragment distribution is a concern

13
Initial Match Percentage