Gigabit Rate Packet Pattern-Matching Using TCAM

1
Gigabit Rate Packet Pattern-Matching Using TCAM

• Fang Yu and Randy H. Katz
• UC Berkeley
• T. V. Lakshman
• Bell Laboratories, Lucent Technologies

2
Motivation

• Numerous malicious probes and worms
• End-host based solution is not sufficient
• It is hard for all end users to apply patches
  quickly
• Worms can contaminate millions of hosts within
  hours
• Network based solution network intrusion
  detection systems (NIDS)
• Perform packet scanning for complicated worm
  patterns in the network
• Stop worms from reaching end hosts
• Easy to manage for network administrators

3
Pattern Matching for NIDS

• Thousands of complicated patterns
• Patterns have variable lengths
• Patterns with correlation
• abc followed by cde within 3 bytes
• Patterns with negation
• user not followed by 0a within 50 bytes
• Require packet payload scanning
• Not supported by most current network devices,
  which support packet header processing only

4
Current Pattern Matching Schemes

• Software based solutions
• Speed is slow
• FPGA solutions
• Build large DFA or NFA for all patterns
• Build a KMP based search engine for each pattern
• Bloom Filters
• One bloom filter for each pattern length
• Not scalable when pattern lengths vary
  dramatically

5
Ternary-CAM (TCAM)

• Fully associative memory compare input string
  with all the entries in parallel
• If multiple matches, report the index of the
  first match
• Each cell takes one of three logic states
• 0, 1, and ?(dont care)
• Current TCAM technology
• Fast Match Time 4 ns
• Size 1-2MB
• Width configurable
• 1024 entries 1024 bytes width
• 2048 entries 512 bytes width

cell
entry
width
6
Pattern Matching with TCAM

• Put all the patterns into the TCAM
• Assume patterns are less or equal to the TCAM
  width
• If less than the TCAM width, pad with ?
• Order the patterns according to lengths in
  reverse order
• When matching entry ABC, report matching of both
  pattern ABC and AB
• Shift one byte each time

7
Analysis

• Scan speed
• 4 ns per TCAM lookup, shift one byte at a time
• 8bits/4ns 2 Gbps worst case scan rate
• Limitation require all the patterns to be
  shorter or equal to the TCAM width
• Set the TCAM width gt longest patterns length
• Pad all short patterns to TCAM width
• Waste TCAM resources
• Can we set TCAM width smaller and cut long
  patterns into smaller patterns?

8
Long Patterns

• Cut long patterns into smaller patterns
• TCAM width w4 bytes
• DEFGABCDL is split into DEFG, ABCD, and L
• Pad the last partial pattern with the tail of
  the second last partial pattern
• DEFGABCDL is split into DEFG, ABCD, and BCDL

DEFGABCDL
DEFGABCDL
DEFG
DEFG
ABCD
ABCD
L
BCDL
Short partial patterns, many TCAM hits
9
Concatenate Partial Patterns into Long Patterns
Matching Table

• Patterns
• ABCDABCD
• DEFGABCDL
• DEFGDEF
• DEF
• ,

Prefix Index Suffix Index Distance Matched Long Pattern Index
1(ABCD) 1(ABCD) 4 ABCDABCD
2(DEFG) 1(ABCD) 4 3(DEGFABCD)
2(DEFG) 3(GDEF) 3 (DEGFDEF
3(DEGFABCD) 1(ABCD) 4 ABCDABCD
3(DEGFABCD) 2(BCDL) 1 DEFDABCDL
Partial Hit List (PHL)
Position Matched entry


Position Matched entry
1 2(DEFG)

Position Matched entry
5 3(DEFGABCD)

10
Correlated Patterns

• One pattern after another
• E.g. ABCD followed by DEF within 10 bytes
• The matching result of ABCD has to be in PHL
  for 10 positions

11
Matching Process

• TCAM reports a miss
• No extra memory lookup
• TCAM reports a hit
• If it is a partial pattern
• For every item in PHL
• One memory lookup into matching table to see
  whether it generates a valid pattern
• Examples based on statistical analysis
• n 2000, mi 200 bytes, w 4 bytes. Associate
  hit rate is 2.2e-5, PHL size is 8.8e-5
• w 8 bytes, associate hit rate is 2.6e-15, PHL
  size is 2.08e-14

Associate hit rate
PHL size
12
Malicious Attack?

• Window distance between two correlated patterns
• After matching a pattern, what is possibility to
  match another at window size j positions later?

• When j 1, probability is
• 1-
• E.g., n 1000 and m4, it is 0.029
• When j increases, the probability increases. If
  jm, then probability 1

.

• Worst case PHL size is at least window size / m

13
Simulation Results on ClamAV

• ClamAv virus signature database
• Version 0.15, which contains simple patterns only
• 1768 patterns, varying from 6 bytes to 2189 bytes

14
Effect of TCAM Width

• Total TCAM space
• Increase when w increases, because of padding
• Mapping Table Size
• Decreases as w increases because of fewer partial
  patterns

15
PHL Size on Real Data

• For each packet, record average and maximum PHL
  size
• Avg mean of the average PHL size over all
  packets
• AvgMax mean of the maximum PHL sizes
• Max maximum PHL size over all packets

TCAM Width MIT Dump MIT Dump MIT Dump Berkeley Dump Berkeley Dump Berkeley Dump
TCAM Width Avg Avg Max Max Avg Avg Max Max
4 0.042 0.27 4 0.03 0.48 4
8 4.8e-6 5.6e-4 8 1.e-6 1.9e-5 7
16 0 0 0 4.3e-7 5.8e-6 3
16
Simulation Results on Snort

• SNORT system (v2.1.2) has 1991 rules
• 1039 simple patterns
• 527 correlated patterns
• Up to 7 sub-patterns
• Set TCAM width as 128 bytes
• Patterns fit into a TCAM size of 295KB

Win- dow Size MIT Dump MIT Dump MIT Dump Berkeley Dump Berkeley Dump Berkeley Dump
Win- dow Size Avg Avg Max Max Avg Avg Max Max
20 0.5523 2.7683 8 0.4702 1.5765 12
40 0.9881 3.5376 14 0.6500 1.8661 18
60 1.3151 3.9960 14 0.7313 1.9652 23
80 1.5491 4.2158 16 0.7587 2.0373 24
100 1.6867 4.3485 18 0.7661 2.0740 25
120 1.7725 4.4475 18 0.7669 2.0768 25
140 1.8308 4.5722 19 0.7669 2.0768 25
160 1.8800 4.6643 19 0.7669 2.0768 25
180 1.9244 4.7386 19 0.7669 2.0768 25
200 1.9662 4.8079 20 0.7669 2.0768 25
17
Conclusions

• Fast speed pattern matching is essential for
  building effective defenses against virus
• Multiple pattern matching with TCAM
• Achieve multi-gigabit rate
• Search for thousands, or tens of thousands
  patterns in parallel
• Support long patterns, correlated patterns, and
  also patterns with negation, wildcards
• Can be extended to support higher rates with
  larger TCAMs

18

• Backup Slides

19
Long Patterns

• What if pattern is longer than the width of TCAM?
• Split it into multiple partial patterns
• For example, TCAM width k4

Pattern index Pattern content
1 ABCD ABCD
2 DEFG ABCD L
3 DEFG DEF
4 DEF
L ? ? ?
Short partial patterns, many TCAM hits
20
Statistical Analysis

• Example
• n 2000, mi 200 bytes, w 4 bytes. Associate
  hit rate is 2.2e-5, PHL size is 8.8e-5
• w 8 bytes, associate hit rate is 2.6e-15, PHL
  size is 2.08e-14
• Assume random input string, independent patterns
• Number of patterns n
• Pattern size mi bytes for pattern i
• TCAM width w
• Total entries for partial items in TCAM
• Associate hit rate is
• Ignoring the dependency between neighboring
  positions,
• PHL size is

21
Synthesized Worst-case Packets

• Four sets of synthesized data
• 1, 10, and 100 randomly inserted virus patterns
  per packet

22
Memory Lookup Process

• TCAM reports a miss
• No extra memory lookup
• Memory lookup process is idle
• TCAM reports a hit
• One memory lookup in the combined pattern table
• Lookups in matching table if PHL is not empty

23
Effects of Memory Ratio on Scan Rate

• Scan ratio
• Total scanning time (including memory lookups)
  vs. the time spent on TCAM lookups only.
• E.g., scan ratio2 ? total scanning rate TCAM
  access rate /2
• Memory ratio
• SRAM to TCAM access times