ClassBench: A Packet Classification Benchmark

1
ClassBench A Packet Classification Benchmark

• David E. Taylor Jonathan S. Turner
• Department of Computer Science Engineering
• Applied Research Laboratory
• det3,jst_at_arl.wustl.edu
• IEEE Infocom 2005

2
Packet classification
Packet Filter Set
3
Motivation for a benchmark

• No benchmark currently exists in industry or
  research community
• Performance of the most effective packet
  classification solutions depends on the
  composition of filter sets
• TCAM capacity depends on port range
  specifications
• Range conversion to prefixes may cause a single
  filter to occupy 2(w-1)k TCAM slots (900 slots
  in the worst case for two 16-bit port numbers)
• w number of bits required to represent a point
  in the range
• k number of fields specified by ranges
• On average, each filter requires 7 TCAM slots
• Fastest algorithms leverage filter set structure
• Decision tree algorithms (Hi-Cuts, Hyper-Cuts)
• Tuple-Space algorithms
• Decomposition algorithms (RFC, P2C, DCFL)

4
Motivation for a benchmark (2)

• Security and confidentiality concerns limit
  access to real filter sets for study and
  performance evaluation
• Well-connected researchers have gained access but
  are unable to share
• Performance evaluation with real filter sets is
  limited to size and composition of sample sets
• Current filter sets contain hundreds to a few
  thousand filters
• Goal develop a benchmark capable of capturing
  relevant characteristics of real filter sets
  while providing structured mechanisms for scaling
  the number of filters, augmenting the
  composition, and analyzing performance effects
• Provide value to three distinct communities
  researchers, product vendors, product consumers

5
Related work

• IETF Benchmarking Working Group (BMWG) developed
  benchmark methodologies for Forwarding
  Information Base (FIB) routers and firewalls
• FIB focuses on performance evaluation of routers
  at transmission interfaces
• Firewall methodology is a high-level testing
  methodology with no detailed recommendations of
  filter composition
• Network Processing Forum has a benchmarking
  initiative
• Produced IP lookup and switch fabric benchmarks
• Thus far, only IBM and Intel have published
  results for IP lookup
• No details regarding packet classification
  benchmarking efforts
• Performance evaluation by researchers
• Most randomly select prefixes from forwarding
  tables and use existing protocol, port range
  combinations
• Baboescu Varghese added refinements for
  controlling the number of zero-length prefixes
  and prefix nesting
• Woo Infocom 2000 provided strong motivation for
  a benchmark
• Provided a qualitative overview of filter
  composition for various environments
• ISP Peering Router, ISP Core Router, Enterprise
  Edge Router, etc.

6
ClassBench architecture
Filter Set Generator
Filter Set Analyzer
size
smoothing
scope
Synthetic Filter Set
Seed Filter Set
Trace Generator
Input Header Trace
scale
locality
Set of Benchmark Parameter Files

• Filter Set Analyzer extracts relevant statistics
  and probability distributions, generates
  parameter file
• Guided by high-level models developed from filter
  set analyses
• Parameter files provide complete anonymity of
  addresses
• Filter Set Generator produces a synthetic filter
  set retaining characteristics specified by input
  parameter file
• Input parameters provide high-level control over
  filter set size and composition
• Trace Generator creates a sequence of packet
  headers to exercise the input filter set
• Input parameters provide high-level control over
  trace size and locality of reference

7
Modeling filter set structure

• Protocol distribution
• Distribution of filters over protocols and
  associated flags
• Port pair class matrix
• Captures structure of source/destination port
  ranges
• Example wildcard on source port, exact match
  (80) on destination port
• Joint distribution of address prefix lengths
• Captures structure of source/destination address
  prefixes
• Example exact match on SA and wildcard on DA
• Address space coverage
• Address prefix correlation and nesting
• Store filter set model in a parameter file
• Goal construct a set of representative parameter
  files

8
Port pair classes

• Common port range specifications can be divided
  into five classes
• WC wildcard, 065535
• LO all system ports, 01023
• HI user ports, 102465535
• AR arbitrary port range
• EM exact match port
• Number of unique arbitrary ranges and exact port
  numbers appearing in databases is limited
• For the AR and EM classes, maintain a
  distribution of filters over unique values
• Combination of source/destination port ranges
  define a port pair class
• 25 port pair classes formed from all permutations
  of port class
• WC/WC, WC/R1, WC/R2, etc.

9
Port pair class matrix

• Distribution of filters over the port pair
  classes for TCP and UDP
• For each bin in the PPC matrix, create a joint
  distribution of prefix pair lengths
• Characterizes relationship between port pair
  class and prefix pair lengths
• Example ,1001,065535,80,TCP,syn

Destination Port
Destination Port
Source Port
Source Port
10
Prefix length joint distributions

• Analyses of real filter sets show unique prefix
  pair length structure
• Combinations including /0 and /32 prefixes are
  common
• Random selection from IP forwarding tables would
  result in combinations dominated by class C
  (24-bit) prefixes
• Strong motivation for capturing joint distribution

11
Representing joint distributions

• Represent joint distribution by a combination of
  total prefix length and source prefix length
  distributions
• Total length distribution 064 sum of SA and
  DA prefix lengths
• For each non-empty bin in total length
  distribution, build a source length distribution
  for the prefix pairs in the bin
• (destination address prefix length) (total
  length) (source address prefix length)
• Total lengthdistribution providesa
  convenientmechanism foraltering
  filterspecificity
• Support for high-levelscope adjustment inFilter
  Set Generator

12
Branching skew definitions

• Capture the address space coverage of the subnets
  defined by address prefixes
• Build two binary trees from the source and
  destination address prefixes in the filters
• At each node, define the weight of the left child
  and right child as the number of filters
  specifying a prefix stored in the left and right
  subtrees, respectively
• Let heavy maxweight of left child, weight of
  right child
• Let light minweight of left child, weight of
  right child
• For each level of the tree 031
• Compute the average number of childrenfor nodes
  with at least one child
• Provides a measurement of the branchingat each
  level
• Compute the average skew for nodeswith two
  children

13
Generating synthetic filter sets

• Use input parameter file along with scale, scope,
  and smoothing parameters to guide filter set
  generation
• Input parameters provide high-level control over
  filter set size and composition
• Abstracts user from detailed model of filter set
  structure contained in parameter files
• Scale parameter sets a target size for the
  synthetic filter set
• Resulting filter set may contain fewer filters
  due to the removal of redundant filters
• Scope parameter biases the tool to generate more
  or less specific filters
• Smoothing parameter provides a structured
  mechanism for introducing new prefix lengths
• Pseudocode provided in papers
• Source code available at http//www.arl.wustl.edu/
  det3

14
Scope definition

• From a geometric perspective, a filter defines a
  hyper-rectangle in d-dimensional space
• Volume of the hyper-rectangle is the product of
  the 1-d lengths specified by the filter fields
• e.g. Number of addresses covered by source
  address prefix
• Scope is a logarithmic measure of d-dimensional
  volume
• Points in d-dimensional space correspond to
  packet headers
• Filter properties are commonly defined as a tuple
  specification, or a vector with fields
• t0, source address prefix length, 032
• t1, destination address prefix length, 032
• t2, source port range width, 0216
• t2, destination port range width, 0216
• t4, protocol specification, Boolean specified,
  not specified

15
Scope adjustment

• Utilize a bias function on the random number used
  to select values from the distributions
• Bias function computes area under line whose
  slope is defined by s
• Precludes recomputation of each distribution
• Scope (s) bias to more or less specific
  filters, -11
• Applied to random variable used to select total
  prefix length and port pair class
• s gt 0 increase scope, decrease specificity
  (prefix length)
• s lt 0 decrease scope, increase specificity
  (prefix length)

S -1
S 0
S 1
16
Sensitivity to scope adjustment

• For example filter sets, scope adjustment
  provided a maximum bias equivalent to 6-bits
  longer or shorter in total prefix length
• Provides for an 64x increase or decrease in the
  average coverage of the filters in the database
• Sensitivity is dependent upon original
  distributions in parameter files

17
Smoothing adjustment

• As filter sets scale in size, we anticipate that
  new address prefix pair lengths will emerge due
  to subnet aggregation and segregation.
• Using scope as a measure of distance, subnet
  aggregation and segregation results in new prefix
  lengths that are near to the original prefix
  length.
• Smoothing provides a structured mechanism for
  introducing new prefix pair lengths
• Smoothing parameter, r, limits the maximum number
  of bits of deviation from the original prefix
  pair
• Manhattan distance in the prefix pair joint
  distribution
• Model the clustering using a symmetric binomial
  distribution
• Given r, a symmetric binomial distribution is
  defined on the range 0 2r
• The probability at each point i in the range is
  given by
• Apply binomial distribution to each non-emptybin
  in the prefix pair length distribution
• Scale each spike in the distribution according
  to the median probability, pr
• Binomially distribute the residue to the prefix
  pair lengths within the r-bit radius
• For bins at edge of distribution, we truncate
  binomial distribution and normalize

18
Single Spike with r 8
(a.) r 8
(b.) r 8, top-view
19
Cumulative smoothing effects

• Bins accumulate residue from bins within an r-bit
  radius
• Example with r 4

20
Trace generator

• Trace Generator produces a sequence of packet
  headers to exercise a given filter set
• Analyze average performance, power consumption,
  caching load-balancing schemes
• Provide high-level control over size of trace and
  locality of reference
• Exhaustive test is intractable
• Selecting a point for every d-dimensional
  polyhedron formed by the overlap of N filters
  requires O((n log n)d) time
• Choose a random filter from the filter set, then
  select a random corner of the filters
• Points in d-dimensional space represent packet
  headers
• Corners often have a higher probability of
  filter overlaps
• Insert m copies of the header into the trace
• m is chosen from a Pareto distribution
• Pareto shape and scale parameters are input
  parameters
• Scale input parameter sets threshold on number of
  headers in the trace
• Threshold scale N

21
ClassBench usage

• Our own research
• Packet Classification using Extended TCAMs, Ed
  Spitznagel, David Taylor, Jonathan Turner, IEEE
  International Conference on Network Protocols
  (ICNP), 2003.
• Scalable Packet Classification using Distributed
  Crossproducting of Field Labels, David Taylor,
  Jonathan Turner, IEEE Infocom, 2005.
• On using content addressable memory for packet
  classification, David Taylor, Ed Spitznagel,
  technical report WUCSE-2005-9.
• Other universities (that we know of)
• Tsinghua University, Beijing, China
• University of Texas, Arlington, USA
• UC Berkeley, USA
• University of Patras, Greece
• A Silicon Valley startup (in stealth mode)

22
A packet classification bake-off

• Currently, five student groups are implementing
  packet classification algorithms on an Intel IXP
  network processor
• Recursive Flow Classification Gupta, McKeown
  1999
• Pruned Tuple Space Search Srinivasan, et. al.
  1999
• HyperCuts Singh, et. al. 2003
• Extension of HiCuts Gupta, McKeown 1999
• Decision tree where internal nodes define
  branching conditions on multiple fields and leaf
  nodes contain lists of filters of bounded size
• Parallel Packet Classification van Luteren,
  Engbersen 2003
• Independent prefix match engines return ternary
  strings
• Search for concatenation of strings in TCAM
• Distributed Crossproducting of Field Labels
  Taylor, Turner 2005
• Using ClassBench tools for standardized
  performance evaluation
• Quantify performance using same filter sets and
  implementation platform
• Analyze scaling properties for filter sets of
  various sizes and compositions

23
Thank you.Questions?

• Full technical report, tools, and parameter files
  from real filter sets are available at
• http//www.arl.wustl.edu/det3