ECE 526

1
ECE 526 Network Processing Systems Design

• System Implementation Principles II
• Varghese Chapter 3

2
Outline

• Review Principle 1-7
• Implementation principles
• Reflect what we learned
• Example TCAM updating
• Cautionary Questions

3
Reviews

• P1 Avoid Obvious Waste
• Example copy packet pointer instead of packet
• P2 Shift Computation in Time
• precompute (table lookup),
• evaluate lazily (network forensics)
• Share Expenses (batch processing)
• P3 Relax Subsystem Requirements
• Trade certainty for time (random sampling)
• Trade accuracy for time (hashing, bloom filter)
• Shift computation in space (fast path/slow path)

4
Reviews

• P4 Leverage Off-System Components
• Examples Onboard Address Recognition
  Filtering, cache
• P5 Add Hardware to Improve Performance
• Use memory interleaving, pipelining (
  parallelism)
• Use Wide-word parallelism (save memory accesses)
• Combine SRAM, DRAM (low-order bits each counter
  in SRAM for a large number of counters)
• P6 Replace inefficient general routines with
  efficient specialized ones
• Examples NAT using forwarding and reversing
  tables
• P7 Avoid Unnecessary Generality
• Examples RISC, microengine

5
P8 Don't be tied to reference implementations

• Key Concept
• Implementations are sometimes given (e.g. by
  manufacturers) as a way to make the specification
  of an interface precise, or show how to use a
  device
• These do not necessarily show the right way to
  think about the problemthey are chosen for
  conceptual clarity!
• Examples
• Using parallel packet classification instead of
  sequential demultiplexing in TCP/IP protocols

6
P9 Pass hints across interfaces

• Key Concept if the caller knows something the
  callee will have to compute, pass it (or
  something that makes it easier to compute) as an
  argument!
• "hint" something that makes the recipient's
  life easier, but may not be correct
• "tip" hint that is guaranteed to be correct
• Caveat callee must either trust caller, or
  verify (probably should do both)
• Example
• Active message, the message carry the address of
  interrupt handler for fast dispatching

7
P10 Pass hints in protocol headers

• Key Concept If sender knows something receiver
  will have to compute, pass it in the header
• Example
• Tag switching, packet contains extra information
  beside the destination address for fast lookup

8
P11 Optimize the Expected Case

• Key Concept If 80 of the cases can be handled
  similarly, optimize for those cases
• P11a Use Caches
• A form of using state to improve performance
• Example
• TCP input "header prediction"
• If an incoming packet is in order and does what
  is expected, can process in small number of
  instructions

9
P12 Add or Exploit State to Gain Speed

• Key Concept Remember things to make it easier
  to compute them later
• P12a Compute incrementally
• Here the idea is to "accumulate" as you go,
  rather than computing all-at-once at the end
• Example
• Incremental computation of IP checksum

10
P13 Optimize Degrees of Freedom

• Key Concept be aware of variables under ones
  control and evaluation criteria used determine
  good performance
• Example memory-based string matching algorithm
• possible transitions from each state for a
  character is 256 (28, ASCII coding using 8
  bit)
• Bit-split algorithm using 8 machines, each
  machine only check for one bit, the total
  possible transitions for a character is 16 (21
  8)

11
P14 Use special techniques for finite universes
(e.g. small integers)

• Key Concept when the domain of a function is
  small, techniques like bucket sorting, bitmaps,
  etc. become feasible.
• Example
• bucket sorting for NAT table lookup
• NAT table is very sparse
• Each bucket is accessed by hashing
• Bucket sort
• Partitioning an array into a finite number of
  bucket
• Each bucket is sorted individually

12
P15 Use algorithmic techniques to create
efficient data structures

• Key Concept once P1-P14 have been applied, think
  about how to build an ingenious data structure
  that exploits what you know
• Examples
• IP forwarding lookups
• PATRICIA trees (data structure) were first
• A special trie, with each edge of patricia tree
  labled with sequences of characters.
• Then many other more-efficient approaches

13
TCAM

• Ternary 0, 1 and (wildcard)
• TCAM specified length of key and associated
  actions
• TCAM lookup compare the query with all keys in
  parallel, output (in one cycle) the lowest memory
  location whose key matches the input
• IP forward uses longest-prefix matching
• DIP 010001 matches both 010001 and 01
• Using TCAM for IP forwarding, requires put all
  longer prefixes occur before any shorter ones.

14
IP Lookup

• All prefixes with the same length are group
  together
• the shortest prefix 0 are in the highest memory
  address
• The packet with DIP 110001 matches prefix of
  both P3 and P5
• P5 is chosen due to longest-prefix matches

15
Routing Table Update

• 11 with P1 needed to insert to routing table
• Naïve create space in group of length-2 prefix,
  and pushing up one position all prefixes of
  length-2 and higher
• Core routing table have 100, 000 entries ? 100,
  000 memory accesses

16
Routing Table Update

• P13 understand the exploit degrees of freedom --
  we can add 11 at any position of group 2, not
  required after 10.
• We can add boundary of group 2 and group 3.

17
Clever Routing Table Updating

• the maximum memory accesses is 32 i.

18
Cautionary Questions

• Q1 Is improvement really needed?
• Q2 Is this really the bottleneck?
• Q3 What impact will change have on rest of
  system?
• Q4 Does BoE-analysis indicate significant
  improvement?
• Q5 Is it worth adding custom hardware?
• Q6 Can protocol change be avoided?
• Q7 Do prototypes confirm the initial promise?
• Q8 Will performance gains be lost if environment
  changes?

19
Summary

• P1-P5 System-oriented Principles
• These recognize/leverage the fact that a system
  is made up of components
• Basic idea move the problem to somebody elses
  subsystem
• P6-P10 Improve efficiency without destroying
  modularity
• Pushing the envelope of module specifications
• Basic engineering system should satisfy spec but
  not do more
• P11-P15 Local optimization techniques
• Speeding up a key routine
• Apply these after you have looked at the big
  picture

20
Reminder