Internet Routers II

1
Internet Routers II Using the Pigeon Hole
Principle to Model Routers
Stochastic Networks Seminar March 1st 2002
Sundar Iyer High Performance Networking
Group, Department of Computer Science, Stanford
University sundaes_at_stanford.edu http//www.stanfo
rd.edu/sundaes
2
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets.
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

3
What is an Ideal Router?
R
R
1
1
NR
R
R
N
N
Total Bandwidth N2R
Departing Packets
Arriving Packets
Interconnect
Memory
Output Queued Switch

• Output Queued (OQ) routers are ideal but not
  practical
• It minimizes the delay faced by a packet
• The bandwidth to each output is NR, the total
  bandwidth is N2R
• The cost and power consumption is prohibitive

4
CIOQ Models
Arbiter
R
R
1
1
R
R
R
R
R
R
N
N
Total Bandwidth NR
N memories
Departing Packets
Arriving Packets
Input Queued Switch

• CIOQ switches offer some advantages over OQ
  switches, but are still not practical
• They can give the same delay guarantees as OQ
  switches
• They need a switching bandwidth of only 2NR
• They have high computational complexity
• The model does not capture many different
  architectures

5
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets.
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

6
The Single Buffered Router Model
R
R
1
1
R
R
N
N
Departing Packets
Arriving Packets
Interconnect
Interconnect
Memory

• Single Buffered Routers buffer packets only once
• The interconnects may be
• physically separate or merged
• one of the interconnects may be a simple pass
  through
• The memory can be
• centralized or distributed
• one or many
• statically or dynamically allotted amongst all
  ports

7
Why a New Model for Routers?

• SB Routers comprise a broader class of routers
• They replace the CIOQ model
• They also include other interesting router
  architectures such as the
• Parallel packet switch, Parallel shared memory
  router, Distributed shared memory router, etc.
• With this model we can compare these routers to
  an ideal router and answer
• Does a router give me the same delay guarantees
  as an output queued switch?
• Can a router give me 100 throughput

8
Parallel Packet Switch
Slow Speed Output Queued Routers
OQ
1
NR/k
NR/k
OQ
2
rate, NR
rate, NR
Departing Packets
Arriving Packets
OQ
NR/k
NR/k
k
9
Parallel Shared Memory Router
Slow Speed Parallel Memories
1
2
k
2NR
rate, NR
rate, NR
NR
NR
Memory Manager
Departing Packets
Arriving Packets
10
Distributed Shared Memory Router
Distributed Line Cards
Arbiter
Arbiter
rate, R
rate, R
1
1
S1R
S2R
rate, R
rate, R
2
2
BW S1NR
BW S2NR
N memories
Departing Packets
Arriving Packets
11
How to Compare Routers?
Output Queued Router
OQ Switch
R
R
OQ
1
1
1
Yes? Emulate
2
2
R
R
N
N
?
Single Buffered Router
Any SB Switch
R
R
1
1
No
R
R
N
N
12
Using the Pigeonhole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

13
An Abstract Model for a Router A Modified
Pigeonhole Principle

• Consider the following model for a switch
• There are P pigeonholes, that can contain an
  infinite number of pigeons
• Assume that time is slotted, and in any one time
  slot T,
• at most N pigeons can arrive and at most N can
  depart
• at most one pigeon can enter or leave a specific
  pigeonhole
• When a pigeon arrives, we know the exact time
  slot at which it will depart
• For any switch
• We need to determine the minimum P, such that all
  N pigeons can be immediately placed in a
  pigeonhole when they arrive, and can depart at
  the right time

14
Solving the Abstract Model

• When a pigeon arrives in a time slot
• No more than N 1 other pigeons arrive at that
  timeslot
• No more than N other pigeons depart at that
  timeslot
• No more than N - 1 other pigeons depart at the
  same time as this pigeon
• By the Pigeonhole Principle,
• P (N) (N-1) (N-1) 1 3N 1
  pigeonholes are sufficient

15
The Constraint Set Technique

• Upon arrival of a packet, find a memory which is
  free now and which is free when it will depart

16
What prevents us from finding such free memories?

• Physical Constraints, which are limitations
  imposed by the hardware
• Memory (E.g. Parallel Shared Memory Router)
  Cant access a memory more than a certain number
  of times in a time period
• Bus (E.g. Parallel Packet Switch) Cant use the
  same bus simultaneously for more than a certain
  number of packets
• Crossbar (E.g. Distributed Shared Memory Router)
  Each port-input and port-output may be busy only
  once in a timeslot

17
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

18
An Example Parallel Shared Memory (PSM) Router
2NR
1
1
2
2
N
N
Arbiter
19
Can a PSM Router Emulate a FIFO OQ Router?

• Let a cell arrive at input i at time t and be
  destined to depart from output port j at time
  DT
• Such a cell must not be written to memories
  which
• Are used to write the other N-1 arriving cells at
  t. (Write Constraint Set)
• Are used to read the N departing cells at t.
  (Read Constraint Set)
• Will be used to read the N-1 departing cells at
  DT. (Future Read Constraint Set)
• There are three constraint sets
• By the pigeonhole principle, 3N memories at rate
  R, or a memory bandwidth of 3NR is sufficient

20
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

21
Distributed Shared Memory Router
Arbiter
Arbiter
1
1
N
N
22
Can a DSM Router Emulate a FIFO OQ Router?

• Let a cell arrive at input i at time t and be
  destined to depart from output port j at time
  DT
• The cell can be written to any intermediate port
  x such that
• The edge (i,x) is available at time t. Since, no
  more than N-1 other cells contend to write at
  time t, no more than floor(N-1)/s1 vertices
  are unavailable. (Write Constraint Set)
• The edge (x,j) is available at time DT. Since, no
  more than N-1 other cells contend to leave at
  time DT, no more than floor(N-1)/s2 vertices
  are unavailable. (Read Constraint Set)
• There are two constraint sets
• By the pigeonhole principle, if suffices that
  floor(N-1)/s1 floor(N-1)/s2 lt N.
• Hence if s1 s2 2, i.e. ss1s24 is enough.
• A bandwidth of 4NR is sufficient

23
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

24
An Abstraction of Delay Guarantees1 Push In
First Out (PIFO) Queue 4 FIFO queues on a
single output

• Requirement
• Multiple FIFO Queues need to be maintained
• An arriving packet joins one of the FIFO queues
• A server serves these queues with different
  weights

A1 4
A2 2
B1 3
C2 1
C1 1
C3 2
D2 2
D1 1
25
What is the Problem with PIFO?

• There are two problems
• Problem with PIFO
• The constraint set technique depends on being
  able to predict the departure time and schedule
  it. The departure time of a cell is not fixed in
  PIFO
• Problem with non PIFO order
• When the memory is shared amongst all outputs,
  the departure order for the router as a whole is
  not even PIFO, even though each output queue is a
  PIFO queue.
• Lets see what causes the latter problem

26
What causes the non-PIFO Order?N4 output ports
shown in different colors, each of which is a
PIFO queue.

• Relative order has changed. It is not PIFO!

27
Steps in PIFO SchedulingRe-ordering the Output
pattern

• Can prevent conflicts by taking care of past and
  future N cells

28
Can a PSM Router Emulate a PIFO OQ Router?

• A cell which arrives at input i at time t,
  destined to depart output port j at time DT ,
  must not be written to memories which
• Are used to write the other N-1 arriving cells at
  t. (Write Constraint Set)
• Are used to read the N departing cells at t.
  (Read Constraint Set)
• Will be used to read the N-1 departing cells of
  that output before it (Future Read
  Constraint Set)
• Will be used to read the N-1 departing cells of
  that output after it (Future Read Constraint
  Set)
• There are four constraint sets
• By the pigeonhole principle, 4N memories at rate
  R, or a memory bandwidth of 4NR is sufficient

29
Using the Pigeon Hole Principle to Model Routers

• Background
• The Single Buffered Model for Routers
• An Abstraction of a Switch Using Pigeons
• Technique Constraint Sets
• Analysis of First In First Out Switches
• Parallel Shared Memory Router
• Distributed Shared Memory Router
• Analysis of Delay Guarantees in Switches
• Parallel Shared Memory Switch
• Summary of Results

30
Summary
Emul-ate?
Arbiter
Total BW
BW of Mem.
Num. Mem.
Type
Yes
None
N(N1)R
(N1)R
N
OQ
Simple
2NR
2NR
1
Shared Memory
Yes
No
Moderate
2NR
2R
N
IQ
Yes
Complex
6NR
3R
2N
CIOQ
PSM
Yes
Simple
4NR
4NR/k
k
Yes
Simple
6NR
6R
N
DSM
Yes
Simple
6NR
6R/k
Nk
PPS