HighPerformance Fair Bandwidth Allocation for Resilient Packet Rings

1
High-Performance Fair Bandwidth Allocation for
Resilient Packet Rings

• V. Gambiroza, Y. Liu, P. Yuan, and Ed Knightly
• ECE/CS Departments
• Rice University
• http//www.ece.rice.edu/networks

2
The Network Edge (Myth)

• Common, but unrealistic view
• Hosts, network of switches (or routers), edge
  Internet router

3
The Network Edge (Reality)

• Ring metro backbone
• Rings are the dominant configuration for their
  fault tolerant properties
• Size 100 nodes, 10 km

4
Why is the Metro Edge Important?

• Why is end-to-end performance poor when
• Core networks are over provisioned
• Campus Ethernets are rarely a bottleneck (100
  Mbps Ethernet, soon GigE)
• Even the wireless last hop is now up to 54 Mbps
  (IEEE 802.11a)
• Answer the metro backbone is increasingly the
  bottleneck on the network path

5
Metro Ring Technologies (1/3) Token Ring/FDDI
(Obsolete)

• One node transmits at a time and each packet
  circulates the entire ring
• No spatial re-use and low utilization

6
Metro Ring Technologies (2/3) SONET

• Circuits between pairs of nodes. Problems
• Cannot re-use unused capacity (no statistical
  multiplexing)
• Cannot burst to full link rate
• Coarse bandwidth granularity (155 Mbps)
• Up to N2 circuits (and high port count)
• ? Low utilization

7
Metro Ring Technologies (3/3) Gigabit Ethernet
(GigE) Rings

• Unfair
• Closest node to hub gets the most bandwidth
• Extent of unfairness depends on protocol
  (TCP/UDP), topology (RTT and number of nodes) and
  traffic inputs

8
Metro Holy Grail

• Simultaneously achieve
• High Utilization (stat muxing, burst to link
  rate)
• Spatial Reuse (many simultaneous flows)
• Fairness (minimum bandwidth share per ingress)

9
Remaining Outline

• RIAS Reference Model
• Resilient Packet Ring Standard
• Limitations of proposed standard
• DVSR Protocol
• Simulation Experiments

10
RIAS A Reference Model for Packet RingsAn
idealized objective for algorithm design and
analysis
Given all instantaneous demand rates (fluid
offered traffic) and corresponding ingress-egress
nodes

• ? Determine instantaneous rate-limiter values
  such that
• All flows obtain a fair bandwidth share
• Spatial reuse (and utilization) is maximized
• There is no queuing or loss on the ring
• Even under idealized settings the reference model
  is not obvious
• How to define a flow? How to fairly employ
  spatial reuse?
• Answer RIAS

11
Illustration of RIAS Fair (1/2)

• Parking Lot
• 4 flows each receive rate ¼
• Need to throttle flows at ingress point to
  ring-wide fair rate
• GPS provides local fairness cannot achieve RIAS

12
Illustration of RIAS Fair (2/2)

• Parallel Parking Lot
• Each flow receives rate ¼ on downstream link
• Left 1-hop flow fully reclaims excess bandwidth
  (RIAS)

http//www.ece.rice.edu/networks/RIAS/
13
What Have We Achieved with RIAS?

• For any scenario of input traffic rates, can
  determine the targeted bandwidth allocations
  (rate limiter values)
• Clear target for algorithm design
• Quantify tradeoffs (simpler hardware design vs.
  deviation from reference model)
• Algorithm performance is evaluated on
• Ability to converge to RIAS-fair rates
• Time to converge to RIAS-fair rates

14
Remaining Outline

• RIAS Reference Model
• Resilient Packet Ring Standard
• Limitations of proposed standard
• DVSR Protocol
• Simulation Experiments

15
Resilient Packet Ring (IEEE 802.17)

• Upcoming standard for metro rings
• Goal resiliency of SONET fair/efficient/spatial
  reuse
• Participants Cisco, Nortel, Luminous, Lantern,
• Node architecture
• Rate limiters, transit/station scheduler,
  counters,

16
RPR Protocol

• Transit traffic has priority over ingress
  station traffic
• Each node measures my_rate of ingress traffic
• If a node is congested
• send my_rate upstream
• upstream nodes throttle to my_rate

17
The Problem RPR

• my_rate is NOT the ring-wide fair rate
• Example of permanent oscillation and throughput
  degradation in RPR

18
Remaining Outline

• RIAS Reference Model
• Resilient Packet Ring Standard
• Limitations of proposed standard
• DVSR Protocol
• Simulation Experiments

19
Distributed Virtual-time Scheduling in packet
Rings (DVSR)

• Sketch of key idea
• Suppose each node performed weighted fair
  queueing at the ingress-aggregate granularity
• Call these rates local IA-fair
• Would have local, but not ring-wide fairness
• However, if we throttle nodes at their ingress
  point to the minimum local IA-fair rate and
  iterate (adapt) we achieve RIAS fair rates

20
Challenge

• How to know the local IA-fair rates without
  actually
• doing fair queueing and measuring them?
• (which would make for a complex transit path)

Solution

• Compute a proxy of virtual time using per-ingress
  byte counts
• Computing exact virtual time is well known to be
  complex
• We use a simple bound using arrival counts
• Communicate virtual-time rate upstream
• Ingress nodes compute minimum rate on the path
  and converge to RIAS fair rates

21
DVSR and the RIAS Reference Model

• DVSR targets to dynamically realize RIAS rates
• Three sources of deviation from RIAS rates due to
  distributed nature of the problem
• Remote control
• Temporally aggregated and delayed information
• Multiple resources

22
Remote Fair Queuing Single Resource Illustration

• Control of upstream rate controllers via
  downstream virtual time progression
• True fair queueing replaced with rate controllers
  multiplexer
• Note no packets queued in mux when D 0

23
Delayed and Temporally Aggregated Control
Information

• Periodically summarize evolution of v(t)
• Evolution of v(t)
• Exact value cannot be computed
• Worst-case increase occurs if all packets arrive
  together
• Compute using byte counts
• Two cases
• Continuously backlogged communicate time average
  
• Multiplexer idle some part of T
• Advertise excess capacity
• v(t) is not reset to 0

24
DVSR Properties

• Complexity
• Ordering rates is O(klogk) (k is at most N/2).
  Performed every control update time (.1 msec
  minimum)
• Developing approximations to avoid
• Rate limit computation
• Sub-allocation of per-link IA fair rates to
  source-destination pairs
• Feedback signal
• Rotating message or RPR-compatible
• Fairness
• Derived a worst-case bound on unfairness

25
Remaining Outline

• RIAS Reference Model
• Resilient Packet Ring Standard
• Limitations of proposed standard
• DVSR Protocol
• Simulation Experiments

26
Simulation Experiments

• Algorithms
• OPNET modules RPR and GigE
• All default settings
• ns-2 implementation of DVSR
• Scenario
• 622 Mbps link capacity
• 200 kByte buffer size
• 1 kByte packet size
• 1 msec ring propagation delay
• 0.1 msec inter-message time

27
Scenarios for Performance Evaluation

• A scenario consists of
• Set of flows (source-destination pairs)
• Demand rates/behavior of flows (TCP/UDP, VBR/CBR,
  )
• Each scenario has at least one performance issue
• Can RIAS fair rates be achieved in steady state?
• If not algorithm has a throughput loss
• Algorithm dynamics
• Convergence time, oscillations, range and
  time-scale of oscillation, throughput loss due to
  oscillation

28
Spatial Reuse in the Parallel Parking Lot
CBR UDP flows sending at the link capacity

• DVSR is within ?1 of RIAS fair rates
• GigE favors downstream flows cannot achieve
  spatial reuse
• RPR achieves only if using multi-choke option

29
Convergence Time in the Parking Lot
Time (s)
Time (s)
DVSR
RPR

• CBR UDP flows with rate 0.4 (248.8Mbps)
• Flow(1,5), (2,5), (3,5), (4,5) begin transmission
  at times 0.0, 0.1, 0.2, and 0.3 seconds
  respectively
• Convergence time 0.2 msec for DVSR, 50 msec for
  RPR
• Richer feedback signal allows faster convergence

30
Conclusions

• Metro edge is a critical part of end-to-end path
• RPR can permanently oscillate in a wide range
• Throughput loss is approximately 15 and is
  dependent on delays and filter settings
• Root cause is unbalanced fair rates (vs. input
  rates)
• DVSR as a high-performance, implementable
  approximation to RIAS fairness
• Significant convergence time and throughput
  improvement of DVSR vs. RPR

31
High-Performance Fair Bandwidth Allocation for
Resilient Packet Rings

• V. Gambiroza, Y. Liu, P. Yuan, and Ed Knightly
• ECE/CS Departments
• Rice University
• http//www.ece.rice.edu/networks