Abilene Premium Service Deployment Experience

1
Abilene Premium Service Deployment Experience

• Internet2 Member Meeting - Atlanta, GA

Stanislav Shalunov ltshalunov_at_internet2.edugt,Ben
Teitelbaum ltben_at_internet2.edugtOctober 31, 2000
2
Overview

• APS motivation and background
• Initial engineering plan
• Limitations and modifications to the plan
• Future directions

3
Abilene Premium Service (APS)

• Goal
• Make APS a reference implementation of the QBone
  architecture
• Why?
• Catalyst Enable wide-area QoS experimentation
  and propagate Abilenes experiences to larger
  community
• Safety Belt Congestion is only one success
  catastrophe away!
• The Team
• UCAID
• Indiana University
• I-TECs (NCNI, OARnet)

• NLANR/NCNE

• Cisco
• Qwest
• Nortel

4
APS Participation

• Participation Criteria
• Abilene connector status necessary but not
  sufficient
• QBone participation
• Capability of access router / Abilene edge card
• Current Participants
• MAGPI (U. Penn)
• iCAIR
• PSC (Penn State)
• OARNet (Ohio State)
• Others in the wings
• NASA Goddard
• Various international

• ANL
• UIUC
• DOE Science GRID (peering transit network)

5
Initial Engineering Plan (obsolete)
(Surveyor SNMP HTTP)
(Firehose CAR)
(Whiteboard CLI)
(MDRR)
Automated Setup
(GTS)
6
APS Phase 1 Sweetwater

• First, crawl
• Ingress policing
• Measurement feedback
• Limited technical support
• No priority queuing yet, but still get benefit of
  Abilenes overprovisioning!
• Goals
• Begin to establish operational practices for
  responding to admissions requests
• Provide participants with the experience of
  sitting behind a policer

7
APS Measurements

• Goal implement QBone measurement architecture
  (collection dissemination) for a single QBone
  domain
• Hoped-for Side Effects
• Tools to ease implementation for others
• Adjustments to QMA based on implementation
  experiences

8
APS Measurement Collection

• Status
• Currently collecting SNMP stats of EF/BE loads
  and CAR profiling
• Abilene Surveyors OC-3 connected and operational
• Near Future
• IPDV along edge-to-edge QBone paths
• Abilene Surveyor timing improvements
• Better external stratum 1 NTP servers
• Improved NTP on Surveyors
• CDMA timing signal (under consideration)
• Collection of AS-level traffic matrices

9
APS Measurement Dissemination

• Ohio I-TEC hosting APS measurement server
• Status
• EF/BE loads and CAR stats served through web
  server (raw data and plots available)
• Implementation of WHOIS server to learn canonical
  names (QBone domains, routers, active and passive
  measurement devices)
• Future more from Paul Schopis in a few
  minutes...

10
Committed Access Rate (CAR)

• Classifies traffic based on certain matching
  criteria (interface, DSCP, or ACL) and meters it
  to a leaky bucket traffic profile
• Depending on metering result, different actions
  applied (drop, transmit, set DSCP,)
• Syntax
• rate-limit input output access-group
  rate-limit acl-index bps burst-normal
  burst-max conform-action action exceed-action
  action no rate-limit input output
  access-group rate-limit acl-index bps
  burst-normal burst-max conform-action action
  exceed-action action

11
CAR Experience

• For the most part, CAR is exactly what the
  DiffServ doctor ordered
• However, there are some limitations
• Performance
• Token bucket depth
• Classification of DiffServ aggregates

12
CAR Limitation 1 Performance

• On E0 edge cards, no ASIC support for CAR
• ? there is a performance hit

pps _at_ 256 bytes
Preliminary E0 CAR performance figures under
retest by ITEC-NC
13
Addressing CAR Performance

• Out of our hands
• The Plan
• Load on access interfaces is still light, so
  performance not really an issue for now
• Wait for E3 edge cards, which will have CAR in
  hardware

14
CAR Limitation 2 Policing Granularity

• Cisco CAR Doc
• burst-normal Normal burst size in bytes. The
  minimum value is bps divided by 2000
• burst-normal ? max (mtu, bps/2000)
• But, Premium service demands token bucket depth
  of 1 MTU
• ? bps ? 3Mbps

15
Continuous State Leaky Bucket Algorithm

• rate ? ltconfigured valuegt
• burst ? ltconfigured valuegt
• deficit ? 0
• last_time ? time()
• while true do
• packet ? next_packet_in_incoming_queue()
• current_time ? time()
• size ? size(packet)
• new_deficit ? deficit - rate(current_time -
  last_time)
• if new_deficit lt 0 then
• new_deficit ? 0
• eligible_for_drop ? false
• else
• eligible_for_drop ? true
• fi
• if eligible_for_drop and new_deficit size gt
  burst then
• exceed_action(packet)
• else
• deficit ? new_deficit size

16
Addressing CAR Policing Granularity

• Again, out of our hands
• The Plan Have raised issue with Cisco and are
  hoping E3 cards will address the limitation

17
CAR Limitation 3 Classification

• CAR can classify by
• DSCP, ACL
• Current SLA
• Profile applied to all injected EF traffic up to
  agreed-upon peak-rate
• firehose classification
• But, we need inputIF, outputIF, DSCP
  classification

18
Virtual Trunk vs.Firehose Classification

• The good news CAR can also classify by
• qos-group (Cisco proprietary concept)
• With packets assigned to QoS groups through QoS
  Policy Propagation via BGP (QPPB)

19
Addressing CAR Classification

• The Plan use QPPB and encourage broader Cisco
  support for the feature
• QoS Policy Propagation via BGP (QPPB)
• BGP used to propagate QoS policies to all border
  routers
• Each BGP community associated with a qos-group
• qos-group field added to forwarding table (FIB)
• Using CAR, packets may then be classified at
  ingress according to their egress point from the
  cloud (next AS)

20
Abilene Architecture Limitation Porous Edge
Problem

• QoS theft problem with current architecture
• MDRR (EF forwarding) on interior interfaces
  easily subverted by unpoliced connectors

21
How to Crisp the Edge?

• EF traffic injected by all Abilene connectors
  must be policed
• Some possible solutions
• DetectionMight detect illegal EF traffic with
  NetFlow and/or passive monitoring gigaPoP would
  be asked to police
• Prevention with performance hitCAR load
  monitoring for E0 connectors
• Prevention without performance hitRe-write DSCP
  with PIRC for E2 connectors (Dunn hack)

22
Looking Forward Resource Accounting and
Admissions

• Problem how to account for link EF capacity and
  commitment in routed network?
• Potential solution DiffServ-Aware MPLS-TE
• Basic idea
• Edge-to-edge MPLS tunnels
• LSP setup constrained by available EF bandwidths
• OSPF augmented to carry QoS link state
  attributes
• see draft-lefaucheur-diff-te-reqts-00.txt
• Solves DiffServ admission control problem
• Does not solve general DiffServ provisioning
  problem
• Status evaluating Cisco implementation of this
  idea (DBP-TE) in EFT

23
Current Engineering Plan
(Surveyor SNMP HTTP WHOIS traffic
matrices)
(CAR QPPB PIRC filtering?)
(Whiteboard CLI)
(MDRR)
(GTS)
24
Building a Stronger User Base

• Very weak demand for Sweetwater
• CAR alone is not enough to motivate
  campus/gigaPoP deployments
• How to increase demand
• Increase BE utilization
• Build user communities for jitter-sensitive apps
• Increase sex appeal by actually treating APS
  traffic better than best effort
• Deploy APS across congested peerings
  (considerable international interest)

25
For more information...

• QBone Home
• http//qbone.internet2.edu/
• Abilene Premium Service Home
• http//www.internet2.edu/abilene/qos/

26
(No Transcript)