Enterprise QoS Strawman May 1998

1
Enterprise QoS StrawmanMay 1998

• Terry Gray
• University of Washington

2
UW Network Overview

• 70,000 accounts
• 30,000 end systems
• 2,000 modems
• 50 remote sites
• IP-only backbone
• 350 Gigabytes/day across backbone
• Internet 30Mbps peak in, 15Mbps peak out
• NWNet founder, NOC
• Center of statewide K20 net
• Home of P/NW Gigapop, SNNAP

3
Strawman Overview

• Focus recurring costs and network reliability.
• Concern Making your most strategic asset
  dependent on previously unused services (KDC and
  DEN DBMS.)
• Hypothesis Per-flow authentication/authorization/
  reservation is a bad idea within an enterprise.
• Goals Simple class-based-queuing on campus, plus
  flexibility to use multiple strategies for WAN
  QoS.
• Strategy Per-port subscription for premium svc,
  plus CBQ based on application type and CAR
  quotas, plus separate backbone for GE research
  connections.

4
QoS Axioms

• QoS doesn't create bandwidth --it just determines
  who will get poor service at congestion points.
• Dynamic bandwidth sharing is about demand shaping
  and statistical muxing. If you want certainty,
  dont share.
• The most important QoS question is how many
  "busy" signals constitute success for your
  network?
• Given a busy signal, users will want to proceed
  anyway.
• Network Managers will not trust end systems.
• Best-effort traffic must be protected from
  hi-priority hogs.

5
The Challenge

• Devise a minimum-complexity scheme to
• shape peak demand to fit instantaneous capacity,
  and
• generate revenue to keep capacity ahead of
  aggregate growth.
• Demand drivers
• User desire, application need, usage
• - Cost, availability, disappointment
• Issue
• cost of controls vs. cost of resources vs. value
  of certainty.
• Problem
• How to keep users/apps happy, while keeping
  users/apps from consuming too much capacity?

6
Toolkit for Demand Shaping (Congestion Avoidance)

• Traffic authorization (premium access demand
  control)
• Eligibility Control
• Admission Control
• Traffic modification (instantenous demand
  control)
• Traffic shaping
• Traffic policing
• Traffic adaptation (feedback-based demand
  control)
• Protocol Adaptation
• Application adaptation
• User adaptation (behavior shaping)

7
Simplified Network Topology
Core Switch
Gigapop
4
Fed Nets
Border Router
Internet2
Core
Router
Router
Core
40
Internet
Interior Switch
Interior Switch
250
PBX
Edge Switch
Edge Switch
1000
Branch Site
30,000
Desktop
Desktop
8
Congestion Avoidance

• Eligibility control
• Global knowledge of application need (port )
• User/application desire (TOS/DS bits)
• User/port privilege --quota/CAR (802.1p bits)
• Admission control
• Busy signal at call setup time
• Needed iff eligibility control inadequate
• Class-Based-Queuing for CoS
• Using need or desire or privilege
• Constrained in backbone by CAR quota
• Enterprise congestion zones
• Subnet --overprovision CBQ
• Backbone --subscriptions/quotas (CAR)
• Wide-Area --quotas, feedback, reservation?

9
Why CBQ is Interesting...Even with 50
hi-priority traffic, delay is constant
Low-Priority
Delay
Hi-Priority
80
100
Load
Multiplexing priorities on a channel improves
efficiency at the cost of certainty.
10
Campus Connections

• Expected Typical Service Levels
• Bronze Shared 10 half-duplex
• Silver Switched 10 full-duplex
• Gold Switched 10/100 full-duplex
• Platinum Switched 10/100 full-duplex CAR
• Platinum Subscription Levels
• Quota-based (Committed Access Rates)
• Multiple CAR levels (and fees) possible
• Separate GE backbone research connections

11
Design Decisions 1

• Why Physical Port Subscriptions?
• Simple, static configuration
• MAC, IP addresses risk of gaming via gateways
• User ID not always right payer -e.g. classrooms
• Why Subscriptions not Usage?
• Fairness problem
• Predictable bills
• Predictable revenue
• Sustainable basis for adding capacity
• Use Application Type when known?
• IPSec??
• Port-agile apps
• Same app, different user need

12
Design Decisions 2

• Why not Advance Reservations?
• Event duration needed in order to schedule
• Big-chunk reservations require sequestered
  bandwidth
• Small-chunk reservations unnecessary
• Apps may need problematic bi-directional
  reservations
• Reservations invite policy complexity
• Even on wide-area, expect subscriptions rather
  than reservations to dominate, except for large
  chunk cases.
• Whither RSVP?
• Campus net RSVP-transparent zone
• End-systems could signal border router/BB if
  needed
• (Or other end-systems)

13
Wide-Area Connections

• Commercial Internet w/Premium option
• Internet 2
• Branch sites w/integrated services
• Federal Research Mission Nets
• Regional/State Networks

14
Borders Commercial Internet

• Wont just be best-effort for long
• Reservation model unlikely
• Quota/CAR approach seems probable
• Pricing model unclear
• Need for recharge likely

15
Borders Internet 2 Connection

• Dual use
• Application research
• Production traffic among members
• If lots of big-chunk reservations needed
• Sequester part of I2 bw for scheduled use
• Remainder production on-demand premium
• Or
• Use quotas for medium-chunk needs
• Manual reconfiguration for special events
• Could permanently sequester bw for some apps

16
Borders Branch Sites

• Could provide dedicated bandwidth for different
  services IP data, IP video, VOIP.
• Border routers connect to POTS and
  videoconferencing gateways.
• Bounded round-robin queue discipline.
• Packets need to be marked by service type, or
  classified by source (gateway) address.

17
Upstream Traffic Handling

• End-System/Application
• Traffic shaping if possible (enforced by policing
  in router)
• Optionally setting TOS/DS bits
• Rate adaptation
• Edge/Closet Switch
• Tag all frames using 802.1p priority bits
  according to the port's premium subscription
  level.
• Inspect TOS bits if not set, then set using
  application type (if that can be determined from
  TCP/UDP port number).
• Option to have app type over-ride original TOS
  setting.
• Use resulting TOS bits for edge switch queuing
  decision.
• Interior/Building Switch
• Use TOS and/or 802.1p bits for queuing decision.
• No CAR enforcement

18
Core Traffic Handling

• First Core Router
• Inbound queue using 802.1p and TOS bits police
  CAR at level corresponding to 802.1p bits
• Outbound set 802.1p frame priority based on
  queuing decision above. Note now the 802.1p
  bits no longer denote premium subscription level
  they reflect subscription level plus TOS bits,
  but CAR level has already been enforced, so
  that's OK.
• Core Switch
• Respect 802.1p bits

19
Outbound Traffic Handling

• Interior Edge Switches (Downstream)
• Respect 802.1p priority bits set by router
• Ignore TOS bits
• Border Router Egress
• Set TOS based on incoming 802.1p and TOS bits
• Optionally modify TOS based on prior RSVP
  negotiation or other bandwidth broker policy.
• Record usage data

20
Open Issues

• What to do with over-quota packets?
• Drop rather than downgrade? (In some schemes
  downgraded packets may arrive out-of-order and be
  dropped by end-system streaming apps anyway.)
• Incoming Traffic
• May cause biggest part of NSP charges
• Respect incoming high-priority marking?
• If so, apply destination quota or recharge?
• CAR for platinum service on campus
• Is it really needed?
• Would it match WAN CAR needs?
• Will users know if they got what they paid for?

21
Reality Check

• Current campus nets are not ready for QoS
  Prepare for forklift upgrades.
• Widespread 802.1p support expected.
• Switching, full-duplex needed but full-duplex
  auto-configuration is a problem.
• Cat 3 wiring is an issue in older buildings.
• How much layer-3 support needed at edge?
• Access from alternate locations implies Layer-2
  authentication.
• Router configuration determines if subnet is
  trusted (will 802.1p bits be believed?)

22
Conclusions

• Future peak/aggregate usage patterns are unknown
  No one can say how much BW and QoS capability
  will really be needed.
• Nevertheless, adequate eQoS without per-flow
  lookups or reservations appears plausible...
• Fast/Gigabit Ethernet infrastructure can reduce
  odds of congestion multiple queues and CAR
  policing provide additional headroom.
• But even a minimalist QoS approach has worrisome
  operational implications vigilance is needed to
  keep things as simple as possible.