Making the Grade: Ensuring Application Performance in an Education Network

1

• Making the Grade Ensuring Application
  Performance in an Education Network
• Presented By
• Sean Applegate
• Mid-Atlantic Systems Engineer

2
The Bottleneck
Problem Traffic on the high-speed LAN hits the
lower-speed WAN access link and congestion
(queuing/dropped packets) occur.
3
Introductory Questions

• How many people here own a PacketShaper?
• How many people here are evaluating a
  PacketShaper?
• How many people have NEVER heard of Packeteer or
  the PacketShaper?
• How many people dont know what your top 10
  applications are and the percent of bandwidth
  they are using?
• How many people are considering increasing WAN
  bandwidth speeds?
• How many people are using other bandwidth mgnt or
  policing technologies to control traffic?

4
Applications Drive Todays Educational
Institutions

• Mission-critical applications are critical to
  education
• All traffic is not created equal

eMail
File Transfers
TCP / IP Application-Neutral
Mission-Critical
Web Surfing
Streaming Music, Quake, etc.
Peer-to-Peer
-
-

Time-Sensitive
5
What am I spending my on?
Are you spending 60-85 of your WAN budget on P2P
applications?
6
Agenda

• Who is Packeteer?
• What is PacketShaper?
• Who is using PacketShaper?
• Implementing Packeteer's Four Step Process
• - Classify
• - Analyze
• - Control
• - Report
• Summary
• Questions

7
Packeteer Fact Sheet

• Founded in 1996, Pioneer of Proactive Bandwidth
  Management
• Headquarters in Cupertino, CA
• US Offices New Jersey, Chicago, Atlanta, Dallas,
  Seattle, Washington D.C., San Diego
• Offices Abroad Netherlands, Hong Kong, Japan,
  and Australia
• Regional Resellers
• Employees 197
• Customer proven
• Shipping since February 1997
• Thousands of PacketShapers shipped worldwide
• 5th generation of software

8
Packeteer Product Family

PacketShaper
PolicyCenter
Internet Bandwidth Management Solutions
Central Configuration Management
9
What is PacketShaper?

• Industry Leading QoS Solution
• CMP Network Telecom Network Infrastructure
  Product of the Year - 2001
• Firmware, Real-time OS (PSOS)
• Classifies 350 Apps at OSI Layers 2-7
• Uses Patented TCP Rate Control to proactively
  control application traffic and prevent queuing
  and reduce latency.
• Over 55 measurement variables for detailed
  analysis
• Managed through an onboard web interface and CLI,
  no external hardware/software required
• Becomes a piece of wire if it fails

10
PacketShaper Product Line
PacketShaper 1500 Series Up to 2 Mbps WAN
capacity
PacketShaper 2500 Series Up to 10
Mbps WAN capacity
PacketShaper 4500 SeriesUp to 45 Mbps WAN
capacity
PacketShaper 6500 SeriesUp to 100 Mbps WAN
capacity
11
Typical k-12 School District Topology
PacketShapers at each shool to manage
school-to-district office and school-to-school
traffic
Internet
PacketShaper to manage the Internet link
12
380 Higher Education Customers in US
Higher Ed Customers in the US
Ed Customers in Ohio

• Stanford Univ
• California Tech
• Yale Univ
• Vanderbilt Univ
• Univ of Miami
• Texas AM
• Clemson Univ
• Univ of Notre Dame
• All Universities of California
• Case Western Reserve Univ
• Ohio Northern Univ
• Dartmouth College
• Howard Univ

• Univ of Dayton
• Miami Univ
• Case Western Reserver Univ
• Cleveland State Univ
• Xavier Univ
• Youngstown State
• Denison Univ
• Bowling Green Univ
• Capital Univ
• Ohio has more higher ed users than any other
  state in the US.
• A Sites are catching up fast!

13
A Complete Solution
Our Example Customer Randolph Macon College

• Problems
• Congested WAN Link
• Poor Visibility at App Layer
• Poor Response Times
• Needed to get control
• Needed better WAN ROI

14
Step 1 Classify - Whats Running on My Network?
Automatically Classify 350 Apps at OSI Layers 2-7
Application
PacketShaper
7
Presentation
6

• Peer-to-Peer Apps
• Aimster
• AudioGalaxy
• CuteMX
• DirectConnect
• Gnutella
• Hotline
• iMesh
• KaZaA/Morpheus
• Napster
• ScourExchange
• Tripnosis.

• Some Other Apps
• H.323
• RTP-I/RTCP-I
• PASV FTP
• HTTP
• Real
• WinMedia
• Shoutcast
• MPEG
• Quicktime
• RTSP
• Chatting Apps
• Games

Session
5
Transport
4
Most Routers Switches
Network
3
2
Data Link
1
Physical
If you cant classify it you cant shape it!
15
Step 1 Classify Traffic Class Criteria

• Inbound/Outbound (travel direction)
• Protocol family
• Service (very diverse, see online list)
• Inside/Outside (location of relevant server)
• Port(s)
• Service Proxy
• IP Address, MAC Address, host name, or host list
• Subnet Mask
• URL (including wildcards)
• Further details (criterion) for Citrix-ICA,
  Oracle-netv2, HTTP 1.1, RTP-I
• Diffserv, IP Precedence, COS/TOS

16
Step 1 Classify Traffics INs and OUTs
17
Step 1 Classify Manual Class Creation
1. From the MANAGE screen, select the parent
class from the traffic tree
2. Click on CLASS, then ADD
3. Define traffics criteria (details on the next
slide).
18
Step 1 Classify Matching Rules

• Classes are made up of matching rules
• Classes can have many matching rules
• multiple matching rules are ORd together
• 1 rule for each of 3 servers.
• single matching rules are ANDd together
• 1 rule that catches traffic from a specific
  server to a specific client.

19
Step 1 Classify More on Matching Rules

• The definition of the traffic in a class is a
  matching rule
• Its a collection of values for the criteria we
  listed
• Traffic Discovery defines matching rules for the
  classes it creates
• You define matching rules for the classes you
  create

20
Step 1 Classify Adv. Matching Rules

• In addition to the basic criteria, such as IP
  address and port numbers, the following advanced
  options are available
• Host Lists
• Details for Citrix, Oracle, HTTP 1.1 and RTP
• Diffserv and IP COS/TOS

21
Step 1 Classify Host Lists

• Instead of a single IP address or a range of IP
  addresses, specify a list of hosts.

Lets you take advantage of LDAP directory
services.
22
Step 1 Classify Application Criteria

• Citrix-ICA, Oracle-netv2, HTTP 1.1, and RTP-I can
  be further classified using the Matching Rule
  Criterion field
• Citrix-ICA by published application, client name
  or priority level
• Oracle-netv2 by database name
• HTTP 1.1 by DNS name or IP address
• RTP-I (real-time protocol for media streaming) by
  Encoding Name, Media Type ("a" for audio, "v" for
  video), or Clock Rate (8000, 16000, 44100, 90000)

23
Step 1 Classify Diffserv, COS/TOS

• Diffserv Code Point (DSCP) (6-bit field)
• Value of 0-63
• COS - Class of Service (3-bit field)
• IP precedence value 0-7
• TOS - Type of Service (4-bit field)
• 802.1q/ISL VLANs
• MPLS

24
Step 1 Classify - Other Settings
Type, Traffic Discovery(within class), Top
Talkers/Top Listeners, RTM, Comments
25
Step 1 Classify Traversing the Class Tree

• PacketShaper examines all passing traffic.
• - Every flow must be assigned to a class.
• It traverses the tree to find the traffics
  correct class.
• Traversal starts at the top
• If you have a special-case class you want
  searched first, make it an Exception class.
• Example All PCs in a subnet to be treated the
  same except one. E.g. DifferentPC
• SubnetA
• SubnetB

26
Step 1 Classify RMC After 24 Hrs
This traffic tree was automatically built by
turning on Traffic Discovery. Only shows
applications on the network.

• 55 Applications AutoDiscovered
• 6 Peer-to-Peer (circled)
• 7 Streaming
• 3 Chat
• 5 Games
• And the usual Internet and network service
  protocols

27
Step 2 Analyze

• Click Report in the PolicyConsole navigation bar.
  
• Youll see 3 graphs for the Inbound link and 3
  graphs for the Outbound link
• Link Utilization
• Network Efficiency
• Top 10 Classes
• Shows whats competing for the bandwidth.

28
Step 2 Analyze - Top Ten Tab
29
Step 2 Analyze - Context-Specific Reports
30
Step 2 Analyze - Monitor Tab
RTM Summary
Class Hits/Rates tell you how busy a service is
Top Talkers/Top Listeners tell you which IP
addresses are using the most bandwidth
31
Step 2 Analyze - Top Talkers / Listeners

• Enable up to 12 top talkers/listeners (total).
• Create classes for top users.

32
Step 2 Analyze - Response-Time Summary

• View delay statistics for all measured classes

33
Step 2 Analyze - Transaction Delay
Is my network causing problems? Or is it one of
my servers?
34
Step 2 Analyze Delay Distribution
Who is the most common culprit? The Server or The
Network?
Approx. 90 of transactions at the
server experience 0 sec delay
Approx. 75 of transactions experience .1 sec
delay on the networkHeres our culprit!!
35
Step 2 Analyze Response Times

• Why measure response time?
• Quantify performance.
• Identify performance problems.
• Develop strategies for bandwidth management,
  server balancing, and topology upgrades.
• Assess results after youve made configuration
  changes.

36
Step 2 Analyze - Measuring Delay

• Server Delay - of ms the server uses to process
  a clients request after all data received.
• Total Delay - of ms from clients request to
  receipt of response.
• Network Delay Total Delay - Server Delay
• Round-Trip Time (RTT) is the of ms for
  client-server exchange of precisely one packet.

37
Step 2 Analyze - RMC Link Performance

• Inbound Link avg peak bps

- Link is fully congested, observe how close the
avg peak bps are. - What are my top 10 types
of traffic?- Am I using this for recreational or
business use?
30 of all TCP data is retransmitted Approx 1/3
of the WAN budget is wasted (700/month).
of TCP Retransmitted Bytes ?
38
Step 2 Analyze RMC Top Applications
Over 72 P2P
FTP
KaZaA
Gnutella
iMesh
HTTP
KaZaA
iMesh
HTTP
Gnutella
39
Step 2 Analyze RMC HTTP/SSL Response Times

• Users are waiting 2 secs for each HTTP
  connection to complete

Users are waiting ? 3 to 8 secs for each SLL
connection to complete
40
Step 3 Control How Do I Control Performance?

• Set policies to control performance
• Per-flow minimum/maximum bandwidth policies
• Per-user minimum/maximum bandwidth policies
• Priority-based policies
• Admissions Control
• Partitions for control of aggregate flows
• PacketShaper implements TCP Rate Control
• Control when and how much data end-systems
  transmit
• Using industry-standard TCP/IP
• Manage traffic flows and aggregate classes with
  bits-per-second accuracy

41
Step 3 Control Applying Policies
42
Step 3 Control Priority Policies
A Priority policy has only one parameter
Low
High
0 1 2 3 4
5 6 7
43
Step 3 Control Priority Policy Guidelines

• Use a priority policy
• When rate is not your primary objective
• If traffic does not burst (surge)
• If traffic is latency-sensitive
• If high-priority flows are small, orif
  low-priority flows are large but not bursty
• Priority policies are appropriate for interactive
  traffic like TN3270 or Telnet (latency-sensitive,
  dont burst, small)

44
Step 3 Control Rate Policy Page
45
Step 3 Control Rate Policy Guidelines

• Guarantee each flow a minimum bits-per-second
  rate
• Give each flow prioritized access to excess
  bandwidth
• Keep a lid on surging, bandwidth-hungry flows
• Guard mission-critical flows
• Give delay-sensitive flows a chance
• Make sure behind-the-scenes TCP Rate Control is
  active
• Remember not to over-commit guaranteed rates!

46
Step 3 Control Never-Admit Policies

• Use a Never-Admit policy
• For TCP or Web traffic, to block a session and
  inform the user

47
Step 3 Control Discard Policies

• When you simply want to toss all packets for a
  traffic class.
• Block a service
• Provide security
• Recommended for blocking non-TCP classes because
  theyre not session-oriented

48
Step 3 Control Ignore Policies

• Ignore policies
• Treat traffic as pass-through
• Exempt a traffic class from bandwidth management
• PacketShaper does not count the statistics

49
Step 3 Control How flows Compete for Excess
Rate policies are satisfied first! Then, at
each priority level, rate policies are given
their burstable chunks and priority polices get
what they want.
50
Step 3 Control How Flows Compete For Demand
51
Step 3 Control - TCP Rate Control
Natural TCP
TCP Rate Control

• Steps
• Measure end-to-end latency
• Forecast when packets will be needed to meet the
  policy
• Tell the Client/Server how much data to send (set
  TCP Window Size)
• Tell the Client/Server when to send the data
  (schedule ACKs)
• PROACTIVE CONTROL!!
• Speed up latency-sensitive flows
• Throttle back big file transfers
• Smooth traffic throughput
• Improve multiplexing, reduce jitter

Sender
Receiver
Receiver
Sender
Time
Time
Bursty Traffic Flow
Smooth Traffic Flow
52
Step 3 Control Multiplexing Gains
Unmanaged Traffic
Gravel
Managed Traffic
Sand
53
Step 3 Control Queuing versus Rate Control
54
Step 3 Control UDP Delay Bound

• Deadline scheduling mechanism
• Provides rate control for UDP
• Not as good as TCP rate control
• Uses a delay bound to
• Set the maximum delay
• Limit buffer utilization per flow
• Allows setting the delay bound from 200 to 10,000
  milliseconds

55
Step 3 Control Partitions

• A partition
• Creates a virtual pipe within a link for an
  aggregate traffic class
• Provides a minimum and maximum bandwidth
  guarantee
• Ranges from 0 Kbps to 45 Mbps
• Enables efficient bandwidth use

Does Not Waste Bandwidth!
56
Step 3 Control Partitions Two Purposes

• Partitions can
• Limit restrain a traffic class to keep it from
  becoming predatory
• Protect shelter a traffic class bandwidth from
  predators

57
Step 3 Control Partitions Can Burst

• You can
• Create a static partition
• Create a partition that can grow (burst) if extra
  bandwidth is available
• Partitions can burst to use
• The entire link
• A predetermined maximum amount of bandwidth

58
Step 3 Control Dynamic Partitions

• Automatically setup and tear down partitions
  based on active users.
• Limit each user to a maximum amount of b/w at all
  times.
• Set a cap on number of active users assigned a
  partition.
• Create an overflow partition for everyone else
• Dynamic Partition usage graph

59
Step 3 Control Creating a Partition
60
Step 3 Control Dynamic Sub-Partitions
Select details to specify sizing and traffic flow
61
Step 3 Control Dynamic Sub-Partition Details
62
Step 3 Control Time of Day Scheduling

• CLI only
• Syntax schedule file
• Use schedule show to see scheduled items.
• Use schedule delete to remove scheduled
  items.
• Schedule commands are stored in RAM so they do
  not span resets.
• To span resets create a file named startup.cmd in
  9.256/
• Put schedule commands in startup.cmd to change
  shaping by time of day.
• When PS boots up it reads startup.cmd and
  schedules commands.
• To immediately apply a new schedule command
  delete old scheduled times and enter run
  startup.cmd to initialize the new commands.

63
Step 3 Control Organizing the Traffic Tree

• Logically organized the classes
• Used low priority rate policies and partitions to
  throttle back aggressive non-latency-sensitive
  file downloads such as P2P traffic, FTP and SMTP
• Used high priority rate policies to improve
  performance of longer lived time-sensitive
  traffic, such as HTTP, SSL, Citrix, RTP-I, etc.
• Used priority policies for short lived flows,
  such as Telnet, RTCP, H.323, tn3270, rsh, rlogin,
  etc.
• Final Config Used
• 10 Partitions
• 35 Policies

64
Step 3 Controlling VoIP and Video Traffic

• Classify and control H.323's at OSI Layers 5-7
• Q.931 (call setup)
• H.245 (call control)
• Gatekeeper Discovery
• Gatekeeper Control (Registration, Admission, and
  Status)
• RTCP-I
• RTP-I
• RTCP-B
• RTP-B
• Classify and control RTP-I traffic by at OSI
  Layers 5-7 by
• Audio or Video
• Codec provide exact amount of BW required
  per-flow with a Rate policy
• Encoding type (GSM or JPEG)
• Prevent other traffic, such as casual web
  browsing, P2P and large file transfers from
  impacting VoIP performance by proactively
  throttling back inbound and outbound bandwidth.

65
Step 4 Report - How Do I Measure Performance and
Plan for the Future?

• PacketShaper lets you make more intelligent
  decisions
• Evaluate the effectiveness of shaping
• See what traffic you are spending your WAN Budget
  on
• Plan for the future of your network through
  capacity planning, trend analysis, etc
• Track application service level agreements based
  on total delay, server delay and network delay
• Set and meet user expectations
• Import data into other reporting systems
• CSV, SNMP, XML
• Complex plugins for HP Openview, Concord eHealth,
  InfoVista, NetCool and other NMS
• Can notify via email or SNMP trap when
  performance is poor or when there is a possible
  DoS attack

66
Step 4 Report Establish Acceptable Performance

• Set a threshold to define good service.

67
Step 4 Report Whats Good, Whats Bad?

• Thresholds let you easily quantify good/bad
  service.

Definition of good responses
Definition of SLA
68
Step 4 Report Monitoring SLAs
SLA Problems are gone!
SLA Problems
69
Step 4 Report RTM Transaction Delay
User-set threshold
70
Step 4 Report Worst Clients/Servers
Tells you which clients/servers have the most
delay
71
Step 4 Report Statistics Data Dump

• Extract lists of variable values for any class.
  Two months of data stored.
• Specify
• One or more variables (definite variety)
• Time period
• Sort order
• Individual statistics or sum totals

72
Step 4 Report How to Get the Data
73
Step 4 Report PacketShaper Events

• PacketShaper Events notify you when thresholds
  are exceeded.
• Currently command line only
• Viewable via the Events Monitor
• Several steps to set it up.

74
Step 4 Report Event set up.

• 4 steps to Event Notification
• Identify the mail server PacketShaper will use to
  send messages.
• Identify the recipients of the email
• Identify the SNMP Server PacketShaper will send
  traps to.
• Register the event.

75
Step 4 Report Setting up email notification

76
Step 4 Report Setting the Recipients List

77
Step 4 Report Setting SNMP Server
78
Step 4 Report Defining Events
Events come in 2 flavors User-Defined Any
measurement engine variable Pre-Defined 17
PacketShaper Pre-defined Events
79
Step 4 Report Defining Events

80
Step 4 Report Registering Events

81
Step 4 Report Event Summaries

82
Step 4 Report Event Monitor
83
Step 4 Report SNMP MIBs

• Standard MIBS
• MIB II
• 10 Basic Groups (system, interfaces, at, ip,
  icmp,tcp,udp,egp,transmission,snmp)
• Private MIBS
• Packeteer MIB
• Packeteer RTM MIB

Download from support.packeteer.com
84
Step 4 Report - Roll Your Own Reports

• Useful APIs
• PolicyConsole HTTP/Javascript
• XML
• PacketWise Server-side Tags
• CGI API

85
Report- Custom Reports via SNMP Authentication

• Step 1 Determine report type
• Step 2 Get an example URL from the WUI
• Step 3 Replace the respective variables with
  your new variables
• Step 4 Turn on snmp look authentication
• CLI sys set dataRetrievalUseSMMPPassword 1
• Append SNMPPASSWORD to end of URL
• Step 5 Put new URL in a web page and the graph
  will be created

86
Step 4 Report RMC Link Performance
Inbound Throughput
Inbound Efficiency
No Shaping
Shaping
No Shaping
Shaping
Outbound Throughput
Outbound Efficiency
Restricted P2P to 300Kbps
Improved Efficiency, better WAN ROI
No Shaping
Shaping
No Shaping
Shaping
87
Step 4 Report RMC Top 10
Inbound Before Shaping 71 P2P
HTTP
HTTP100 Increase
Inbound After Shaping 34 P2P
88
Step 4 Report RMC Main Apps

• Rate shaped P2P back and capped at 1.5Mbps with
  a partition
• Rate shaped HTTP/SLL so they would perform faster

Rate shaped P2P back and ? capped with at
300kbps Rate shaped HTTP/SSL ?so they would
perform better
89
Step 4 Report RMC HTTP Response Times
Outside Web Server Normalized Network Response
Times
Inside Web Server Normalized Network Response
Times
No Shaping
Shaping
No Shaping
Shaping
90
Packeteers PacketShaper

• Provides the application infrastructure that
  enables you to
• Know whats on your network
• Get visibility into and control over bandwidth
  usage
• Control recreational traffic
• Reserve bandwidth for teaching, learning, and
  research
• Make intelligent decisions about capacity
  planning
• And much more

91
Tools

• http//support.packeteer.com
• PolicyConsole API (ask support for it)
• Boilerplate Reporting Portal
• Stanford PacketShaper email list
• Send email to majordomo_at_lists.stanford.edu
• Msg body (no subj) subscribe packeteer-edu
• Archive http//www.stanford.edu/group/networking/
  netlists
• Initial Shaping Script
• Tons of Perl Scripts
• Online White Papers
• PacketGuide (v5.2)
• FREE Online Training every Friday
• Regional Training Classes

92
Questions Contact Info

• Questions?
• Sean Applegate, Packeteer Mid-Atlantic SE
• [phone]
• sapplegate_at_packeteer.com
• Resellers
• Stratacache 937-224-0485
• Vector 513-786-6618
• DPS 513-489-4200
• DDS 216-676-1760