Measuring Performance Chapter 12 CSE807

1
Measuring PerformanceChapter 12 CSE807
2
Performance Measurement

• To assist in guaranteeing Service Level
  Agreements
• For capacity planning
• For troubleshooting performance problems
• For Performance modeling

3
Step I

• What are the information sources of performance
  data
• What are the monitoring tools available for
  measuring response time and resource usage
• What techniques are used to calculate model input
  parameters

4
Sources of Performance Data

• Monitors
• Provide raw data
• Transformation and analysis
• Product specifications
• Manufacturers
• Benchmarks

5
Performance Measurement Framework

• Application oriented workload
• Workload classes
• Should be tied to SLAs
• Responsiveness
• Availability
• Reliability
• Throughput

6
Capacity Planning Frameworkper application

• Measure and chart as a proactive analysis as a
  factor of time
• Bandwidth usage
• CPU and Memory utilization
• End-to-end Response Time for frequently used
  transactions
• Number of concurrent users

7
Troubleshooting Framework

• Monitor collect data for problem detection
• Paging
• I/O hot spots
• Network collisions, error rates
• CPU usage
• Queue depths

8
Troubleshooting Framework SystemAccounting Data
Example

• sar -u
• HP-UX FIADGMS1 B.11.00 B 9000/804 02/26/01
  
• 000000 usr sys wio idle
  
• 010000 37 8 19 35
• 020000 22 5 34 39
• 030000 9 2 16 73
• 040000 7 2 14 77
• 050000 18 4 27 51
• 060000 6 2 14 78
• 070000 2 2 5 91
• 080000 1 1 4 94
• 082000 36 2 2 60
• 084000 51 1 1 46
• 090000 43 2 9 46
• 092001 35 4 15 45
• Average 33 4 13 5088

9
Performance Tuning Framework

• Operating System Level
• Establish kernel maximum and minimum values -
  resource allocation per process
• Application and Database Level
• Oracle SGA
• Priority values for classes of workload
• Application code tuning

10
Performance Management of Network-based
Environment

• Clients
• Application Response Monitor (ARM)
• Workstation Performance Monitors
• Networks
• Sniffers, Network Monitoring software
• Openview, Tivoli, CiscoWorks
• Servers, Proxies
• OS monitoring tools

11
Measuring C/S Performance
Client
Server
LAN
LAN
Network
12
Measurement Techniques

• Specify Reference Points
• e.g. where to start the measurements from
• Specify Measurements
• e.g. Packets, collisions, error rates statistics
• Instrument and Collect Data
• Install configure measurement tools
• Analyze and Transform Data
• Accounting data, web logs are too detailed
• Transform into useful information

13
A Representation of Measurement Process
Application Programs -----------------------------
-- Server Hardware ------------------------------
Operating System ------------------------------ H
ardware
Instrument collect data
Analyze transform data
Specify Reference Point
Specify Measurements
14
Type of Measurements

• Event Mode
• State of the system defined by a set of variables
• e.g. I/O request, Completion of a transaction
• Tools
• Special code inserted at specific point of OS
• Records generated with date, time, event and
  other requested information
• High overhead

15
Type of Measurements(cont.)

• Sampling Mode
• At pre-defined time instants
• Sampled at timer interrupts based on hardware
  clock
• Overhead depends upon
• of variables
• Size of sampling interval
• large sampling interval will reduce statistical
  significance of variable

16
Data Collection Tools

• Hardware Monitors
• Collects data at pre-defined signals
• Electronic probes to sense the state of hardware
• Registers, memory locations, I/O channels
• Do not create overhead
• Do not collect any software related information
• Portable

17
Data Collection Tools(cont.)

• Software Monitors
• Event or interrupt driven
• Can record information regarding software and
  operating system
• Use resources
• Easy to install and use
• System Level - Global CPU, I/O, memory, queue
  etc.
• Program Level - Application oriented, program
  identification, elapsed time, memory usage

18
Software Monitors

• Accounting Systems
• Application level granularity
• Most popular
• Program Analyzers
• Used for measuring software related data
• TP monitoring
• Transaction counts, CPU/transaction

19
Software Monitors(cont.)

• Logs
• NT Performance Monitor log can be used to graph
  the usage statistics
• HP-UX Measureware log data can be extracted for
  historical trend analysis
• Web Servers logs (Example 12.1)
• Access Logs
• Referrer Logs
• Agent Logs
• Error Logs

20
Performance Model Parameters

• Queues or Devices
• Workload Classes
• Workload Intensity
• Service Demands

21
Performance Model Parameters

• Queues
• Determine what is the point of view
• Determine the relevant queues
• Queues in a System
• Jobs waiting for components
• CPU, Memory, I/O

22
Performance Model Parameters

• Workload Classes
• Logical grouping of workload
• OPEN vs. CLOSED
• Open useful for forecasting
• Realistic for web servers modeling

23
Performance Model Parameters

• Workload Intensity - define the amount of work
  system needs to perform
• Arrival Rate - of cars arriving at the toll
  plaza per minute
• Number of customers - of customers in bank per
  hour
• Pages requested per transaction
• Average packet size

24
Performance Model Parameters

• Service Demand
• For hardware resources
• Service time for satisfying a request in the
  queue
• Does not include waiting time
• Non-linear scaling for additional CPU caused by
  OS overhead to manage CPU allocation
• Example 12.2

25
Performance Model Parameters

• Parametric Estimation - combination of
  information provided by program analyzers
• Identify the type of execution environment
• Specify the measurement process
• Monitor the system and collect the data
• Estimate input parameters
• Example 12.3

26
Collecting Performance Data

• Networks
• SNMP - Simple Network Management Protocol
• Manager and Agent
• Management Information Database (MIB)
• e.g. Openview, Tivoli
• LAN analyzers - packets/sec, collision rate
• RMON - Remote Monitoring MIB

27
Collecting Performance Data

• Servers
• NT - Performance Monitor
• Counter by object types - CPU, Memory, disk etc.
• Averaging Counters
• Difference Counter - last and previous
• Chart, report, alert and logs

28
Collecting Performance Data

• UNIX
• SAR - System Activity Report, a sampling tool
• ps
• vmstat
• iostat
• netstat
• logfiles

29
Summary

• Guidelines for performance measurement
• Types of monitors
• Measurement techniques
• Specify reference point
• Specify measurements
• Instrument and collect data
• Analyze and transform