Web%20Performance,%20Load,%20and%20Stress%20Tests

1
Web Performance, Load, and Stress Tests

• Mainframe and Open System Network
• Natural Conference Boston
• Dieter W. Storr
• Los Angeles Times
• May 2004
• info_at_storrconsulting.com

2
Performance, Load, and Stress TestsContents

• What are performance, load, and stress tests?
• Why and when should you test?
• What response time do you expect?
• What parts play a role in the network?
• Network basics
• Logical and physical transactions

3
Performance, Load, and Stress TestsContents

• What tools can you use to test these parts?
• When should you start performance tests?
• Web site performance, analysis and tuning
• Checklist
• Links

4
What are Performance Tests?

• Test each part of the system, such as web server,
  web applications, databases, etc.
• Tests can run usually without requesting page
  images in order to concentrate the testing on the
  script and code itself

?
?
?
?
5
What are Load Tests?

• Load tests are performed by testing the website
  using the best estimate of the traffic your
  website needs to support.
• Define the maximum time it should take for a page
  to load -- from a usability and customer
  experience standpoint, not from a technical one.

?
?
?
?
6
What are Stress Tests?

• To simulate brute force attacks that apply
  excessive load to system, such as web server, web
  applications, databases, etc.
• To estimate the maximum load that your system can
  support
• To learn the traffic thresholds of your system
• How it will respond after exceeding its threshold

?
?
?
?
7
What are Stress Tests?

• Ramp test
• A variation of stress test
• The number of users is increased over the life of
  the test

?
?
?
?
8
Why Performance Tests?

• Determine that all functions do what they meant
  to do (function test)
• Ensure that tasks and objectives are easily to be
  accomplished (usability test)
• Guarantee a result from the website in an
  acceptable time (performance test)

9
Why Performance Tests?

• Optimize each part of the system for increased
  web traffic
• Ensure that the end-users mouse click is not met
  with silence

10
Why Load Tests?

• To compare the maximum time it should take for a
  page to load with the effective used time.
• Calculate the impact of exceeding that maximum
  time. Will you lose sales or prospective
  customers?
• Calculate the anticipated load and load pattern
  for the website.
• Determine to do some work on the application or
  webserver.

11
Why Stress Tests?

• Determine the maximum load
• Test excessive system load
• Determine the traffic thresholds of your system
• Experience the reaction of the system after
  exceeding its threshold
• Ramp tests can help to determine what maximum
  load a server can handle while providing optimal
  access to web resources

12
When Performance Tests?

• You cannot start early enough!
• Start before a single line of code is written
• Network
• Load balancer
• Application
• Database
• Web servers
• During development, all software engineers and QA
  team should have access to performance test tools

13
When Load Tests?

• As soon as web pages are working
• QA team should conduct the first load tests
• Performance testing should be part of the regular
  testing routine each day for each build of the
  software

14
What Response Time Do You Expect?

• Does your company set limits?
• KEEP in mind
• Many websites today have a serious business
  mission gt to take money
• Reason Advertisement opportunities, selling
  retail products, etc.
• Websites and applications need to be up and
  running at all times

15
What Response Time Do You Expect?
Source Jakob Nielsen, Designing Web Usability,
New Riders Publishing, 2000
16
What Response Time Do You Expect?

• 95 of all web requests should be processed in
  less than 10 seconds!

BE AWARE If performance slows even a little,
fickle web users are likely to jump quickly to a
competitors site
17
What Parts Play a Role in The Network?
Basic Connections
Web Browser
Web Server
CGI
NATURAL
ADABAS
18
What Parts Play a Role in The Network?

• Hubs
• Switches
• Routers
• Firewalls
• Load Balancer
• Application Servers
• Database Servers
• Web Servers

19
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• In data communications, a hub is a place of
  convergence where data arrives from one or more
  directions and is forwarded out in one or more
  other directions. A hub usually includes a switch
  of some kind.See USB 7-Port HubTo connect
  severalUSB devices

20
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• In telecommunications, a switch is a network
  device that selects a path or circuit for sending
  a unit of data to its next destination.
• See 8-Port Network Switch

21
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• A switch may also include the function of the
  router, a device or program that can determine
  the route and specifically what adjacent network
  point the data should be sent to.
• In general, a switch is a simpler and faster
  mechanism than a router, which requires knowledge
  about the network and how to determine the route

22
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• In telecommunication networks, a bridge is a
  product that connects a local area network (LAN)
  to another local area network that uses the same
  protocol (for example, Ethernet or token ring).

23
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• On the Internet, a router is a device or, in some
  cases, software in a computer, that determines
  the next network point to which a packet should
  be forwarded toward its destination.See 4-Port
  Cable/DSL Router

24
Network Basics

• Hub / Switch / Bridge / Router / Firewall
• A system designed to prevent unauthorized access
  to or from a private network. Firewalls can be
  implemented in both hardware and software, or a
  combination of both.PIX Firewall from CISCO
• Private Internet Exchange Firewall
• Gauntlet Firewall from NAI
• Network Associates Inc.

25
Network Basics
Bridge
26
Internet
Firewall
SSL
CISCO Router
e.g. Apache
Web Server
Web Server
Firewall
e.g. WebLogic BEA
Appl. Server
Appl. Server
e.g. Attachmate
Open System
Screen Scraper
Screen Scraper
Firewall
Main-frame
Com-plete NATURAL ADABAS
CICS COBOL VSAM
Oracle FirstLogic DBs
27
Logical and Physical Transactions?
Web Browser
Web Server
Appl. Server
NATURAL
ADABAS
28
Response Time / Click Time Tests
Web Browser
Function Display my account
Web Server
Appl. Server
NATURAL
ADABAS
29
What Tools Can You Use to Test?

• Web Server Stress Tool
• User simulation
• Logging period
• Select URLs
• URL recorder
• Data merging
• Browser simulation
• Reviewing summary and detailed log results
• Analyzing graphical results

Web Browser
30
CSS Stress Test 5 Users 10 Clicks 25 Sep 2003,
1152-1153
31
CSS Stress Test 5 Users 10 Clicks 14 Oct 2003,
1405
32
09/25/2003 115202 AM --- WEBSERVER STRESS
TOOL User 1 Click 1 started, used URL 1
(CSS-Prod-1) POST-DATA
"emaildieter.storr_at_latimes.compasswordltsnipgt
User 1 Click 1 Request 1 HTTP State
Initialized (0 ms) User 1 Click 1 Request 1
HTTP State DNS Lookup begin (7 ms) User 1
Click 1 Request 1 HTTP State DNS Lookup done
(17 ms) User 1 Click 1 Request 1 HTTP State
Connected (504 ms) User 1 Click 1 Request 1
HTTP State Waiting for header (512 ms) User 1
Click 1 Request 1 HTTP State Receiving Header
(5937 ms) (Time to
First Byte) User 1 Click 1 Request 1 HTTP
State Waiting for body (5947 ms) User 1 Click
1 Request 1 HTTP State Done (5999 ms) User 1
Click 1 CLICK-Request 1 Time6008 ms,
TFB5937ms,
Bytes16038, HTTP-StatusCode200 (OK) User 1
Click 1 !Success! All requests of click done
(6046 ms)
33
What Tools Can You Use to Test?

• APAS/INSIGHT

ADABAS
DWSCWUSE SUMMARIZE (LS150)
COUNT (HD'COMMANDS')
PERCENT (HD' OF TOTAL' PFG10)
SUM(TOTAL-IO) (HD'TOTAL IO')
PCT(TOTAL-IO) (HD' OF TOTAL' PFG10)
MEAN(TOTAL-IO) (HD'AVG IO')
MAX(TOTAL-IO) (HD'MAX IO')
MEAN(DURATION) (HD'AVG DUR')
MAX(DURATION) (HD'MAX DUR')
MEAN(ENQ-TIME) (HD'AVG ENQ')
MAX(ENQ-TIME) (HD'MAX ENQ')
BY USER-ID
WHERE USER-ID(2) 'CW'
INSIGHT-LINES 30
PRINT TO STRESS01
34
ADABAS on Enterprise Server CSS Stress Test 5
Users 10 Clicks 25 Sep 2003, 1152-1153
USER ADABAS ID
COMMANDS OF TOTAL -------- ----------
----------- CW0016 269
CW0017 269 CW0018
269 CW0019 369
CW0020 369 CW0021
369 CW0022 369
CW0023 369
CW0024 269 CW0025
269 ---------- -----------
3,190 gt
AVG DUR SEC MAX DUR SEC ------------ ---------
0.00033 0.006 0.00052 0.008
0.00036 0.005 0.00058 0.014
0.00041 0.011 0.00043 0.008
0.00045 0.008 0.00038 0.007
0.00034 0.006 0.00049 0.009
------------ --------- 0.014
35
What Tools Can You Use to Test?

• APAS/INSIGHT

ADABAS
0090 DWSCWOBX EXTRACT (LS150)
0100 NAT-LOAD-COUNT
(HD'OBJ-LOAD') 0110 DATE TIME
0120
NAT-PROGRAM CMD 0130
USER-ID 0140
GLOBAL-ID
0150 WHERE USER-ID(2) 'CW' AND
0160 (NAT-PROGRAM
'IWS00P00' OR 0170
NAT-PROGRAM 'IWD10P10') 0180
INSIGHT-LINES 200 0190
PRINT TO STRESS01 0200
36
ADABAS on Enterprise Server CSS Stress Test 5
Users 10 Clicks 25 Sep 2003, 1152-1153
NATURAL PROGRAM OBJ-LOAD --------
-------- IWD10P10 50 --------
50
37
Response Time / Click Time Tests
200ms - 20s
Web Browser
Function Display my account
Web Server
Appl. Server
NATURAL
ADABAS
0.00033s - 0.014s
38
What Tools Can You Use to Test?

• STRESS -- self-written

ADABAS
1 Start process with heavy CPU workload
2
Start process with many ADABAS Calls
3
Start process with heavy CPU workload and
many ADABAS calls
4
Start process with many ADABAS calls and
excessive terminal I/O's
5
Start asynchronous processes (ADABAS CPU)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
(No Transcript)
44
What Tools Can You Use to Test?
TP Monitor

• Com-plete
• UCTRL
• NETSPY
• TMON/MVS

45
UCTRL
174914 TID 6 COMTEST
USER ZINDWS 03/16/04
--- COM-PLETE CONTROL FACILITY ---
UCT1 Function
FC Function FC Function
FC -------------------- --
-------------------- -- --------------------
-- ADABAS interface ... AI Program lookaside
.. IP Task groups ........ PG ADABAS
statistics .. AS Program in-stor dir. OP
Task list .......... PL Buffer activities .. BA
Resident programs .. RP Thread groups
...... TG Common storage ..... CS Find
program ....... FP Thread subgroups ...
SG DB2 thread status .. D2 Queue overview
..... QO Thread list ........ TL FBPM
statistics .... FB Roll activities .... RA
User activities .... UA Message log ........ CM
Server overview .... SO User
status......... US
Terminal overview .. TO VSAM statistics ....
VS Terminal status
.... TS
Print
statistics ... PR Select
function ....
Operand ............

Operator command

Enter-PF1---PF2---PF3
---PF4---PF5---PF6---PF7---PF8---PF9---PF10--PF11-
-PF12--- End

46
NETSPY
47
TP Monitor Com-plete Consumption
48
TMON

• Alerts
• Average response time
• CPU usage
• I/O usage
• Number of ended transactions rate

49
What Tools Can You Use to Test?
Screen Scraper

• Attachmate
• Run Time Application
• Management Control Services (MSC) -- monitoring
  pools and sessions
• Java console running in foreground

50
What Tools Can You Use to Test?
Screen Scraper

• Attachmate Run Time Application
• Number of sessions to host (mainframe)
• Maximum sessions
• Number of failures
• Queue size

51
What Tools Can You Use to Test?
Screen Scraper

• Attachmate Servers
• NT utilities
• Sampling at 10-second intervals

52
What Tools Can You Use to Test?
Screen Scraper

• Attachmate Servers
• Analyze results
• CPU utilization
• Memory utilization
• Run queues
• Swap space utilization
• Disk I/O - most busy which disk hit the hardest
• I/O wait gt 10
• Page file - swaps

53
What Tools Can You Use to Test?
Content Server Switch

• Cisco 2900 Switch
• Manual testing/sampling
• Analyze results
• Load on network components
• Packet throughput

54
What Tools Can You Use to Test?
Router

• Cisco 4000 Router (10BaseT)
• Manual testing/sampling
• Analyze results
• Load on network componentshttp//www.scd.ucar.edu
  /nets/devices/routers/ios_analysis.html
• Packet throughput
• Router Configuration http//www.swcp.com/jgentry
  /topo/cisco.htm

55
(No Transcript)
56
What Tools Can You Use to Test?
RDBMS

• Oracle
• Oracle utilities
• Application-specific log files
• Performance Monitoring and Tuning Tools
• TKProf
• UTLBSTAT.SQL and UTLESTAT.SQL - Begin and end
  stats monitoring
• Statspack
• Oracle Enterprise Manager - Tuning Pack

57
What Tools Can You Use to Test?
RDBMS

• Oracle
• Performance Monitoring and Tuning Tools (cont)
• DBXRAY A real-time graphical snapshot of the
  health of your Oracle database- anytime,
  anywhere.
• Patrol Intelligent and non-intrusive for
  proactive database monitoring, including Oracle
  Parallel Server and Oracle Symmetric Replication
  environments.
• SQL-Explorer for Oracle The most advanced SQL
  tuning tool available. Identify, model and tune
  problematic SQL statements to boost application
  performance.

58
What Tools Can You Use to Test?
RDBMS

• Oracle
• Analyze results
• Number of Oracle processes
• Buffer hit cache
• Block I/O rate
• Number of cursors created
• Number of connections
• Log information regarding number of
  registrations, starts, etc.

59
What Tools Can You Use to Test?

• SSL Accelerator
• Secure Socket Layer (protocol between HTTP and
  TCP) -- appliance or card
• Manual testing/sampling
• Analyze results
• Load on network components
• Packet throughput
• Optimizing SSL processing for Web
• http//www.nwfusion.com/news/tech/2000/1023tech.ht
  ml?nf

SSL
60
What Tools Can You Use to Test?
Firewall

• PIX
• Manual testing/sampling
• Analyze results
• Load on network components
• Packet throughput

61
What Tools Can You Use to Test?
Firewall

• Gauntlet (several)
• Manual testing/sampling
• Analyze results
• Memory
• Disk I/O
• Full system log

62
What Tools Can You Use to Test?
Web Server

• Sun OS
• Solaris utilities
• Sampling at 10-second intervals

63
What Tools Can You Use to Test?

• Sun OS
• Analyze results
• CPU utilization
• Memory utilization
• Run queues
• Swap space utilization
• Disk I/O -- most busy which disk hit the hardest
• I/O wait gt 10
• Page file -- swaps

Web Server
64
What Tools Can You Use to Test?
Web Server

• Apache Application
• Log Analyzer
• Analyze results
• Number of sessions
• Bytes per session
• Source address
• Destination address

65
What Tools Can You Use to Test?
Application Server

• Sun OS
• Solaris utilities
• Sampling at 10-second intervals

66
What Tools Can You Use to Test?

• Sun OS
• Analyze results
• CPU utilization
• Memory utilization
• Run queues
• Swap space utilization
• Disk I/O -- most busy which disk hit the hardest
• I/O wait gt 10
• Page file -- swaps

Application Server
67
What Tools Can You Use to Test?
Application Server

• BEA WebLogic Application
• WebLogic administrative console
• Spotlight

68
What Tools Can You Use to Test?
Application Server

• BEA WebLogic Application
• Analyze results
• Number of active queues
• Number of connections
• Number of active sockets
• Types of access
• Time of access

69
Web Site PerformanceAnalysis and Tuning

• In-House or Web Caching
• Near the network edge, e.g. proxy cache or
  browser static HTML, JavaScript, and image files
• Cache-control header no-cache re-fetch
  expiry date
• Compression
• HTML is compressed before being sent through the
  network.

70
Web Site PerformanceAnalysis and Tuning

• SSL Acceleration (box or card)
• Consider the number of transactions per second
  (TPS). TPS will be more important on SSL
  Acceleration appliances than cards.
• An SSL Acceleration hardware appliance (box) can
  deal with transactions for all your web servers
  and scale quickly.
• Network Load Balancing
• Round Robin
• Lowest response time
• Least connections
• Proximity

71
Checklist -- Getting Started?

• Determine your environment
• Mainframe, Open System, Middleware
• Evaluate Stress Test Software
• Write your own or buy one
• Understand the different components of a web site
  download
• DNS lookup
• Redirect
• First byte download
• Base page download
• Content download

72
Checklist -- Getting Started?

• What kind of content?
• Dynamic or static
• .html or image
• Number of servers
• Where does traffic comes from
• Internet, Intranet, or Extranet

73
Create a Stress Test Plan

• Test Organization
• Overview
• Participants
• Technical team
• Coordination team
• Application team
• Objectives
• Infrastructure
• Application
• Assumptions / pretest conditions
• Communication plan

74
Create a Stress Test Plan

• Environment / Test Components
• Overview
• Overall success criteria
• Transactions to be tested
• Logical and physical
• Application functions
• Data indentification
• Script development
• High-level system diagram
• Components
• Responsibility
• Metrics
• Tool / method

75
Create a Stress Test Plan

• Sequence of Events
• Summary
• Process
• Schedule of events

76
Web Speed is no Magic!
77
Links to Interesting Websites

• http//www.edwardtsai.com/web-site-performance.htm
  l
• http//www.nwfusion.com/news/tech/2000/1023tech.ht
  ml?nf
• http//www.scd.ucar.edu/nets/devices/routers/ios_a
  nalysis.html
• http//www.swcp.com/jgentry/topo/cisco.htm

78
Thank You!