Automating your Website Testing

1
Automating your Website Testing

• By
• Andrew L. Pollner and Lei Wang
• ALP International Corporation

2
Why Load Test?

• To Simulate Multiple Users on a System
• To Measure end-to-end performance
• To Measure Server Performance
• Database (Oracle)
• Application (CF)
• Middleware (Web Logic)
• Web (Netscape)

3
When Should Loadtesting Occur?

• When sizing a server
• After Application Has Stabilized
• When other users are not on the system
• After notifying Sys Admin.s, DBAs, Test Managers

4
Whats Needed for Loadtesting?

• Knowledge of
• Application
• Database
• Environment
• LoadRunner!

5
Loadtesting Concepts - Part I

• Measured event (i.e. Time it takes to return
  data to a screen after a QUERY button is pressed)
• Synchronized point where transaction is to occur
  simultaneously

• Transaction
• Rendezvous

6
Loadtesting Concepts - Part II

• Vuser
• Sgroup
• Scenario

• A defined user type which
• executes a specific test
• A group of Vusers which with a
• characteristic profile
• A combination of sgroups

7
Types of Vusers

• Web
• C/S GUI
• Terminal Emulation
• SQL

8
Using LoadRunner - Web

• Record HTTP Script
• Generalize trace
• Add Transaction points
• start_transaction(ClaimWriteback)
• end_transaction(ClaimWriteback, PASS)
• Add Rendezvous
• rendezvous(Login) rendezvous(ClaimWriteback)

9
Start Synchronization
Rendevouz 1 gt
Rendevouz 2 gt
Rendevouz 3 gt
10
Step Synchronization
Rendevouz 1 gt
Rendevouz 2 gt
Rendevouz 3 gt
11
Using LoadRunner - DB

• Get SQL trace
• Generalize SQL trace
• Add Transaction points
• start_transaction(ClaimWriteback)
• end_transaction(ClaimWriteback, PASS)
• Add Rendezvous
• rendezvous(Login) rendezvous(ClaimWriteback)

12
Product Categories

• GUI/Web vusers
• Netscape Browser
• IE Browser
• Client Server
• RTE vusers
• Terminal Emulation
• SQL (dB) vusers
• Client Server

13
LoadRunner Family of Products

• Both Client AND Server Load Testing
• Wide range of platforms
• Win 2000
• NT
• Win95/98
• UNIX
• Reporting
• Automatically for each user
• By transaction

14
Client AND Server Load TestingTesting the Site
SYBASE SERVER
SQL SERVER
NT

• Single point of control Controller

15
Client AND Server Load TestingTesting the Site
SYBASE SERVER
SQL SERVER
NT

• Single point of control Controller
• Drive multiple real NT GUI clients End-to-End

16
Client AND Server Load TestingTesting the Site
SYBASE SERVER
NT
SQL SERVER
NT

• Single point of control Controller
• Drive multiple real NT GUI clients End-to-End
• Stress SQL Server with multiple virtual NT clients

17
Client AND Server Load TestingTesting the
Enterprise
SYBASE SERVER
NT
NT/UNIX

• Single point of control
• Drive multiple real clients
• Stress SQL Server virtual clients NT
• Stress Sybase virtual clients NT/UNIX

18
Advanced Loadtest Measurement

• Mean vs. Median
• 90 Percentile
• Performance under load
• System Performance Measurement
• CPU
• SWAP
• DISK

19
Analyzing Loadtest Results - I
CLIENT
LAN
WAN
SERVER
SERVER
Sybase
SQL Server
Total System Performance Client, LAN, SQL
Server, WAN, Sybase
Client Load
20
Analyzing Loadtest Results - II
CLIENT
LAN
WAN
SERVER
SERVER
Sybase
SQL Server
Client Load
Transaction Performance LAN, SQL Server, WAN
Sybase
Server Load
21
Analyzing Loadtest Results - III
CLIENT
LAN
WAN
SERVER
SERVER
Sybase
SQL Server
Client Load
Transaction Performance WAN, Sybase
Server Load
22
Analyzing Loadtest Results - IV
CLIENT
LAN
WAN
SERVER
SERVER
Sybase
SQL Server
Client Load
Transaction Performance SYBASE
Server Load
23
Analyzing Loadtest Results - V
CLIENT
LAN
WAN
SERVER
SERVER
Sybase
SQL Server
Client Load
Transaction Response Time Client
Transaction Response Time LAN, SQL Server
Server Load
24
Conclusions

• Loadtesting
• Is a necessary component in testing an
  application and its environment
• Has multiple purposes
• Comes in many flavors
• Is an extension to Functional testing
• Is complex!

25
DEMO!
26
Virtual User Generator
27
LoadRunner Controller
28
Controller Vusers
29
Vuser Protocol
30
Transaction Report
31
Page Faults Graph
32
On-line Monitors
33
QA