Load Testing of Web Sites Daniel A' Menasc George Mason University

1
Load Testing of Web Sites Daniel A.
MenascéGeorge Mason University
Internet Computing, IEEE Volume 6, Issue 4,
July-Aug. 2002 Oct. 22. 2007 Byungchul
Park fates_at_postech.ac.kr DPNM Lab., Dept. of
CSE, POSTECH
2
Contents

• Introduction
• QoS Measures
• Availability
• Response time
• Load Testing
• How to use it
• When to use it
• Testing parameters and result
• Load Testing and Performance Relationship
• Tool Requirements
• Conclusion

3
Introduction

• Web applications quality of service
• Response time
• Throughput
• Availability
• Poor QoS ? frustrated customers, lost business
  opportunities
• Company expenditures F(sites expected traffic)
• To maximize ROI, we must determine when and how
  to upgrade IT infrastructure.
• This paper describe the QoS factors load testing
  addresses, how to conduct load testing, and how
  it addresses business needs at several
  requirement levels.

4
QoS Measures (1/2)

• QoS is key to assessing how well Web-based
  applications meet customer expectations on two
  primary measures availability and response time
• Availability
• The percentage of time customers can access a
  Web-based application
• Availability vary according to application type
• critical applications vs. normal applications
• Availability vary according to the time of day or
  for special events
• Flash crowds
• Availability vary according to different
  customers
• Geographic location

5
QoS Measures (2/2)

• Response Time
• End-to-end response time to determine how
  customers perceive such things as page download
  and keyword search time
• A customers perception of a Web applications
  response time varies according to many different
  factors
• your sites ISP,
• the customers ISP,
• the customers ISP-connection bandwidth, which
  networks route packets from the customer to your
  Web site
• the delays imposed by your Web sites third party
  services
• End-to-end response time is time- and
  space-dependent
• E.g. International Olympics Committee site
  measurement result (9.0 sec for Asian users, 3.97
  sec for users in Europe)

6
Load Testing
7
Load Testing How It Works

• The load generator mimics browser behavior
• The load generator can emulate thousands
  concurrent users to test Web site scalability
• Virtual user each emulated browser
• A key load-testing concept
• You must ensure that your virtual users
• Follow patterns similar to real users
• Use realistic think times
• React like frustrated users, abandoning a Web
  session if response time is excessive
• Obtaining metrics such as response time and
  throughput for each load intensity value

8
Load Testing When to Use It

• Several circumstances call for load testing
• Example 1 Anticipating a significant traffic
  increase
• Q. The peak of starting session is 3,000 per
  hour. What will be the response time when the
  sites load increases to 6,000 sessions per hour?
• Example 2 Adding new functionality or
  redesigning
• Q. How this will affect response time?
• Q. How can you detect potential performance
  problems and fix them before they occur?
• Example 3 Planning to implement a new services

9
Load Testing Testing Parameters and Results

• Three main parameters to vary during a load test
• Workload intensity
• Workload mix
• Customer behavior parameters
• Typical load test results include
• Number of completed and abandoned sessions per
  hour
• Revenue and potential lost revenue throughput
• Individual page download time and transaction
  completion time
• ?Function of the number of started session per
  hour

10
Load Testing and Performance Relationship (1/3)

• Simply increasing the number of virtual users
  until we achieve the desired load
• Load testing can be time-consuming and expensive
• We can obtain faster, albeit less accurate,
  results by combining load testing with analytic
  or simulation performance models
• Speed up scalability analysis with load testing

11
Load Testing and Performance Relationship (2/3)

• Definition of variables
• NVU number of virtual users
• NC number of concurrent requests a Web site is
  processing
• Z average think time, in seconds
• R average response time for a request, in
  seconds
• X0 average throughput, in requests per second
• Response Time law 23
• Throughput is a functions of the load level and
  the service demands on individual site resources
• Di a requests service demand at resource i as
  the average total time the request spends
  receiving service from the resource 3

12
Load Testing and Performance Relationship (3/3)

• Combining equation 1-3, we get
• We can estimate the number of virtual users we
  need to generate a given value of Nc
• Estimating maximum throughput 3

13
Tool Requirements (1/3)

• There are several important requirements related
  with the four layers of the hierarchical
  framework for e-business 3
• Top level business model
• Track revenue throughput and potential lost
  revenue throughput
• Carry out load test under the most realistic and
  thorough conditions possible to avoid over- and
  under-provisioning the IT infrastructure
• Understand how business decisions affect the IT
  infra structure

14
Tool Requirements (2/3)

• Functional level (functions that implement the
  site)
• Load test functions supported by many different
  technologies, including Flash, JavaScript,
  ActiveX, cookies, and SSL
• Perform load testing for the functions you offer
  wireless clients
• Account for functions that use streaming media
• Customer behavior model (users navigation
  patterns)
• Flexibly and easily record scripts that represent
  different types of interactions
• Easily adapt the load-testing scripts to changes
  in customer behavior over time
• Realistically model customer behavior

15
Tool Requirements (3/3)

• Model of IT resources (to support site
  activities)
• Test Web applications on a regular basis in the
  actual production environment
• Assess the impact of changes in the system
  architecture
• Detect the IT infrastructure elements that causes
  performance problem
• Carry out load tests on-demand and at scheduled
  times
• Existing load-testing tools
• Keynote Systems (www.keynote.com)
• Mercury Interactive (www.mercuryinteractive.com)
• Cyrano (www.cyrano.com)
• Empirix (www.eimpirix.com)
• Segue (www.segue.com)

16
Conclusion

• High-volume Web sites are becoming more complex
  due to several factors
• Use of third-party services
• Geographical distribution and duplication
• Streaming media features
• Wireless access
• These factors significantly affect Web sites
  performance and scalability
• Variability in the measurement will increase in
  the future
• We have to collect more data to achieve
  statistically meaningful result

17
Question and Discussion