1
Web2K Bringing QoS to Web Servers
Bhoj, Preei, Ramanathan, Srinias and Singhal,
SharadHP Labs Technical Reports, 2000

• Network Computing Lab, EECS, KAIST
• Hyun-ju Lee

2
Content

• Introduction
• System architecture
• Acceptor design
• Performance Evaluation
• Configuration model
• Summary

3
Introduction

• Motivation
• Mission-critical commerce services is motivated
• Ex) online banking, reservations, stock trading,
• Stable service to user is very important
• Its related directly to revenues
• Session-based admission control is important
• Session a series of requests issued by a user
  within a pre-defined, period of time(e.g., 30
  mininutes)

4
Introduction(Contd)

• Approaches to provide QoS
• No modification to the OS or the web application
  server software
• Predictable QoS
• Differentiated service to different classes of
  users
• Session-based admission control
• ? Web2K Server

5
Web2K Server Overview

• The Web2K server does not rely on any operating
  system support to enable differential QoS for
  users
• Middleware approach
• Middleware in the form of a dynamically linked
  library that is transparently added to a web
  server
• An intermediary between the web server and the
  TCP/IP stack
• Classification of incoming requests as belonging
  either to a premium class or to a basic class
• Premium class requests are serviced first

6
System Architecture

• The middleware layer intercepts a web servers
  TCP socket calls, receives and prioritizes
  requests, and forwards them to the web server for
  processing

7
System Architecture(Contd)

• Acceptor
• Listen on the web servers TCP port
• Receive new connection
• Session management existing session or not
• Introduce a special cookie
• Classification of requests
• All requests in a session share the same class
• Class my be upgraded depending on the context of
  the underlying transactions
• Admission control
• Decide whether or not the request should be
  admitted for service
• Return a reject response or queue the request in
  priority queue

8
System Architecture(Contd)

• QoS Library
• At the time of installation
• Map TCP socket calls made by the web server to
  corresponding calls to a library of QoS-enabled
  functions
• Web server receive new HTTP requests from the
  priority queues, rather than via the TCP socket
  library

9
Acceptor Design

• Traffic-Aware Request Processing

10
Acceptor Design(Contd)

• Operate in repeated cycles
• Receive up to n connections from the servers TCP
  listen queue in each cycle
• Rapidly draining the TCP listen queue
• Rare TCP timeouts
• Low response time both for requests that are
  serviced as well as for requests that are
  rejected
• Priority can be used even under overload
• Process a subset m of the requests it received

11
Acceptor Design(Contd)

• Keep the number of requests waiting for service
  in the servers TCP listen queue to a minimum
• Maintain an internal queue of requests that it
  has accepted but not yet processed acceptor
  queue
• Smaller value of m when the server is overloaded
• Larger value of m when the server is lightly
  loaded
• Adjusts the value of m based on the input load
• Doubled or halved
• Advantages
• Significant improvement of the response times
  seen by users
• Decision of request deny can be based on the
  classification of requests
• Provide an operating system independent method of
  determining the occupancy of the listen queue

12
Acceptor Design(Contd)

• Early Prioritization in the Acceptor

13
Acceptor Design(Contd)

• Decide which requests to process in each cycle
• Session management
• If detect a premium request, it immediately
  processes the request, adding the request to the
  premium queue(from which the web server receives
  new requests)
• If detect a basic request, hold it in pending
  request queue(ESR, NSR)
• To process pending basic request, use
  look-ahead(k)
• Scan from the acceptor queue looking for premium
  requests before it decides to process a basic
  request
• Provide prioritization

14
Acceptor Design(Contd)

• Advantages
• Depending on the look-ahead value, premium
  requests are serviced at higher priority than
  basic requests
• Classification overheads are minimal for premium
  requests as compared to basic requests

15
Acceptor Design(Contd)

• Predictive Queue Control
• During period time t-T, t
• in(t) of requests that the acceptor receives
• out(t) of requests for that class that the
  web server application processes
• q(t) length of the basic priority queue at t

16
Acceptor Design(Contd)

• Current queue length at t q(t)
  q(t-T)in(t)-out(t)
• target queue length target_q(t)
• e(t) target_q(t)-q(t)
• of requests that can be added to the priority
  queue during the next periodt, tT
• target_in(t) in(t)q0e(t)q1(t-T)
• q0, q1 constants that are determined based
  on the transient response
• Controller determine target_in(t)

17
Acceptor Design(Contd)

• Actuator ensure that no more than target_in(t)
  requests are added to the priority queue
• Estimate of the incoming request rate
• In_rate_estimate(t)
• Probability p (target_in(t) /
  in_rate_estimate(t))
• Whenever possible, attempt to admit requests
  belonging to existing session
• Requests that are denied admission by the
  actuator are held back in a set of discard queues
• Requests in discard queues access admitted or
  denied(terminate TCP connection)
• Avoid rejection of requests for short spikes in
  server load

18
Performance evaluation

• Test methodology
• Apache server(vanilla web server)/Web2K server
• Two type of workload
• Static/dynamic
• Key metrics
• Percentage of incoming premium sessions that were
  successfully handled by the server
• Average time for the successful completion of a
  premium session
• Total number of sessions successfully handled by
  the server, for a given workload

19
Performance evaluation(Contd)

• Test configuration
• OS R-class HP-UX 11.0 system
• Three workstations connected to the server
• basic user session(2) / premium user session(1)
• Look_ahead 12
• target priority queue length 500
• Httperf for the performance study
• Workload
• The content used to generate the synthetic
  workload mirrored a subset of a popular on-line
  banking web site
• Static workload 10 HTML pages, 30 embedded
  images
• dynamic workload two of the HTML accesses were
  replaced by accesses to CGI programs

20
Performance evaluation(Contd)

• Performance Results
• Static workload
• Each user session 40 unique URL
• 10 HTML pages(20020kbyte), 30 embedded images
• Premium session 10 sessions/sec
• Basic session 650 sessions/sec

21
Performance evaluation(Contd)

• The of successful session (premium/basic)

22
Performance evaluation(Contd)

• The duration of successful session(premium/basic)

23
Performance evaluation(Contd)

• Dynamic workload
• Each user session 40 unique URL
• 8 HTML pages(20020kbyte), 2 CGI program(25 ms),
• 30 embedded images
• Premium session 4 sessions/sec
• Basic session 644 sessions/sec

24
Performance evaluation(Contd)

• The of successful session (premium/basic)

25
Performance evaluation(Contd)

• The duration of successful session(premium/basic)

26
Performance evaluation(Contd)

• Overheads of the Web2K Server
• The total of successful session(static/dynamic)
• Static workload 1030 of overhead
• Dynamic workload about 10 of overhead

27
Performance evaluation(Contd)

• Tuning the Web2K Server
• Many parameters
• statically assigned values
• TCP listen queue length, acceptor queue length,
  etc.
• Look_ahead value, priority queue occupancy length
• Be based on operator preference
• Determine the level of prioritization, Web2K
  overheads,
• Response time

28
Performance evaluation(Contd)

• Choosing the look-ahead
• Total successful sessions for static workload

29
Performance evaluation(Contd)

• of successful each class session
• 100 of successful premium session
• look ahead value
• ( of premium requests/ of
  total requests) -1

30
Performance evaluation(Contd)

• Response time

31
Performance evaluation(Contd)

• Choosing priority queue occupancy targets

32
Configuration model

• QoS-aware processing
• Depend on the operator-specified preferences,
  rather than static guarantee specifications
• Degree of differentiation
• Minimum response time/maximum of sessions
• Provide User interface
• Specify operators preference
• Display the Web2K servers performance

33
Summary

• QoS enabled web server
• Differentiated service to different classes of
  users without requiring any modifications to the
  OS or the web application server software
• Traffic-aware request processing
• Handle the majority of incoming requests, even at
  extreme overloads
• Predictable QoS
• Session-based admission control