A Case Study of Web Server Benchmarking Using Parallel WAN Emulation

1
A Case Study ofWeb Server Benchmarking
UsingParallel WAN Emulation

• Carey Williamson
• Rob Simmonds Martin Arlitt
• University of Calgary

2
Network Emulation

• A hybrid performance evaluation methodology that
  combines aspects of implementation with
  simulation modeling
• A network emulator is a network simulator with an
  interface that allows client applications to
  interact with it in real-time
• A simulator that talks back (IP packets)
• Why? Provides a reliable, repeatable test
  environment for distributed applications
• Internet games, video conferencing, Web, ...

3
Simulation
Real World
4
IP-TNE

• The Internet Protocol Traffic and Network
  Emulator (IP-TNE) is a network emulator based on
  IP-TN (packet-level IP network simulator)
• Enables interaction between IP based clients via
  an IP-TN simulated network (in real time!)
• Distributed applications can interact with IP-TNE
  without modification

5
IP-TNE Overview
6
IP-TNE Overview (contd)

• CCTKit with real-time extensions provides an
  environment for fast network emulation (PDES)
• IP-TNE provides routing methods suitable for
  shared environments and dedicated test
  environments
• Now has HTTP and TCP client models that can be
  used for Web server benchmarking

7
So What?

• Flexible routing model support
• High-performance packet reading and writing via
  raw sockets (1 Gbps)
• Can model an arbitrary IP internetwork
• Detailed IP protocol models
• IPv4, ICMP, ping, traceroute, pchar, MTU, ...
• Supports parallel execution on shared
  memory multiprocessors
• Blazingly fast! - CLW, 2002

8
Example Web Benchmarking
Client 1
Client 2
Client 3
Web Server
...
Client C
9
WAN Emulation (1 of 3)
Client 1
Client 2
Client 3
Web Server
...
Client C
Centralized Approach
10
WAN Emulation (2 of 3)
Client 1
Client 2
Client 3
Web Server
...
Shim Approach (NISTnet, DummyNet, WASP)
Client C
11
WAN Emulation (3 of 3)
Client 1
Client 2
Client 3
Web Server
...
Client C
Our IP-TNE Approach
12
Objectives of Case Study

• Evaluate new approach to WAN emulation, and
  demonstrate feasibility
• Confirm prior results by Nahum et al. on effects
  of WAN conditions on Web server performance
• How fast can Apache Web server go?
• How fast can IP-TNE go?

13
Experimental Setup

• IP-TNE on Compaq ES-40 (4 CPU)
• Apache (1.3.23) on another ES-40
• Gigabit Ethernet (1 Gbps) in between
• OS is Compaq Tru64 (v5.1A)
• ANML for defining network model
• e.g., simple regular WAN topology

14
(No Transcript)
15
(No Transcript)
16
Web Workload Model

• Static content only
• Closed-loop workload generator
• Fixed-size Web objects
• Small (1 KB)
• Large (64 KB)
• Variable-size Web objects
• Median 3 KB
• Mean 9 KB
• Pareto heavy tail (alpha 1.2)
• Zipf-like document popularity profile

17
Performance Metrics

• Two primary metrics
• HTTP transaction rate (trans/sec)
• Network throughput (Mbps)
• Several secondary metrics
• Response time
• Connection failure rate
• Packet loss rate
• ...

18
Results (Fig. 4a)

• For 1 KB transfers with HTTP/1.0
• Single client 170 transactions/sec
• Transaction rate scales up with number of clients
  up to about H 32
• Transaction rate flattens, then drops sharply as
  num clients is increased more (closed loop)
• Peak rate achieved 3800 trans/sec
• Peak throughput approximately 40 Mbps
• Transaction rate is (strongly) inversely related
  to the client round trip time (RTT)

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Results (Fig. 4b)

• For 64 KB transfers with HTTP/1.0
• Single client 18 transactions/sec
• Transaction rate scales up with number of clients
  up to about H 32
• Transaction rate flattens, then drops slightly as
  num clients is increased more
• Peak rate achieved 220 trans/sec
• Peak throughput approximately 115 Mbps
• Transaction rate is (weakly) inversely related
  to the client round trip time (RTT)

25
(No Transcript)
26
Results (Fig. 4c)

• For variable-size transfers with HTTP/1.0
• Single client 60 transactions/sec
• Transaction rate scales up with number of clients
  up to about H 32
• Transaction rate flattens, then drops
  as num clients is increased more
• Peak rate achieved 1300 trans/sec
• Peak throughput approximately 90 Mbps
• Transaction rate is inversely related to the
  client round trip time (RTT)
• Behaviour is in between 1 KB and 64 KB results

27
(No Transcript)
28
Results (Fig. 5a)

• Concurrent connections with HTTP/1.0
• Single client 600 transactions/sec
• Qualitatively similar results to before, except
  that fewer clients are needed to drive the server
  to full load
• Conceptually concurrent connections are no
  different than adding more clients

29
(No Transcript)
30
Results (Fig. 5b)

• Persistent connections with HTTP/1.1
• Single client 300 transactions/sec
• Qualitatively similar results to before, except
  that transaction rate is about 70 higher than
  for HTTP/1.0 (since multiple HTTP
  reqs per TCP conn)
• Peak transaction rate 6500 trans/sec
• Much less dependency on RTT effects

31
(No Transcript)
32
Results (Fig. 5c)

• Pipelined persistent connections with HTTP/1.1
• Single client 800 transactions/sec
• Qualitatively similar results to before, except
  that transaction rate is about 100 higher than
  for HTTP/1.0
• Peak transaction rate 7600 trans/sec
• Much less dependency on RTT effects

33
(No Transcript)
34
Results (Fig. 6a)

• Effect of WAN RTT delays
• Increasing the per-link propagation delay
  increases the client RTT delay, which in turn
  reduces the transaction rate and throughput (as
  expected)
• As RTT increases, more and more clients are
  needed in order to drive the Web server to full
  load
• Similar to Nahum et al. 2001

35
(No Transcript)
36
Results (Fig. 6b)

• Effect of bandwidth asymmetry
• For asymmetric access technologies such as ADSL
  (Asymmetric Digital Subscriber Line), the
  upstream link from the client to the server can
  sometimes be the bottleneck for TCP, even though
  it is primarily carrying ACKs only
• Depends on normalized bandwidth ratio
• Greater asymmetry, worse performance

37
(No Transcript)
38
Results (Fig. 6c)

• Effect of WAN packet losses
• Decreasing the router queue size at the
  bottleneck link increases the packet loss ratio
  (as expected)
• As the level of packet loss increases, the HTTP
  transaction rate and the network throughput
  decrease
• Similar results to Nahum et al. 2001

39
(No Transcript)
40
Results (MTU Size)

• Effect of MTU size
• The smaller the MTU size, the lower the
  transaction rate and network throughput
• Reasons
• Smaller MTU size increases the relative header
  overhead per packet
• Smaller MTU size increases the number of packets
  that need to be sent
• Smaller MTU size (and MSS) increases the number
  of RTTs required by TCP

41
Summary and Conclusions

• The IP-TNE is a useful tool for Web server
  benchmarking
• Demonstrates feasibility of WAN emulation using a
  single computer
• Confirms prior results by Nahum et al. studying
  the effects of WAN conditions on Web server
  performance
• Demonstrates performance advantages of HTTP/1.1

42
Future Work?

• Increase traffic to your Web site!!!
• Guarantee 20 increase in traffic!
• 1000s of new clients per month!!!!!
• Send to carey_at_cpsc.ucalgary.ca

43
Future Work with IP-TNE

• Validation of IP-TNE (and IP-TN)
• Benchmarking IP-TNE vs IP-TNE
• Benchmarking Web caching appliances
• Evaluating SRPT scheduling in WAN setting
• Connection/packet-level scheduling algorithms
• Evaluating CATNIP approach to TCP/IP
• Evaluating portable (wireless) Web servers
• Workload sensitivities (Zipf, Pareto, corr, mods)
• Experiments with dynamic content (CGI, etc)
• Asymmetric networks, Ensemble-TCP
• Parallel TCP connections friend or foe?
• Evaluating effect of TCP SACK in WAN