Design%20and%20Evaluation%20of%20a%20Model%20for%20Multi-tiered%20Internet%20Applications

1
Design and Evaluation of a Model for
Multi-tiered Internet Applications

• Bhuvan Urgaonkar
• Internship project talk Services Management
  Middleware Dept, IBM
• Aug. 20, 2004

2
Internet Data Centers

• Internet applications run on data centers
• Server farms
• Provide computational and storage resources
• Service-level agreements
• Response time guarantees

• Problem Need good application models to
  determine right resource allocations

3
Multi-tiered Applications

• Internet applications multiple tiers
• Example 3 tiers HTTP, J2EE app server, database
• Replicable components
• Example clustered HTTP, J2EE server

4
Existing Application Models

• Several models for single-tier apps
• Queuing models for web servers Chase et al
  (USITS 03), Chandra et al (IWQoS 2003)
• PODC 2004 G/G/1 based model
• Model only one (bottleneck) tier
• Ranjan et al (IWQoS 2002), Villela et al (IWQoS
  2004)

5
Black-box Approach

• Black-box approach
• Treat application as a black-box
• Measure response time from outside
• Increase allocation if response time gt SLA
• Use a model to decide how much to allocate
• Strawman 1 black-box for multi-tier apps
• Problems
• Unclear which tier needs more capacity
• Bottleneck tier may not be replicable

6
Black-box Approach
7
Extension of Single-tier Model

• Strawman 2 use single-tier provisioning
  independently at each tier
• Example
• Breakdown resp time into per-tier delays
• Use G/G/1 model for each tier
• Problems
• How to breakdown resp time?
• G/G/1 based model found to be very conservative!
• Wasted capacity

8
Talk Outline

• Motivation
• Multi-tier Application Model
• Preliminary Evaluation
• Ongoing Work / Discussion
• Summary

9
Key Insights/Observations
requests
Load balancing gateway
database
http
J2EE

• Tier i requests service from tier (i1)
• Scheduling PS closest tractable among policies
• Session-based workloads
• A session consists of a succession of requests
• Think times are user dependent
• Quantity of interest Per-request resp time

10
Queuing-theoretic Model

• Example 3-tier application
• Natural model Network of queues shown below

X_2
X_1
X_3
Z
. . .

• Capturing session-based workload
• Infinite server system
• Closed-queuing system

11
Mean-value Analysis

• MVA algorithm
• Inputs Avg service times, visit ratios
• Computes avg. delays and resp. time

EZ
EX_3
EX_2
EX_1
. . .
Visit ratios
V_01
V_2
V_1
V_3
12
MVA Algorithm

• Key relation Am(N) Qm(N-1)
• Am customers an arriving customer finds in
  queue m
• Qm customers in queue m

• for m 1 to M do Qm 0
• for n 1 to N do begin
• for m 1 to M do
• Rm Dm for the infinite server
• Rm Dm(1Qm) for other servers
• X n / S Rm throughput
• for m 1 to M do Qm XRm Littles Law
• end

13
MVA Algorithm Discussion

• Can handle any service time distrib if scheduling
  discipline is PS
• Extension to multiple classes exists
• Need to measure service times and visit ratios on
  a per-class basis
• Gives only averages, not distribs
• Each queue is really only modeling a single
  resource

14
Finding Model Parameters

• Visit ratios
• Easy to obtain from various logs
• E.g. Apache-tomcat-mysql
• V_apache 2
• V_mysql avg queries per servlet
• V_tomcat V_mysql 1

15
Finding Model Parameters

• Service times
• Apache and Tomcat can be made to log time spent
  at and beyond them
• X_apache
• (T_apacheT_tomcat)/2
• X_tomat
• (T_tomcatV_mysqlX_mysql)/(V_mysql1)
• X_mysql
• avg. query exec. time

16
What we havent captured

• Inter-tier load balancers
• Resources held at tier i while awaiting response
  from tier (i1)
• Increased service times at high loads
• E.g. context switches, protocol processing,
  contention for locks
• Tails of response times
• Multiple resources
• Load imbalances due to session affinity

17
Test Applications

• RUBiS (eBay like auction app)
• BrowseItems, PutBid, AuthorizeBid, PutComment,
  RegisterUser, SellItem, SearchItem, StoreBid,
• RUBBoS (slashdot like b-board app)
• AcceptStory, BrowseStories, ModerateComment,
  PostComment, RegisterUser, RegisterStory,
  ViewStory,

18
Experimental Setup
Mysql
Tomcat
Apachemod_jk
Client

• Rubis (e-auctions), Rubbos (b-board)
• Apache, Tomcat, Mysql
• Apache mod_jk redirector
• Tiers 1 and 2 are replicable
• Java client
• Average think time 1 sec
• One thread per session

19
RUBiS Response Times

• Model works well in a restricted region
• Tomcat had a connection limit of 75

20
RUBiS CPU Utilizations

• App tier is the bottleneck

21
RUBiS Processes and Conns.
22
RUBBoS Response Times

• Again, works well in a restricted region

23
RUBiS DB-intensive workload

• Replaced SELECT with SELECT SQL_NOCACHE

• Database tier is bottleneck

24
Query Caching at the Database

• Able to capture effect of query caching at DB
• Interesting to do Caching at app tier
• Reduced visit ratio at database

25
Multiple Classes of Sessions

• Class 1 App server intensive
• Class 2 Database intensive

26
Talk Outline

• Motivation
• Multi-tier Application Model
• Preliminary Evaluation
• Ongoing Work / Discussion
• Summary

27
Multiple Servers at a Tier
Mysql
Apachemod_jk
Client
Tomcat
Mysql
Apachemod_jk
Client
Tomcat
28
Load Imbalance

• Session affinity
• Variable session requirements

• Ongoing Introduce a skew factor for adjusting
  the visit ratios to the servers in the replicated
  tier

29
Applying the Model to more Apps/Implementations

• EJB based implementations of RUBiS and RUBBoS
• TPC-W
• Workloads that stress resources other than CPU
• More??

30
Misc. Issues/Discussion

• Investigate utility of the model in
• Capacity planning
• Dynamic provisioning
• Admission control
• Measurements
• How many observations to gather?
• Handling incr. svc times at higher loads
• Context switching, locks, protocol processing,
• Response time tails

31
Summary

• Network of queues based model
• Experimental evaluation for 2 apps
• Model works well in limited operating regions
• Simple enhancements to handle multiple classes,
  multiple servers, load imbalance
• More work needed on several aspects
• updating service times, how many observations,
  resp time tails, more apps and workloads

32
Acknowledgements

• Mike Spreitzer
• Asser Tantawi
• Giovanni Pacifici

Thank you!
33
Extension of Single-tier Model