An%20Analytical%20Model%20for%20Multi-tier%20Internet%20Services%20and%20its%20Applications

1
An Analytical Model for Multi-tier Internet
Services and its Applications

• Bhuvan Urgaonkar, Giovanni Pacifici, Prashant
  Shenoy, Mike Spreitzer,
• Asser Tantawi
• University of Massachusetts and IBM TJ Watson

2
Internet Applications

• Proliferation of Internet applications

auction site
online game
online store

• Growing significance in personal, business
  affairs
• Focus Modeling Internet applications

3
Why Model Internet Applications?

• Capacity provisioning
• How many servers does the application need?
• Performance prediction
• E.g., predict response time
• Application configuration
• Tune various application parameters
• Request policing
• Turn away excess requests during overloads

4
Internet Application Architecture
queries
search moby
response
Melvilles Moby Dick Music CDs by Moby
HTTP
J2EE
Database
request processing in an online bookstore

• Multi-tier architecture
• Each tier uses services provided by its successor
• Session-based workloads
• Caching, replication

5
Existing Application Models

• Models for Web servers Chandra03, Doyle03
• Do not model Java server, database etc.
• Black-box models Kamra04, Ranjan02
• Unaware of bottleneck tier
• Extensions of single-tier models Welsh03
• Fail to capture interactions between tiers
• Existing models inadequate for multi-tier
  Internet applications

6
Talk Outline

• Motivation
• Application Model
• Evaluation of the Model
• Dynamic Capacity Provisioning
• Summary and Future Research

7
Baseline Application Model
clients
application

• Model consists of two components
• Sub-system to capture behavior of clients
• Sub-system to capture request processing inside
  the application

8
Modeling Clients
Z
Client 1
Z
Client 2
application
clients
Z
Client N
Q0

• Clients think between successive requests
• Infinite server system to capture think time Z
• Captures independence of Z from processing in
  application

9
Modeling Request Processing
pM1
p3
p1
p2
S1
S2
SM
Q1
Q2
QM
N
tier 1
tier 2
tier M

• Transitions defined to capture circulation of
  requests
• Request may move to next queue or previous queue
• Multiple requests are processed concurrently at
  tiers
• Processor sharing scheduling discipline
• Caching effects get captured implicitly!

10
Putting It All Together
pM1
p3
p1
p2
Z
S1
S2
SM
client
Z
client
Q1
Q2
QM
Q0
N
tier 1
tier 2
tier M

• A closed-queuing model that captures a given
  number of simultaneous sessions being served

11
Model Solution and Parameter Estimation

• Mean Value Analysis (MVA) Algorithm
• Computes mean response time
• Visit ratios
• Equivalent to trans. probs. for MVA
• Vi ?i / ?req ?req at policer, ?i from logs
• Service times
• Use residence time Xi logged at tier i
• For last tier, SM XM
• Si Xi ( Vi1 / Vi ) Xi1
• Think time
• Measured at the entry point of application

12
Talk Outline

• Motivation
• Application Model
• Evaluation of the Model
• Dynamic Capacity Provisioning
• Summary and Future Research

13
Evaluation of Baseline Model

• Auction site RUBiS
• One server per tier

Apache
JBOSS
Mysql
75
150

• Concurrency limits not captured

14
Handling Concurrency Limits
Z
S1
S2
SM
Z
Q1
Q2
QM
Q0
N
dropped requests

• Requests may be dropped due to concurrency limits
• Need to model the finiteness of queues!

15
Handling Concurrency Limits
Z
S1
S2
SM
Z
Q1
Q2
QM
Q0
N
drop
QM
Q1
drop
pM
drop
p1
drop
drop
drop
S1
SM

• Approach Subsystems to capture dropped requests
• Distinguish the processing of dropped requests

16
Response Time Prediction

• Enhanced model can capture concurrency limits

17
Query Caching at the Database

• Caching effects
• Captured by tuning Vi and/or Si

• Bulletin-board site RUBBoS
• 50 sessions
• SELECT SQL_NO_CACHE causes Mysql to not cache the
  response to a query

• More model enhancements
• Replication at tiers
• Multiple session classes

18
Prototype Data Center

• 40 Linux servers
• Gigabit switches
• Multi-tier applications
• Auction (RUBiS)
• Bulletin-board (RUBBoS)
• Apache, JBOSS (replicable)
• Mysql database

19
Dynamic Capacity Provisioning

• Auction application RUBiS
• Factor of 4 increase in 30 min

Server allocations
Workload
Response time

• Server allocations increased to match increased
  workload
• Response time kept below 2 seconds

20
Talk Outline

• Motivation
• Baseline Application Model
• Evaluation of the Model
• Dynamic Capacity Provisioning
• Summary and Future Research

21
Summary and Future Work

• Analytical model for Multi-tier Internet
  Applications
• Mean-value analysis
• Concurrency limits, replication, caching,
  multiple classes
• Model validation using 3-tier applications
• Dynamic provisioning, request policing
• Future work
• Handling load imbalances at replicated tiers
• Handling more diverse workloads
• Handling other kinds of scheduling disciplines at
  servers

22
Thank you!
More information at http//www.cs.umass.edu/
bhuvan