Title: An Analytical Model for Multi-tier Internet Services and its Applications
1An Analytical Model for Multi-tier Internet
Services and its Applications
- Bhuvan Urgaonkar, Giovanni Pacifici, Prashant
Shenoy, Mike Spreitzer, - Asser Tantawi
- University of Massachusetts and IBM TJ Watson
2Internet Applications
- Proliferation of Internet applications
auction site
online game
online store
- Growing significance in personal, business
affairs - Focus Modeling Internet applications
3Why Model Internet Applications?
- Capacity provisioning
- How many servers does the application need?
- Performance prediction
- E.g., predict response time
- Application configuration
- Tune various application parameters
- Request policing
- Turn away excess requests during overloads
4Internet Application Architecture
queries
search moby
response
Melvilles Moby Dick Music CDs by Moby
HTTP
J2EE
Database
request processing in an online bookstore
- Multi-tier architecture
- Each tier uses services provided by its successor
- Session-based workloads
- Caching, replication
5 Existing Application Models
- Models for Web servers Chandra03, Doyle03
- Do not model Java server, database etc.
- Black-box models Kamra04, Ranjan02
- Unaware of bottleneck tier
- Extensions of single-tier models Welsh03
- Fail to capture interactions between tiers
- Existing models inadequate for multi-tier
Internet applications
6Talk Outline
- Motivation
- Application Model
- Evaluation of the Model
- Dynamic Capacity Provisioning
- Summary and Future Research
7Baseline Application Model
clients
application
- Model consists of two components
- Sub-system to capture behavior of clients
- Sub-system to capture request processing inside
the application
8Modeling Clients
Z
Client 1
Z
Client 2
application
clients
Z
Client N
Q0
- Clients think between successive requests
- Infinite server system to capture think time Z
- Captures independence of Z from processing in
application
9Modeling Request Processing
pM1
p3
p1
p2
S1
S2
SM
Q1
Q2
QM
N
tier 1
tier 2
tier M
- Transitions defined to capture circulation of
requests - Request may move to next queue or previous queue
- Multiple requests are processed concurrently at
tiers - Processor sharing scheduling discipline
- Caching effects get captured implicitly!
10Putting It All Together
pM1
p3
p1
p2
Z
S1
S2
SM
client
Z
client
Q1
Q2
QM
Q0
N
tier 1
tier 2
tier M
- A closed-queuing model that captures a given
number of simultaneous sessions being served
11Model Solution and Parameter Estimation
- Mean Value Analysis (MVA) Algorithm
- Computes mean response time
- Visit ratios
- Equivalent to trans. probs. for MVA
- Vi ?i / ?req ?req at policer, ?i from logs
- Service times
- Use residence time Xi logged at tier i
- For last tier, SM XM
- Si Xi ( Vi1 / Vi ) Xi1
- Think time
- Measured at the entry point of application
12Talk Outline
- Motivation
- Application Model
- Evaluation of the Model
- Dynamic Capacity Provisioning
- Summary and Future Research
13Evaluation of Baseline Model
- Auction site RUBiS
- One server per tier
Apache
JBOSS
Mysql
75
150
- Concurrency limits not captured
14Handling Concurrency Limits
Z
S1
S2
SM
Z
Q1
Q2
QM
Q0
N
dropped requests
- Requests may be dropped due to concurrency limits
- Need to model the finiteness of queues!
15Handling Concurrency Limits
Z
S1
S2
SM
Z
Q1
Q2
QM
Q0
N
drop
QM
Q1
drop
pM
drop
p1
drop
drop
drop
S1
SM
- Approach Subsystems to capture dropped requests
- Distinguish the processing of dropped requests
16Response Time Prediction
- Enhanced model can capture concurrency limits
17Query Caching at the Database
- Caching effects
- Captured by tuning Vi and/or Si
- Bulletin-board site RUBBoS
- 50 sessions
- SELECT SQL_NO_CACHE causes Mysql to not cache the
response to a query
- More model enhancements
- Replication at tiers
- Multiple session classes
18Prototype Data Center
- 40 Linux servers
- Gigabit switches
- Multi-tier applications
- Auction (RUBiS)
- Bulletin-board (RUBBoS)
- Apache, JBOSS (replicable)
- Mysql database
19Dynamic Capacity Provisioning
- Auction application RUBiS
- Factor of 4 increase in 30 min
Server allocations
Workload
Response time
- Server allocations increased to match increased
workload - Response time kept below 2 seconds
20Talk Outline
- Motivation
- Baseline Application Model
- Evaluation of the Model
- Dynamic Capacity Provisioning
- Summary and Future Research
21Summary and Future Work
- Analytical model for Multi-tier Internet
Applications - Mean-value analysis
- Concurrency limits, replication, caching,
multiple classes - Model validation using 3-tier applications
- Dynamic provisioning, request policing
- Future work
- Handling load imbalances at replicated tiers
- Handling more diverse workloads
- Handling other kinds of scheduling disciplines at
servers
22Thank you!
More information at http//www.cs.umass.edu/
bhuvan