Resource Overbooking and Application Profiling in Shared Hosting Platforms

1
Resource Overbooking and Application Profiling in
Shared Hosting Platforms

• Bhuvan Urgaonkar
• Prashant Shenoy
• Timothy Roscoe
• UMASS Amherst and Intel Research
• Fifth USENIX OSDI, Boston, Dec 2002

2
Introduction
cluster
E-commerce
Streaming
Clients
Games

• Proliferation of Internet applications
• E-commerce, streaming media, online games,
• Commonly hosted on clusters of servers
• Cheaper alternative to large multiprocessors

3
Hosting Platforms

• Hosting platform server cluster that runs
  third-party applications
• Applications pay for server resources
• CPU, network bandwidth, memory, disk
• Platform provider guarantees resource
  availability
• Challenge Maximize hosted applications while
  providing resource guarantees

4
Design Challenges

• How to determine an applications resource needs?
• How to provision resources to meet these needs?
• How to map applications to servers in the
  platform?
• How to handle dynamic variations in load?

5
Talk Outline

• Introduction
• Inferring Resource Requirements
• Provisioning Resources
• Mapping Applications to Servers
• Experimental Evaluation
• Related Work

6
Terminology

• Hosting platform models
• Dedicated Applications get integral nodes
• Shared Applications may get fractional nodes

Applications
Platform nodes

• Capsule component of an application running on a
  node

7
Provisioning By Overbooking

• Worst-case provisioning is wasteful
• Low utilization of resources
• Applications may be tolerant to occasional
  violations
• E.g., CPU guarantees should be met 99 of the
  time
• Possible to provide useful guarantees even after
  provisioning less than worst-case needs
• Overbook resources to improve utilization
• E.g., Airline reservations

8
Application Profiling

• Profiling process of determining resource usage
• Run the application on an isolated set of nodes
• Subject the application to a real workload
• Model CPU and network usage as ON-OFF processes

Begin CPU quantum
End CPU quantum
time
ON
OFF

• Use the Linux Trace Toolkit Yaghmour00

9
Resource Usage Distribution
ON-OFF PROCESS
Measurement Interval
time
CDF
PDF
1
0.99
Cumulative Probability
A
B
0
1
Fractional usage
Fractional usage
10
Profiles of Server Applications
Postgres Server, 10 clients
0.1
0.08
0.06
Probability
0.04
0.02
0
0
0.2
0.4
0.6
0.8
1
Fraction of CPU

• Applications exhibit different degrees of
  burstiness
• Need to capture variability in resource usage

11
Capturing Burstiness Token Bucket

• Token Bucket (s, ?)
• Resource usage over t s.t ?

s1.t ?1
s2.t ?2
usage
?2
time
?1
ON-OFF PROCESS
time

• Choose (s, ?) based on a high percentile

12
Resource Overbooking Mechanism

• Applications specify overbooking tolerance Oi
• Probability with which capsule needs may be
  violated
• Controlled overbooking via admission control
• Resource requirements of all capsules are met
• SK (sk Tmin ?k)(1 - Ok)
  CTmin
• Overbooking tolerances of all capsules are met
• Pr (SKUk gt C) min (O1,,Ok)
• A node that has sufficient resources for a
  capsule is feasible for it

13
Mapping Capsules to Nodes
1
1
1
1
2
2
2
Final Mapping
3
3
3
3
4
4
capsules
capsules
nodes
nodes

• A bipartite graphs of capsules and feasible nodes
• Greedy mapping consider capsules in
  non-decreasing order of degrees
• Multiple feasible nodes gt random, best fit,
  worst fit

14
Talk Outline

• Introduction
• Inferring Resource Requirements
• Provisioning Resources
• Mapping Applications to Servers
• Experimental Evaluation
• Related Work

15
The SHARC Prototype

• A Linux-based Shared Hosting Platform
• 6 Dell Poweredge 1550 servers
• Gigabit Ethernet link
• Software Components
• Profiling
• Vanilla Linux Linux Trace Toolkit
• Control plane
• Overbooking, placement
• QoS-enhanced Linux kernel
• HSFQ schedulers

16
Experimental Setup

• Prototype running on a 5 node cluster
• Each server 1 GHz PIII with 512MB RAM and
  Gigabit ethernet
• Control plane runs on a dedicated node
• Applications run on the other four nodes
• Workload mix of server applications
• Apache web server with SPECWeb99 (static
  dynamic HTTP)
• PostgreSQL database server with pgbench (TPC-B)
  benchmark
• MPEG streaming server with 1.5 Mb/s VBR MPEG-1
  clients
• Quake I game server with terminator bots

17
Resource Overbooking Benefits
Placement of Apache Web Servers
1400
No Ovb
Ovb1
1200
Ovb5
1000
800
Web Servers Placed
600
400
0
20
40
60
80
100
120
140
Number of Nodes

• Small amounts of overbooking can yield large
  gains
• Bursty applications yield larger benefits

18
Performance with Overbooking
Performance of Apache
Performance of Postgres
70
25
60
20
50
15
40
Throughput (trans/s)
Throughput (req/s)
30
10
20
5
10
0
0
Isolated
100th
99th
95th
Average
Isolated
100th
99th
95th
Average
CPU Provisioning
CPU Provisioning

• Performance degradation is within specified
  overbooking tolerance

19
Handling Flash Crowds

• Detect overloads by online profiling

• Reacting to overloads (ongoing work)
• Compute new allocations
• Change allocations, move capsules, add servers

20
Related Work

• Single node resource management
• Proportional share schedulers WFQ, SFQ, BVT,
• Reservation based schedulers Nemesis, Rialto,
• Cluster-based resource management
• Cluster Reserves Aron00
• MUSE Chase01 economic approach
• Oceano IBM, Planetary computing HP
• Clusters for high availability Porcupine
  Saito99
• Grid computing Globus

21
Concluding Remarks

• Resource management in shared hosting platforms
• Application profiling to determine resource usage
• Controlled overbooking to improve utilization
• Mapping applications to servers
• Future work
• Handling dynamic workloads
• Managing memory and disk bandwidth
• URL http//lass.cs.umass.edu