Quality of Service Guarantee for ClusterBased Internet Service

1
Quality of Service Guarantee for Cluster-Based
Internet Service

• Chang Li, Gang Peng, Kartik Gopalan, Tzi-cker
  Chiueh
• Computer Science Department
• State University of New York at Stony Brook
• Email changli, gpeng, kartik,
  chiueh_at_cs.sunysb.edu

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
CNN
Ebay
6
(No Transcript)
7
Goals of our work

• A generic framework for building cluster-based
  internet service
• service independent, platform independent
• QoS guarantee for performance isolation among
  subscribers
• Performance optimization
• load balancing,

8
Challenges of the work

• Request scheduling
• Resource usage accounting
• Service specific logic

9
Challenges of the work

• Request scheduling
• Resource usage accounting
• Service specific logic
• Decouple these functionalities

10
Challenges of the work

• Request scheduling
• Resource usage accounting
• Service specific logic
• Decouple these functionalities
• Unintrusive to server operating system

11
The architecture
Dispatcher
client
Internet
Router
Switch
12
Serve a request
Dispatcher
client
Internet
Router
Switch

• client connects to Server

13
Serve a request
Dispatcher
URL Request
client
Internet
Router
Switch
14
Serve a request
Dispatcher
URL Request
client
Internet
Router
Switch
15
Serve a request
Dispatcher
URL Request
client
Internet
Router
Switch
16
Serve a request
Dispatcher
URL Request
client
Internet
Router
Switch
17
Serve a request
Dispatcher
URL Response
client
Internet
Router
Switch
18
Serve a request
Dispatcher
client
Internet
Router
Switch
19
Serve a request
Dispatcher
client
Router
Switch
20
A dilemma of the architecture

• To associate a server node with an incoming
  request, we need to examine the request content
• To receive the packet that contains the request
  content, we need to identify a proper server node
  that can set up a TCP connection with the
  requesting client

21
A dilemma of the architecture

• To associate a server node with an incoming
  request, we need to examine the request content
• To receive the packet that contains the request
  content, we need to identify a proper server node
  that can set up a TCP connection with the
  requesting client
• Solution
• Decoupling connection establishment from
  processing a request

22
A challenge of the architecture

• Make response packets bypass the dispatcher

23
A challenge of the architecture

• Make response packets bypass the dispatcher
• Solution
• Separate the incoming traffic of a TCP connection
  from the outgoing traffic

24
TCP splicing
Dispatcher
client
Router
25
TCP splicing
Dispatcher
client
Router
26
TCP splicing
Dispatcher
client
Router
27
TCP splicing
Dispatcher
client
Router
28
A distributed TCP splicing
Dispatcher
client
Router
29
Software module of TCP splicing
30
An example of distributed TCP splicing
Dispatcher
IP address D
client
IP address S
Router
IP address C
Cluster address D
31
Connection Establishment
Dispatcher
IP address D
SYN, 5
client
IP address S
Router
IP address C
32
Connection Establishment
Dispatcher
IP address D
SYN, 5
client
IP address S
Router
IP address C
33
Connection Establishment
Dispatcher
SYNACK, 10
IP address D
client
IP address S
Router
IP address C
34
Connection Establishment
Dispatcher
IP address D
SYNACK, 10
client
IP address S
Router
IP address C
35
Connection Establishment
Dispatcher
IP address D
ACK, URL, 6
client
IP address S
Router
IP address C
36
Connection Establishment
Dispatcher
IP address D
ACK, URL, 6
client
IP address S
Router
IP address C
37
Connection Establishment
Dispatcher
IP address D
client
IP address S
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt

38
Connection Establishment
Dispatcher
SYN, 5
IP address D
client
IP address S
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt

39
Connection Establishment
Dispatcher
SYNACK, 20
IP address D
client
IP address S
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt

40
Connection Establishment
Dispatcher
ACK, URL, 6
IP address D
client
IP address S
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt

41
Connection Establishment
Dispatcher
IP address D
client
IP address S
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt
• Second TCP lap ltD, S, src_seq 5, dst_seq 20gt

42
Data Transmission
Dispatcher
IP address D
S, D, 21
client
Router
IP address S
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt
• Second TCP lap ltD, S, src_seq 5, dst_seq 20gt

43
Data Transmission
Dispatcher
IP address D
IP address S
client
Router
IP address C
D, C, 11
S, D, 21

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt
• Second TCP lap ltD, S, src_seq 5, dst_seq 20gt

44
Data Transmission
Dispatcher
IP address D
IP address S
client
D, C, 11
Router
IP address C

• First TCP lap ltC, D, src_seq 5, dst_seq 10gt
• Second TCP lap ltD, S, src_seq 5, dst_seq 20gt

45
Overhead study of TCP splicing
46
Request scheduling

• Server node selection
• Request selection

47
Request scheduling

• Server node selection
• Various optimizations
• Request selection
• 
  

48
Request scheduling

• Server node selection
• Various optimizations
• Request selection
• Weighted round robin (WRR)
• 
  

49
Request scheduling

• Server node selection
• Various optimizations
• Request selection
• Weighted round robin (WRR)
• No idea about the resource a request will consume
  on dispatching it

50
Request scheduling

• Server node selection
• Various optimizations
• Request selection
• Weighted round robin (WRR)
• No idea about the resource a request will consume
  on dispatching it
• Predict per-request resource usage using history

51
Request scheduling

• Server node selection
• Various optimizations
• Request selection
• Weighted round robin (WRR)
• No idea about the resource a request will consume
  on dispatching it
• Predict per-request resource usage using history
• Feedback to correct the prediction

52
Performance deviation from ideal reservation
53
Resource usage accounting

• What to account
• Accounting granularity

54
Resource usage accounting

• What to account
• CPU, disk and network bandwidth
• Accounting granularity

55
Resource usage accounting

• What to account
• CPU, disk and network bandwidth
• And ideally more
• Accounting granularity

56
Resource usage accounting

• What to account
• CPU, disk and network bandwidth
• And ideally more
• Accounting granularity
• Per-request

57
Resource usage accounting

• What to account
• CPU, disk and network bandwidth
• And ideally more
• Accounting granularity
• Per-request
• Per-server

58
Resource usage accounting

• What to account
• CPU, disk and network bandwidth
• And ideally more
• Accounting granularity
• Per-request
• Per-server
• Per process-set

59
Service-specific components

• Request classification
• Service specific optimization

60
Service-specific components

• Request classification
• Hostname, URL,
• Service specific optimization

61
Service-specific components

• Request classification
• Hostname, URL,
• Service specific optimization
• Content-aware request dispatching to leverage
  caching effect

62
Test bed set-up

• One primary Dispatcher 450 MHz Pentium-III, 64
  Mbytes memory and 2 100Mbits NIC
• Eight Server nodes 600 MHz Celeron, 64 Mbytes
  memory and 2 100Mbits NIC
• One 24-port Fast Ethernet switch
• Workloads synthetic and extracted from SPECWeb99
• Measurement unit Generic Request (10 ms
  CPU/disk, 2 Kbytes bandwidth)

63
Performance Isolation
64
Scalability study
65
Related work

• QoS over different types of resources
• CPU, network, disk,
• TCP splicing, TCP migration
• Cluster reserve
• Specific to web service virtualization
• And more

66
Concluding remarks

• A generic framework providing QoS guarantee to
  cluster-based internet service
• Only need simple OS support
• Introduce a distributed TCP splicing technique
• The experiment results show our system does
  guarantee QoS with acceptable deviation and
  overhead incurred is limited

67
Future work

• Exploit this scheme on different Internet
  services
• Video streaming,
• Multi-tier cluster system
• Web server, app server, db server, storage
  server,
• Resource provisioning over different resources
• Memory, energy,

68
Performance deviation from ideal reservation
69
Cluster-based internet service

• Computation outsourcing becomes popular for many
  reasons
• Several applications or subscribers are sharing a
  cluster of servers
• There services have similar functional components
• Most requests are read-only or read-most

70
Cluster-based internet service
71
Outline of the presentation

• The goals of our work
• The architecture of our proposal
• Gage a Web hosting service implementation
• Performance evaluation
• Related work
• Our contribution

72
A distributed TCP splicing