Increasing%20Application%20Performance%20In%20Virtual%20Environments%20Through%20Run-time%20Inference%20and%20Adaptation

1
Increasing Application Performance In Virtual
Environments Through Run-time Inference and
Adaptation

• Ananth I. Sundararaj
• Ashish Gupta
• Peter A. Dinda
• Prescience Lab
• Department of Computer Science
• Northwestern University
• http//virtuoso.cs.northwestern.edu

2
Summary

• Dynamically adapt existing, unmodified
  applications running on unmodified operating
  systems in virtual environments to available
  resources
• Adaptation mechanisms are application independent
  and controlled automatically without user or
  developer help
• Demonstrate feasibility of adaptation at the
  level of collection of VMs connected by Virtual
  Networks
• Show that its benefits can be significant for two
  classes of applications

3
Outline

• Virtual machine grid computing
• Virtuoso system
• Networking challenges in Virtuoso
• Enter VNET
• VNET, VTTIF Adaptive virtual network
• Evaluation
• Summary

4
Virtual Machine Grid Computing
Aim
Deliver arbitrary amounts of computational power
to perform distributed and parallel computations
1
New Paradigm
Traditional Paradigm
5
2
Grid Computing using virtual machines
Resource multiplexing using OS level mechanism
Grid Computing
4
3a
6a
3b
Problem1
6b
Virtual Machines What are they?
Complexity from resource users perspective
Solution
Problem2
How to leverage them?
Complexity from resource owners perspective
5
Virtual Machines
Virtual machine monitors (VMMs)

• Raw machine is the abstraction
• VM represented by a single
• image
• VMware GSX Server

6
The Simplified Virtuoso Model
Virtual networking ties the machine back to
users home network
Users LAN
Specific hardware and performance
VM
Basic software installation available
Orders a raw machine
Virtuoso continuously monitors and adapts
User
7
Users View in Virtuoso Model
Users LAN
VM
User
8
Outline

• Virtual machine grid computing
• Virtuoso system
• Networking challenges in Virtuoso
• Enter VNET
• VNET, VTTIF Adaptive virtual network
• Evaluation
• Summary

9
Virtual Networks
VM traffic going out on foreign LAN
Foreign hostile LAN
X
Users friendly LAN
IP network
Virtual Machine
Host

• A machine is suddenly plugged into a foreign
  network. What happens?
• Does it get an IP address?
• Is it a routeable address?
• Does firewall let its traffic
• through? To any port?

Proxy
VNET A bridge with long wires
10
A VNET Link
Ethernet Packet Captured by Interface in
Promiscuous mode
First link
Second link (to proxy)
Host Only Network
VM 1
eth0
ethz
ethy
vmnet0
vmnet0
IP Network
VNET
VNET
Ethernet Packet Tunneled over TCP/SSL Connection
Host
Host
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Local traffic matrix inferred by VTTIF
Periodically sent to the VNET on the Proxy
11
Virtual Topology and Traffic Inference Framework
(VTTIF) Operation
Ethernet-level traffic monitoring
VNET daemons collectively aggregate a global
traffic matrix for all VMs

• Application topology is recovered using
• normalization and pruning algorithms

12
Dynamic Topology Inference by VTTIF
VNET Daemons on Hosts
Aggregated Traffic Matrix
VNET Daemon at Proxy
2. Low Pass Filter Aggregation
Smoothed Traffic Matrix
1. Fast updates
3. Threshold change detection
Topology change output
13
Outline

• Virtual machine grid computing
• Virtuoso system
• Networking challenges in Virtuoso
• Enter VNET
• VNET, VTTIF Adaptive virtual network
• Evaluation
• Summary

14
Adaptation
Applications

1. BSP
2. Transactional ecommerce

Application performance measure

1. Application throughput

1. VTTIF
2. Network monitoring

Monitoring and inference

1. Single hop
2. Worst fit

Optimization metric

1. Single metric
2. Combined metric

Adaptation algorithm

1. Overlay topology
2. Forwarding rules
3. VM migration

Adaptation mechanisms
15
Optimization Problem (1/2) Topology Only

• Informally stated
• Input
• Network traffic load matrix of application
• Output
• Overlay topology connecting hosts
• Forwarding rules on the topology
• Such that the application throughput is maximized

The algorithm is described in detail in the paper
16
Illustration of Topology Adaptation in Virtuoso
Fast-path links amongst the VNETs hosting VMs
Resilient Star Backbone
Foreign host LAN 1
Users LAN
VM 1
Host 1 VNET
IP network
Proxy VNET
Merged matrix as inferred by VTTIF
Foreign host LAN 2
VM 2
VM 4
VM 3
Host 3 VNET
Host 2 VNET
Host 4 VNET
Foreign host LAN 4
Foreign host LAN 3
17
Evaluation

• Reaction time of VNET
• Patterns A synthetic BSP benchmark
• Benefits of adaptation (performance speedup)
• Eight VMs on a single cluster, all-all topology
• Eight VMs spread over WAN, all-all topology

Wide-Area testbed
DOT Network
CMU
Northwestern
VM 6
VM 8
Proxy
University of Chicago
VM 1
VM 5
VM 7
18
Reaction Time
19
Benefits of Adaptation
Benefits accrued as a function of the number of
fast-path links added

• Patterns has an all-all topology
• Eight VMs are used
• All VMs are hosted on the same cluster

20
Benefits of Adaptation
Benefits accrued as a function of the number of
fast-path links added

• Patterns has an all-all topology
• Eight VMs are used
• VMs are spread over WAN

21
Optimization Problem (2/2) Topology Migration

• Informally stated
• Input
• Network traffic load matrix of application
• Topology of the network
• Output
• Mapping of VMs to hosts
• Overlay topology connecting hosts
• Forwarding rules on the topology
• Such that the application throughput is maximized

The algorithm is described in detail in the paper
22
Evaluation

• Applications
• Patterns A synthetic BSP benchmark
• TPC-W Transactional web ecommerce
  benchmark
• Benefits of adaptation (performance speedup)
• Adapting to compute/communicate ratio
• Adapting to external load imbalance

23
Effect on BSP Application Throughput of Adapting
to Compute/Communicate Ratio
24
Effect on BSP Application Throughput of Adapting
to External Load Imbalance
25
TPCW Throughput (WIPS) With Image Server Facing
External Load
No Topology Topology
No Migration 1.216 1.76
Migration 1.4 2.52
26
Outline

• Virtual machine grid computing
• Virtuoso system
• Networking challenges in Virtuoso
• Enter VNET
• VNET, VTTIF Adaptive virtual network
• Evaluation
• Summary

27
Summary

• Dynamically adapt existing, unmodified
  applications running on unmodified operating
  systems in virtual environments to available
  resources
• Adaptation mechanisms are application independent
  and controlled automatically without user or
  developer help
• Demonstrate feasibility of adaptation at the
  level of collection of VMs connected by Virtual
  Networks
• Show that its benefits can be significant for two
  classes of applications

28
For More Information

• Future Work
• Free network measurement (Wren) Collaboration
  with CS, WM
• Applicability of a single optimization scheme
• Related Talk at HPDC 2005
• J. Lange, A. Sundararaj, P. Dinda, Automatic
  Dynamic Run-time Optical Network Reservations
• Wednesday, July 27, 200 P.M.
• Please visit
• Prescience Lab (Northwestern University)
• http//plab.cs.northwestern.edu
• Virtuoso Resource Management and Prediction for
  Distributed Computing using Virtual Machines
• http//virtuoso.cs.northwestern.edu
• VNET is publicly available from above URL

29
Backup slides start from here
30
Isnt It Going to Be Too Slow?
Application Resource ExecTime (103 s) Overhead
SpecHPC Seismic (serial, medium) Physical 16.4 N/A
SpecHPC Seismic (serial, medium) VM, local 16.6 1.2
SpecHPC Seismic (serial, medium) VM, Grid virtual FS 16.8 2.0
SpecHPC Climate (serial, medium) Physical 9.31 N/A
SpecHPC Climate (serial, medium) VM, local 9.68 4.0
SpecHPC Climate (serial, medium) VM, Grid virtual FS 9.70 4.2
Small relative virtualization overhead compute-in
tensive
Relative overheads lt 5
Experimental setup physical dual Pentium III
933MHz, 512MB memory, RedHat 7.1, 30GB disk
virtual Vmware Workstation 3.0a, 128MB memory,
2GB virtual disk, RedHat 2.0 NFS-based grid
virtual file system between UFL (client) and NWU
(server)
31
Isnt It Going To Be Too Slow?
Synthetic benchmark exponentially arrivals of
compute bound tasks, background load provided by
playback of traces from PSC Relative overheads lt
10
32
Isnt It Going To Be Too Slow?

• Virtualized NICs have very similar bandwidth,
  slightly higher latencies
• J. Sugerman, G. Venkitachalam, B-H Lim,
  Virtualizing I/O Devices on VMware Workstations
  Hosted Virtual Machine Monitor, USENIX 2001
• Disk-intensive workloads (kernel build, web
  service) 30 slowdown
• S. King, G. Dunlap, P. Chen, OS support for
  Virtual Machines, USENIX 2003
• However May not scale with faster NIC or disk

33
Why VNET? A Scenario
Foreign hostile LAN
Users friendly LAN
IP network
User has just bought
Virtual Machine
34
Why VNET? A Scenario
VM traffic going out on foreign LAN
Foreign hostile LAN
X
Users friendly LAN
IP network
Virtual Machine
Host

• A machine is suddenly plugged into a foreign
  network. What happens?
• Does it get an IP address?
• Is it a routeable address?
• Does firewall let its traffic
• through? To any port?

Proxy
VNET A bridge with long wires
35
A VNET Link
Ethernet Packet Captured by Interface in
Promiscuous mode
First link
Second link (to proxy)
Host Only Network
VM
eth0
ethz
ethy
vmnet0
vmnet0
IP Network
VNET
VNET
Ethernet Packet Tunneled over TCP/SSL Connection
Host
Host
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Local traffic matrix inferred by VTTIF
Periodically sent to the VNET on the Proxy
36
A VNET Link
Ethernet Packet Captured by Interface in
Promiscuous mode
First link
Second link (to proxy)
Host Only Network
VM 1
eth0
ethz
ethy
vmnet0
vmnet0
IP Network
VNET
VNET
Ethernet Packet Tunneled over TCP/SSL Connection
Host
Host
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Ethernet Packet is Matched against the Forwarding
Table on that VNET
Local traffic matrix inferred by VTTIF
Periodically sent to the VNET on the Proxy
37
VNET startup topology
Foreign LAN 1
TCP Connections
Users LAN
VM 1
Host 1 VNET
IP network
Proxy VNET
Foreign LAN 2
Host 3 VNET
VM 2
VM 4
Host 2 VNET
Host 4 VNET
VM 3
Foreign LAN 3
Foreign LAN 4
38
VTTIF

• Traffic characterization and topology inference
  for applications
• Ethernet-level traffic monitoring
• VNET daemons collectively aggregate a global
  traffic matrix for all VMs
• Application topology is recovered using
  normalization and pruning algorithms

39
VTTIF Operation
Synced Parallel Traffic Monitoring
Traffic Filtering and Matrix Generation
Matrix Analysis and Topology Characterization
40
Reaction time of VTTIF
41
Benefits of Adaptation
Benefits accrued as a function of the number of
fast-path links added

• Patterns has an all-all topology
• Eight VMs are used
• All VMs are hosted on the same cluster

42
Benefits of Adaptation
Benefits accrued as a function of the number of
fast-path links added

• Patterns has an all-all topology
• Eight VMs are used
• VMs are spread over WAN

43
(No Transcript)
44
(No Transcript)
45
Adaptation Algorithms

• Topology adaptation
• Implied traffic intensity between VNET daemons
• Links established in order of decreasing traffic
  intensity
• Cost constraint c
• Migration
• A worst-fit algorithm
• Combining algorithms
• Migration algorithm is run first
• The overlay topology is next determined
• Finally the forwarding rules are computed

46
Present and Future

• Demonstrated the feasibility of adaptation at the
  level of collection of VMs connected by VNET
• Showed that its benefits can be significant for
  two classes of applications
• Studying the computational complexity of the
  generic incarnation of adaptation problem
• Exploring the applicability of a single
  optimization scheme for a wide-range of
  distributed applications

47
Summary

• Dynamically adapt applications in virtual
  environments to available resources
• Demonstrate the feasibility of adaptation at the
  level of collection of VMs connected by Virtual
  Networks
• Show that its benefits can be significant for two
  classes of applications
• Exploring the applicability of a single
  adaptation scheme for a wide-range of distributed
  applications