Virtuoso: Resource Management and Prediction for Distributed Computing Using Virtual Machines

1
Virtuoso Resource Management and Prediction for
Distributed Computing Using Virtual Machines

• Peter Dinda
• Ashish Gupta
• Ananth I. Sundararaj
• Department of Computer Science
• Northwestern University

2
Outline

• Objective
• Background
• Proposed Idea
• Research Issues
• Implementation Status
• Demo
• Conclusions

3
Objective

• Propose a new paradigm for Grid computing
• Operating System level Virtual Machines (VM)
• Middleware mechanisms to manage VMs in a
  distributed environments
• To demonstrate the feasibility of the proposed
  idea
• Quantitative results
• Functionality perspective
• Performance perspective

4
Background

• Grid Computing
• Fundamental Goal Seamlessly multiplex the
  distributed computational resources of providers
  among users across a wide area network
• Current Approach Resources are multiplexed using
  the mechanisms found in typical operating systems

5
Proposal Raise the level of Abstraction
OS Virtual Machine
OS User
6
Virtual Machines

• History
• First came about in the 1960's on mainframes as a
  way to create less complex multi user time share
  environments
• What is it?
• A virtual machine is an abstraction of a physical
  machine
• Created using a Virtual Machine Monitor (VMM)
  running on a physical machine
• Gives the illusion of working on a separate
  machine

7
Virtual machines contd..

• Architecture
• The abstraction of a virtual machine is that each
  user appears to have a dedicated machine at their
  disposal, the hardware of which they can access
  directly

8
VMware

• VMware, Inc. was founded in 1998
• Currently the largest producer of virtual machine
  software

9
Proposed Idea

• To use virtual machines in grid computing
• Problems with traditional approach to grid
  computing ( OS User approach)
• Security
• Site dependence
• Migration of user tasks
• Virtual machines address the above mentioned
  problems

10
Proposed Idea contd..

• Other Advantages of using Virtual Machines
• Isolation
• Customization
• Legacy Support
• Administrator Privileges
• Resource Control

11
Research Issues

• Architecture
• Virtual Networking
• Performance
• Data Management
• Resource Management
• Integration with existing Grid Infrastructure

12
Architecture

• Typical Components include
• User
• Information server
• Physical server
• Image server
• Data server
• These components are distributed across the nodes
  of the grid

13
Physical Servers
Image Server
Data Server
User Front End
Information Service
14
Architecture contd...
15
Framework

• To build a virtual computational grid support
  framework
• Virtual machine instances and capability for
  instantiating virtual machines are advertised via
  a grid information service
• A user consults it to locate a physical machine
  to instantiate a virtual machine on
• The user may also consults the information
  service to locate virtual image server with a
  base OS installation or may decide to customize
  one

16
Framework Cont

• The user can now instantiate a virtual machine on
  the selected physical machine
• An operating system can then be installed using a
  virtual CD-ROM and the virtual machine can be
  customized
• The user (owner of the virtual machine) could
  negotiate terms with the physical server
• Virtual networking would put the virtual machine
  on the users network

17
Current Work

• Create an infrastructure for experimenting with
  our ideas
• Initial Part
• A menu driven application which provides a
  framework for grid computing using virtual
  machines

18
(No Transcript)
19
Perl Scripts
20
Current Goals

• Remote Instantiation
• Remote Display
• Virtual Network of Virtual Machines
• Remote CD-ROM access
• Migration of Virtual Machines

21
Remote Display

• Requirements
• A dedicated console for the remote virtual
  machine
• Remote Display options
• X-Server
• VNC
• Windows Remote Desktop
• Issues
• Interactivity
• Security

22
(No Transcript)
23
Virtual Networking

• Requirement
• A virtual network which connects the VMs
  instantiated by the user
• Should appear as a part of the user local network

24
Integration into Network

• Case I
• Physical server (VM host) is VM friendly
• VM host provides IP addresses to dynamic VM
  instances
• Case II
• VM host does not provide IP addresses to dynamic
  VM instances
• Traffic is tunneled through at the Ethernet
  level, between the remote virtual machine and the
  local network of the user
• Remote machine would appear to be connected to
  the local network
• We assume that we are able to establish a TCP
  connection to the remote site, in particular the
  remote site only allows SSH traffic

25
Issues

• Ethernet Level Tunneling and Routing
• Performance of virtual network
• Overlay network Optimization
• User and application (parallel applications)
  requirements
• Efficient Communication
• Adapt to Migration of virtual machines

26
Virtual Networking Extensions

• An overlay network could be formed among the
  remote virtual machines giving rise to a virtual
  LAN
• The overlay network could optimize itself with
  respect to the communication between the virtual
  machines
• To maintain network connectivity during and after
  migration of virtual machines

27
VMWare Networking Options

• Bridged Mode (Ist case)
• Just like any other machine on the network
• Host Only Networking (2nd case)
• Host-only networking creates a network that is
  completely contained within the host computer

28
(No Transcript)
29
Current Implementation

• Framework
• Each physical server will run a daemon called the
  bridge daemon
• Bridge daemon tunnels traffic via a ssh tunnel
  from remote network to virtual network adapter
• The daemons can communicate with each other using
  a text based protocol and reconfigure themselves
  to establish desired network configurations

30
Current Implementation

• Demo

31
Resource Management

• The use of Virtual machines create new challenges
  for scheduling and resource management
• Resource perspective
• Virtual machines provide a mechanism for
  controlling how and when resources are used
• Resource owners will be more conducive to
  allowing their resources to be used
• The user sees a raw machine completely at their
  disposal, while the resource owner only sees a
  single entity to be scheduled
• A real time schedule will be required to
  accommodate the resource owners constraints and
  user needs

32
Resource Management contd..

• Application perspective
• Applications have to adapt to the dynamically
  available resources
• Virtual machines simplify this as they can carry
  the preferred execution environment
• Virtual machines are themselves a resource and
  need to be registered and advertised

33
Data Management

• This involves
• Transfer of VM images such that they can be
  instantiated anywhere and migrated when necessary
• On demand transfer
• Seamless migration while keeping remote data
  connections active
• Support for location independent access to user
  files
• Move the computation closer to the data

34
Conclusions

• Provided an argument for the use of virtual
  machines in grid computing
• Detailed research issues involved
• Explained the current implemented framework