Virtual Machine Universe in Condor

1
Virtual MachineUniverse in Condor
2
What is VM universe?

• A job user can submit a virtual machine to Condor
  
• Condor runs the virtual machine and sends back a
  result virtual machine
• support VMware server and Xen

3
Big picture
Submit machine
Execute machine
Startd
Schedd
Starter
Shadow
VM GAHP
4
Benefits of VM universe

• platform independence
• environment independent on host machine
• checkpoint
• networking in a virtual machine
• snapshot disk
• input CDROM image

5
Snapshot disk

• All modified data will be stored into snapshot
  disks without changing original VM disk files.
• VM disk files in a shared file system can be
  safely shared among multiple jobs
• Can reduce disk space for result and checkpoint

6
Submit description file with shared file system

• universe vm
• executable WindowsXP
• vm_type vmware
• vm_memory 256
• vm_checkpoint TRUE
• vm_networking TRUE
• vm_networking_type dhcp
• vmware_dir /shared/windows_vm
• vmware_should_transfer_files FALSE
• vmware_snapshot_disk TRUE
• initialdir /result1
• Queue
• initialdir /result2
• Queue

7
Snapshot disk with shared file system
Execute machine 1
Submit machine
/result1
Execute machine 2
/result2
Shared file system
/windows_vm
8
Submit description file without shared file system

• universe vm
• executable WindowsXP
• vm_type vmware
• vm_memory 256
• vm_checkpoint TRUE
• vm_networking TRUE
• vm_networking_type dhcp
• vmware_dir /windows_vm
• vmware_should_transfer_files TRUE
• initialdir /result1
• vmware_snapshot_disk TRUE
• Queue
• initialdir /result2
• vmware_snapshot_disk FALSE
• Queue

9
Snapshot disk without shared file system
Submit machine
Execute machine 1 (Job 1)
Job 1 submit description ... vmware_snapshot_disk
TRUE Initialdir /result1 Job 2 submit
description ... vmware_snapshot_disk
FALSE Initialdir /result2
Execute machine 2 (Job 2)
/windows_vm
10
Snapshot disk without shared file system
Submit machine
Execute machine 1 (Job 1)
Job 1 /result1
Job 2 /result2
Execute machine 2 (Job 2)
/windows_vm
11
Input CDROM image

• VM universe can not use input or argument
  parameter in a job submit description file as
  other universes do
• With input CDROM images, a job user may run the
  same VM several times on different input data sets

12
Submit description file with input CDROM image

• universe vm
• executable WindowsXP
• vm_type vmware
• vm_memory 256
• vm_checkpoint TRUE
• vm_networking TRUE
• vm_networking_type dhcp
• vmware_dir /windows_vm
• vmware_should_transfer_files FALSE
• vmware_snapshot_disk TRUE
• initialdir /result1
• vmware_cdrom_files a.iso
• Queue
• initialdir /result2
• vmware_cdrom_files a.txt, b.txt
• Queue

13
Input CDROM image
Submit machine
Execute machine 1
VM
Job 1 submit description ... vmware_cdrom_files
a.iso Job 2 submit description ... vmware_cdro
m_files a.txt, b.txt
Execute machine 2
VM
14
VMware VM universe

• Snapshot disk
• Input CDROM image
• Can be used on either Linux host or Windows host

15
Xen VM universe

• No support of snapshot disk
• VM disk file in a shared file system can not be
  shared among multiple job unless it is read-only.
  
• Input CDROM image
• Can be used on only Linux host

16
Checkpoint

• Periodic checkpoint and vacate checkpoint
• All modified VM disk files and a file for VM
  memory will be transferred back to a submit
  machine
• When snapshot disks are used, snapshot disk files
  and a file for VM memory will be transferred.

17
Suspend

• Hard suspend Memory being used by a VM will be
  released and the memory will be saved into a file
• Soft suspendMemory being used by a VM will not
  be released and the VM will be just paused like
  SIGSTOP

18
Networking issues when restarting from checkpoint

• MAC and IP address for VM are also preserved when
  checkpointed
• When restarting the checkpointed VM, MAC and IP
  address dont change.
• If we use NAT for VM networking, different
  execution machines may have different MAC and IP
  address of NAT gateway.
• In VMware, if we install VMware tool inside VM,
  the tool program will automatically execute DHCP
  renew when a VM is restarted.

19
Future work

• Support snapshot disks in Xen VM universe
• For result, get only output files from VM instead
  of all VM files.
• Support another Virtual machine program (e.g.
  QEMU)

20
Summary

• We are testing VM universe.
• Hopefully VM universe will be included in Condor
  6.9.x.
• Questions?