High Performance VMMBypass IO in Virtual Machines - PowerPoint PPT Presentation
1 / 22
Title:
High Performance VMMBypass IO in Virtual Machines
Description:
VMs and VMM. Introduced in 1960s. Key component: Virtual Machine Monitor (Hypervisor) Allows running different VMs on a single physical machine ... – PowerPoint PPT presentation
Number of Views:48
Avg rating:3.0/5.0
Title: High Performance VMMBypass IO in Virtual Machines
1
High Performance VMM-Bypass I/O in Virtual
Machines
- Jiuxing Liu, Wei Huang, Bulent Abali,
Dhabaleshwar Panda - High Performance Communication (CS8803)
- Paper Presentation
- Adit Ranadive
2
VMs and VMM
- Introduced in 1960s
- Key component Virtual Machine Monitor
(Hypervisor) - Allows running different VMs on a single physical
machine - Examples VMWare ESX Server, Denali, Xen
3
Xen Virtual Memory Monitor
- Developed by the Systems Research Group of the
University of Cambridge Computer Lab
4
Xen Split Driver Architecture
5
High Speed Interconnects
- Very low latencies (lt5µs)
- Examples Myrinets, Quadrics, Infiniband, VIA
- Communication protocol offloaded to NIC
- Provide OS bypass I/O
- Low CPU overhead, reduced latency
- Benefits to VMs
6
Proposed Idea
- Based on OS bypass I/O design of interconnects
(Infiniband) - Overcome problems of Infiniband in virtualized
environments - High latencies
- Higher CPU overhead
- I/O accesses
- VMM is bottleneck
7
Infiniband Architecture
- Host Channel Adapters -gt NICS
- Queue-based model for communication
- Queue Pair (QP), Work Queue Request (WQR),
Completion Queues (CQ), Completion Queue Entries
(CQE) - Notifications using doorbells
- Management Datagrams (MADs)
- Buffer management TPT, Pinning
8
Infiniband Interface
9
VMM Bypass I/O
- Problem of Device virtualization
- Para-virtualizing devices
10
VMM Bypass I/O (contd)
- I/O handled by another VM
11
Prototype Design
Xen-IB driver with split driver model
12
Prototype Design
- Need direct access to HCA from guest VM
- Allowing Infiniband Privileged Accesses
- Initialization and resource management Opening
HCA, Creating QP, Creating CQ - Memory registration Memory regions to be used
- Event handling Communication between CQ/QP and
guest VM
13
VMM-Bypass Accesses
- QP Accesses
- Completion of communication
- CQ Polling
- Both in critical path
- Requirements
- UAR pages available to guest domain
- QP and CQ buffers visible to guest domain
14
VMM-Bypass Accesses
- UAR Mapping implemented by front-end driver
- Loading of front-end driver
- Start of user application
- 2 stage model for QP/CQ buffers
- Registration through IDD
- Frontend sends CQ/QP creation commands to IDD
- Virtualizing Infiniband Management Operation
15
VMM-Bypass Accesses
Design of IB driver
16
Performance Evaluation
- Setup
- Infiniband Cluster
- Dual Intel Xeon 3.0 Ghz CPUs, 2 GB Memory
- Mellanox MT23108 PCI-X IB HCA
- Infiniband Latency and Bandwidth
- Event/Interrupt Handling Overhead
- Memory Registration
- IPoIB Performance
- MPI Performance
17
Infiniband Latency
Infiniband Send/Receive Latency
Infiniband RDMA Write Latency
18
Infiniband Bandwidth
Infiniband RDMA Write Bandwidth
Infiniband Send/Receive Bandwidth
19
Event/Interrupt Handling Overhead
Inter-domain Communication One way Latency
Send/Receive Latency using Blocking VERBS API
20
Memory Registration and IPoIB Performance
Memory Registration Time
IPoIB Netperf Throughput
21
MPI Performance
MPI Latency
MPI Bandwidth
MPI NAS Benchmark
22
Limitations
- Safe Device Access
- All privileged accesses through Backend
- UAR pages contain virtual access points
- Limit size of buffers and allow frequent reuse
- Performance Isolation
- QoS among different VMs
- VM Check-pointing Migration
- Insufficient knowledge in VMM
- VM state stored on I/O device
Recommended
CrystalGraphics Presentations
