Providing QoS with Virtual Private Machines

1
Providing QoS with Virtual Private Machines

• Kyle J. Nesbit, James Laudon, and James E. Smith

2
Motivation for QoS

• Multithreaded Chips
• Resource sharing
• Higher utilization
• E.g., Niagara
• Inter-thread interference
• Applications
• Soft real-time applications
• Cell-phones and game consoles
• Fine-grain parallel applications
• Scheduling and synchronization
• Server consolidation
• Hosting services

3
QoS Objectives

• Isolation
• Priority
• Fairness
• Performance
• Objectives are combined
• E.g., Isolation and performance

4
QoS Framework

• Separation of Objectives, Policies, Mechanisms
• Well structured solutions

Objectives
Proc. 1
Proc. 4
Proc. 2
Proc. 3
L1 Cache
L1 Cache
L1 Cache
L1 Cache
Interconnect
Mechanisms
Global Policy
Local Policy
Bandwidth K
Local Policy
L2 Cache (Capacity C)
Memory Controller
Local Policy
Bandwidth L
Main Memory (Capacity M)
5
Local PolicyResource-Directed
Service
Isolation Allocations Minimum Service
Unallocated or Unused
Work Conserving Policies
Allocated Consumed
Priority Vector Relative Service
Same Priority Unused
Higher Priority Consumed
Fairness Multiple Definitions?
6
Global Policy Virtual Private Machines
Real-Time Thread
Background Thread
Background Thread
Background Thread
Fairness Policy
VPM 1
VPM 2
VPM 3
VPM 4
Proc. 4
Proc. 1
Proc. 2
Proc. 3
L1 Cache
L1 Cache
L1 Cache
L1 Cache
BW .1K
BW .1K
BW .5K
BW .1K
L2 Cache (Capacity .5C)
L2 Cache (Capacity .1C)
L2 Cache (Capacity .1C)
L2 Cache (Capacity .1C)
Memory Cntl.
Memory Cntl.
Memory Cntl.
Memory Cntl.
BW .5L
BW .1L
BW .1L
BW .1L
Main Memory
Main Memory
Main Memory
Main Memory
Priority 0
Priority 0
Priority 0
Priority 3
7
Global PolicyPerformance-Directed

• Global optimization problem
• Use local policies to control resources
• Optimize one bottleneck and the bottleneck
  appears somewhere else
• Performance-directed policies need to fit into
  the VPM policy
• E.g., optimize aggregate performance within a
  priority level

8
Status

• Completed
• Secondary cache ISCA 07 and SDRAM memory
  system mechanisms Micro 06
• Bandwidth mechanisms
• Cache capacity mechanisms
• Ongoing and Future Work
• Multithreaded Processors
• Work conserving cache capacity policy
• Priority policy
• Aggregate performance policy

9
Conclusion

• Objectives Isolation, Priority, Fairness,
  Performance
• Implementation Separation of policies and
  mechanisms
• Abstraction Virtual Private Machines
• Composable global policies that coexist on a per
  application basis

10
Questions and Comments

• Do Virtual Private Machines meet all of the
  requirements of software controlled
  micro-architecture resource management?