Update on Datacenter in A Box

1
Update on Datacenter in A Box

• Zhangxi Tan and David Patterson

2
RAD Datacenter / Node
Policy Maker
Per node SW stack
Load- Balancer (IDLB)
Intrusion- Detection (IDID)
Service (IDS)
Web 2.0 Applications
Firewall (IDF)
Roby on Rails Interpreter
Web Svc APIs
Trace, X-trace
Local OS functions
Invisible Routing Layer, trace, X-trace
Virtual Machine Monitor
Actuator Commands
Sensor Data
1. Energy? 2. Run Killer App?
AWE-gen

• RAMP is the simulator for datacenter

3
Agenda

• Introduction and retrospective overview
• Improvement since June 06
• Disk and temperature emulation
• Future work

4
Introduction

• RAMP-Research Accelerator for MultiProcessing to
  emulate 1,000 CPU
• FPGA based HW emulation
• Softcore processors run real workload (OS and
  Apps)
• Scalability, reproducibility, observability,
  cost, space and power
• Repurpose RAMP to emulate datacenter
• Experiment with DC configurations processors,
  disks, networks
• Evaluate future systems before buy vs. after

5
June 06 status

• Internet in a box Version 0
• 3 Xilinx XUP board (2993) with 12 processors
• uClinux and research application (i3)
• Limitations
• Software base is poor
• No MMU, no fork, no full version of linux
• Every software need porting
• Processor is too slow (100 MHz vs 3 GHz)
• No local storage per nodes

6
Improvement
7
Agenda

• Introduction and retrospective overview
• Improvement since June 06
• Disk and temperature emulation
• Future work

8
Disk and Thermal Emulation

• Local disk is an essential part for datacenter
• Local physical storage
• Variable disk specifications (VM only have a
  function module)
• In the context of real workload
• Temperature is a critical issue in DC
• Cooling, reliability
• How the workload will affect the temperature in
  datacenter is an interesting topic

9
Methodology

• HW Emulator (FPGA) 32-bit Leon3 with, 50MHz, 90
  MHz DDR memory, 8K L1 Cache (4K Inst and 4K Data)
• Target system Linux 2.6 kernel, 50 MHz 2 GHz
• PC storage, trace logger and model solver
  (offline or online)

• Emulating IDE disk with Ethernet based network
  storage (ATA over Ethernet) DiskSim
• AoE Encapsulate IDE command in Ethernet packet
• DiskSim widely used disk simulator (provide
  access timing based on disk specification)
• Thermal emulation is done by Mercury suite
  (ASPLOS 06)
• Sample CPU/disk activities periodically and send
  to a central emulator
• Emulator takes system configuration and predict
  temperature based on Newtons laws of cooling
• Disk state will help power estimation
• Time dilation makes target looks faster
• Reprogram HW timer to make jiffies longer in
  terms of wall clock
• Slow down memory accordingly, when speeding up
  processor

10
HW specification!

• Do not believe it!

11
Experiments

• Thermal emulation model (validated in Mercury)
• Physical layout from Dell PowerEdge 2850
• 3 GHz Xeon, 10K RPM SCSI
• Emulated disk model (validated disk model in
  Disksim)
• Seagate Cheetah 9LP
• 10K RPM, 5 ms avg seek time
• Several programs run in target system with
  various time dilation factors
• Dhrystone CPU intensive benchmark
• Postmark A file system benchmark (disk
  intensive)
• Unix command with pipe (both disk and CPU
  intensive)
• cat alargefile grep a search pattern gt
  searchresultfile
• 100 MB file size
• Emulation output
• Performance statistics
• System temperature

12
Dhrystone result (w/o memory TD)
How close to a 3 GHz x86 8000 Dhrystone MIPS?
Memory, Cache, CPI
13
Dhrystone w. Memory TD
Keep the memory access latency constant - 90 MHz
DDR DRAM w. 200 ns latency in all target (50MHz
to 2GHz)- Latency is pessimistic, but reflect
the trend
14
Postmark file system benchmark

• Speed-up factor is larger than TDF (overhead)
• How close to modern SATA disk? Twice throughput
  if run the same benchmark.

15
Disk emulation performance

• Overhead analysis
• lt1.4ms sending packet (no zero-copy, VM)
• Burst of requests (service time lt 10ms, including
  Disksim), AoE protocol segmentation
• Larger TDF offset overhead
• Overall emulated disk time still a little longer
  than simulated timing in disksim (2.8 ms)

16
Emulated disk R/W time in target

• Pretty deterministic result with different TDF

17
CPU Temperature Emulation
50 MHz
250 MHz
500 MHz
1 GHz
2 GHz

• Need calibration to get correct absolute value
• Trend is accurate

18
Disk Temperature Emulation
50 MHz
250 MHz
500 MHz
1 GHz
2 GHz
19
Limitations and Future Work

• Limitations
• AoE limits the maximum number of RW sectors to 2
  !(Ethernet packet limitation)
• Naïve memory dilation (constant delay)
• Future work
• Better statistic time dilation model (CPI,
  distribution), still simple HW
• Emulate real-life disk controller (e.g. Intel
  ICH) less overhead