Realistic CPU Workloads Through Host Load Trace Playback http://www.cs.cmu.edu/~pdinda/LoadTraces

1
Realistic CPU Workloads Through Host Load Trace
Playback http//www.cs.cmu.edu/pdinda/LoadTrace
s

• Peter A. Dinda
• David R. OHallaron
• Carnegie Mellon University

2
Talk in a Nutshell

• Workloads to evaluate distributed systems
• Prediction-based systems
• Shared benchmarks
• Reconstruct time-varying CPU contention behavior
  from traces of Unix load average
• Real (non-parametric)
• Reproducible
• Comparable
• Artifacts (http//www.cs.cmu.edu/pdinda/LoadTrace
  s)
• Playload tool
• Collection of host load traces

3
Outline

• Evaluation of distributed systems
• Adaptive applications
• Prediction systems
• Host load traces
• Synthetic Vs trace-based workloads
• Host load trace playback
• Evaluation
• Conclusion

4
Evaluating Distributed Systems

• Workloads critical in evaluation
• Shared benchmarks desperately needed
• SPEC?
• Synthetic versus trace-based workloads
• Parametric versus non-parametric
• Ideally
• Real workloads
• Reproducible
• Comparable
• Sharable

5
Evaluating Prediction Systems

• Prediction systems model workload
• RPS, NWS, adaptive applications, etc.
• Synthetic workloads assume a model
• Model may be wrong or incomplete
• Easy to use in simulation or in a testbed
• Trace-based workloads assume no model
• However, traces may not be representative
• Harder to use, especially in a testbed
• How do we use traces in a testbed?
• Host load traces

6
Host Load Traces

• Periodically sampled Unix load average
• Exponentially averaged run queue length
• Measure of contention for CPU
• Complex statistical properties SciProg99
• High variability, strong autocorrelation
  function, self-similar, epochal behavior,
• Difficult to synthesize

7
Host Load and Running Time
8
Available Host Load Traces

• DEC Unix 5 second exponential average
• Full bandwidth captured (1 Hz sample rate)
• Long durations
• Available! http//www.cs.cmu.edu/pdinda/LoadTrace
  s

9
Host Load Measurement
unknown sample rate f2 Hz estimated
Trace File
Sample1
h
Sample2
exponential average, tau5 s
f1 Hz

Ready Queue
Kernel
User
10
Host Load Trace Playback
Load Measure
applied load
Load Generator
Trace File
h-1

target load
measured load
error
-
Sample1
h
Sample2
11
What are h and h-1?
time constant for recorded host
run queue length
trace file
h
applied load (recovered run queue length)
h-1
time constant for playback host
measured load
12
Load Generator
1.5 load for 1 second
Master
1.0 load for 1 second
0.0 load for 1 second
0.5 load for 1 second
...
...
Worker Processes
13
Load Generator
// Split w into n cycles while (!done) if
(uniformrand(1.0) lt p) compute for w/n
seconds else sleep for w/n seconds
p load for w seconds
(simplified)
donew seconds have elapsed Time-based
playback donewp CPU seconds have been used
Work-based playback
14
Time-based Playback
External continuous 1.0 load
External load amplitude modulates applied load
15
Work-based Playback
External continuous 1.0 load
External load amplitude and frequency modulates
applied load
16
Evaluation

• Traces described earlier
• Example uses one hour 1997 axp0 trace
• Characterization of signals and errors
• Summary stats
• Distributions
• Autocorrelation
• Multiple platforms
• Digital Unix
• Solaris
• Linux
• FreeBSD

17
Evaluation Summary Stats
18
Evaluation on Alpha/DUX
Target
Error Histogram
Measured
Error ACF
19
Evaluation on Sparc/Solaris
Target
Error Histogram
Measured
Error ACF
20
Evaluation on P2/FreeBSD
Target
Error Histogram
Measured
Error ACF
21
Evaluation on P2/Linux
Target
Error Histogram
Measured
Error ACF
22
Conclusion

• CPU Workloads from traces of Unix load average
• Reproduce contention behavior (ignoring
  priorities)
• Real, non-parametric workloads
• Reproducible
• Artifacts (http//www.cs.cmu.edu/pdinda/LoadTrace
  s)
• Playload tool
• Collection of host load traces
• Future
• Benchmarks
• Priorities, memory, disk, etc.
• In-kernel?

23
Feedback?
Use error signal to better track the load trace
24
The Problem With Feedback
External Load
Applied Load
Load Generator
Load Measure
Trace File

h-1
x
level
Effect of Combined Load
z
error
-
h-1
Feedback would try to make SUM of applied load
and external load in system track the load trace
25
Making Feedback Work
External Load
Applied Load
Load Generator
Load Measure
Trace File

h-1
x
level
Effect of Combined Load
Estimated Effect of External Load
z
error
Signal Separation
-
h-1
Estimated Effect of Applied Load
Load Source Models
26
Why Host Load Traces for Evaluating Distributed
Systems?

• Real
• Comparable and reproducible
• Analogous to a SPEC benchmark
• Usable in simulation and experimentation
• Non-parametric and non-synthetic
• Especially important for prediction systems