Recap

1
Recap
2
Measuring Performance

• A computer user response time (execution time).
• A computer center manager - throughput - the
  total amount of work done in a period of time.
• CPU time a very good and fair measure of
  performance.
• CPU time can also be divided into user CPU time
  (program) and system CPU time (OS).

3
Aspects of CPU Execution Time
4
Factors Affecting CPU Performance
Instruction Count I
CPI
Clock Cycle C
Program
X
X
Compiler
X
X
Instruction Set Architecture (ISA)
X
X
X
X
Organization
X
Technology
5
Example tradeoff between C and CPI

• Assume stores can execute in 1 cycle by slowing
  clock 15
• Should this be implemented?

6
Simple Example

• Old CPI 0.43 x 1 0.21 x 1 0.12 x 2 0.24 x
  2 1.36
• New CPI 0.43 x 1 0.21 x 1 0.12 x 1 0.24 x
  2 1.24
• Speedup old time/new time
• I x old CPI x C/I x new CPI x 1.15 C
• 1.36 / (1.24 x 1.15) 0.95
• Answer Dont make the change

7
Some Caveats

• Inter-dependence of I, CPI, and C Improvement In
  One May Impact Another
• increasing pipeline depth tends to increase clock
  speed but may increase CPI
• Change In ISA To Reduce Instruction Count May
  Require a Design With Slower Clock gt May Not
  Improve Performance
• CPI Depends on Instruction Mix gt Smaller
  Instruction Count May Not Improve Performance

8
Code Size Performance
No Correlation!
Hardware Independent Metrics Do Not Predict
Performance
9
Benchmarks and Benchmarking

• In lack of a universal task pick some programs
  that represent common tasks
• Use representative programs to compare
  performance of systems
• CAUTIONS
• Comparisons are as good as the benchmarks are in
  representing your real workload.
• Many parameters affect measured performance

10
Example We must use the same compiler

• Compiler enhancements and performance

?1998 Morgan Kaufmann Publishers
11
Benchmark Suites

• A Suite Is a Collection of Representative
  Benchmarks From Different Application Domains
• Weakness of Any One Benchmark Likely to Be
  Compensated By Another
• Standard Performance Evaluation Corporation
  (SPEC)
• Most Popular Benchmark Suite
• Suite Consists of Kernels, Small Fragments, Large
  Applications
• SPEC2000 CINT2000, CFP2000
• http//www.spec.org/
• Benchmark suites for servers
• SPECSFS measures performance of File servers
• SPECWeb measurers performance of Web servers

12
SPEC CPU2000 Programs

• Benchmark Language Descriptions
• 164.gzip C Compression
• 175.vpr C FPGA Circuit Placement and Routing
• 176.gcc C C Programming Language Compiler
• 181.mcf C Combinatorial Optimization
• 186.crafty C Game Playing Chess
• 197.parser C Word Processing
• 252.eon C Computer Visualization
• 253.perlbmk C PERL Programming Language
• 254.gap C Group Theory, Interpreter
• 255.vortex C Object-oriented Database
• 256.bzip2 C Compression
• 300.twolf C Place and Route Simulator

CINT2000 (Integer)
Source http//www.spec.org/osg/cpu2000/
13
SPEC CPU2000 Programs

• 168.wupwise Fortran 77 Physics / Quantum
  Chromodynamics
• 171.swim Fortran 77 Shallow Water Modeling
• 172.mgrid Fortran 77 Multi-grid Solver 3D
  Potential Field
• 173.applu Fortran 77 Partial Differential
  Equations
• 177.mesa C 3-D Graphics Library
• 178.galgel Fortran 90 Computational Fluid
  Dynamics
• 179.art C Image Recognition / Neural Networks
  
• 183.equake C Seismic Wave Propagation
  Simulation
• 187.facerec Fortran 90 Image Processing Face
  Recognition
• 188.ammp C Computational Chemistry
• 189.lucas Fortran 90 Number Theory /
  Primality Testing
• 191.fma3d Fortran 90 Finite-element Crash
  Simulation
• 200.sixtrack Fortran 77 Nuclear Physics
  Accelerator Design
• 301.apsi Fortran 77 Meteorology Pollutant
  Distribution

CFP2000 (Floating Point)
Source http//www.spec.org/osg/cpu2000/
14
Top 20 SPEC CPU2000 Results (2005)

15
How to Summarize Performance