SPEC HPG Benchmarks for HPC Systems

1
SPEC HPG Benchmarks for HPC Systems
Kumaran Kalyanasundaram for SPEC High-Performance
Group

• Kumaran Kalyanasundaram, PhD
• Chair, SPEC HPG
• Manager, SGI Performace Engineering

2
SPEC HPGs Purpose

• The High Performance Group focuses on the
  development of application benchmarks for high
  performance computers.

3
SPEC HPG

• Founded in 1994 (Perfect Benchmarks initiative
  became HPG).
• Members from industry and academia.
• Two active benchmarks - SPEC OMP SPEC HPC2002.
• New MPI2006 benchmark currently under
  development.

4
SPEC HPG Benchmark Suites
MPI2006
OMPL2001
OMP2001
HPC96
HPC2002
Founding of SPEC HPG
Jan 1994
Oct 1995
June 2001
June 2002
Jan 2003
2006
5
SPEC OMP

• Benchmark suite developed by SPEC HPG (High
  Performance Group)
• Benchmark suite for performance testing of shared
  memory processor systems
• Uses OpenMP versions of SPEC CPU2000 benchmarks
  and candidates

6
Why Did SPEC Choose OpenMP?

• Benchmark suite is focused on SMP systems
• OpenMP is a standard, and is applicable to
  Fortran, C, and C.
• Directive based OpenMP allows serial version to
  remain largely intact.
• Quickest path to parallel code conversion.

7
OMP/CPU2000 Similarities

• Same tools used to run the benchmarks
• Similar run and reporting rules
• Uses geometric mean to calculate overall
  performance relative to a baseline system
• Similar output format

8
SPEC OMP Benchmark Principles

• Source code based
• Limited code and directive modifications
• Focused on SMP performance
• Requires a base run
• with no source modifications
• single set of compiler flags for all benchmarks
• SPEC supplied tools required to run benchmark

9
OMPM2001 Benchmarks
10
OMP vs CPU2000
11
Program Memory Footprints
12
SPEC HPC2002 Benchmark

• Full Application benchmarks(including I/O)
  targeted at HPC platforms
• Serial and parallel (OpenMP and/or MPI)
• Currently three applications
• SPECenv weather forecast
• SPECseis seismic processing, used in the search
  for oil and gas
• SPECchem comp. chemistry, used in chemical and
  pharmaceutical industries (gamess)
• All codes include several data sizes

13
SPEC MPI2006

• An application benchmark suite that measures
  CPU, memory bw, interconnect, compiler, MPI
  performance.
• Search program is open till March 31st, 06
• Candidate codes in the areas of Comp. Chemistry,
  weather forecasting, HE Physics, Oceanography,
  CFD, etc.

14
Future Goals

• Very large data sets for MPI2006.
• Follow onto SPEC OMPM(L)2001.
• Update SPEC HPC2002 suite.