Case%20Study:%20PETSc%20ex19

1
Case Study PETSc ex19

• Non-linear solver (snes)
• 2-D driven cavity code
• uses velocity-velocity formulation
• finite difference discretization on a structured
  grid
• Instrumentation
• MPI
• MPI wrapper interposition library
• PDT (tau_instrumentor)
• PETSc library sources
• Application
• Selective instrumentation (tau_reduce)

2
Configuring TAU

• Profiling with wallclock time (on a quad PIII
  Linux machine)
• configure mpiinc/usr/local/packages/mpich/incl
  ude mpilib/usr/local/packages/mpich/lib
  pdt/usr/pkg/pdtoolkit/ -useropt-O2
  LINUXTIMERS
• Tracing
• configure mpiinc/usr/local/packages/mpich/incl
  ude mpilib/usr/local/packages/mpich/lib
  pdt/usr/pkg/pdtoolkit useropt-O2
  LINUXTIMERS
• Profiling with PAPI
• configure mpiinc/usr/local/packages/mpich/incl
  ude mpilib/usr/local/packages/mpich/lib
  pdt/usr/pkg/pdtoolkit/ -useropt-O2
  papi/usr/local/packages/papi
• setenv PAPI_EVENT PAPI_FP_INS
• setenv PAPI_EVENT PAPI_L1_DCM

3
Compiling TAU Makefiles

• Include TAU Stub Makefile (ltarchgt/lib) in the
  users Makefile.
• Variables
• TAU_CXX Specify the C compiler used by TAU
• TAU_CC, TAU_F90 Specify the C, F90 compilers
• TAU_DEFS Defines used by TAU. Add to CFLAGS
• TAU_LDFLAGS Linker options. Add to LDFLAGS
• TAU_INCLUDE Header files include path. Add to
  CFLAGS
• TAU_LIBS Statically linked TAU library. Add to
  LIBS
• TAU_SHLIBS Dynamically linked TAU library
• TAU_MPI_LIBS TAUs MPI wrapper library for C/C
• TAU_MPI_FLIBS TAUs MPI wrapper library for F90
• TAU_FORTRANLIBS Must be linked in with C linker
  for F90.
• TAU_DISABLE TAUs dummy F90 stub library
• Note Not including TAU_DEFS in CFLAGS disables
  instrumentation in C/C programs (TAU_DISABLE
  for f90).

4
Profiling ex19 (wallclock time)
5
Profiling ex19 (wallclock time)
6
Tracing ex19
7
Callgraph Display for ex19
8
Using TAU with PAPI (PAPI_FP_INS)
9
Using TAU with PAPI (PAPI_L1_DCM)
10
Using TAU with PAPI (PAPI_FP_INS)
11
Using TAU with PAPI (PAPI_FP_INS)
12
Support Acknowledgement

• TAU and PDT support
• Department of Energy (DOE)
• DOE 2000 ACTS contract
• DOE MICS contract
• DOE ASCI Level 3 (LANL, LLNL)
• U. of Utah DOE ASCI Level 1 subcontract
• DARPA
• NSF National Young Investigator (NYI) award