Comparing Cray Tasking and OpenMP NERSC User Services

1
Comparing Cray Tasking and OpenMPNERSC User
Services

• Overview of Cray Tasking
• Overview of OpenMP
• Directives Comparison
• Mixing OpenMP with MPI

2
Cray Shared-Memory Multiprocessors

• 1982 X-MP
• 1, 2, or 4 processors
• 1985 Cray-2
• 4 processors
• 1988 Y-MP
• 8 processors
• Cray for the 90s C90
• 16 processors

3
Cray Tasking

• Microtasking
• Parallel loops
• Macrotasking
• Parallel regions
• Autotasking
• Emphasis on auto

4
History of OpenMP

• Defined by OpenMP Architecture Review Board
• Compaq/DEC, HP, IBM, Intel, SGI/Cray, Sun, etc.
• Endorsed by software and application vendors
• Absoft, The Portland Group, etc.
• ANSYS, Fluent, Livermore Software, NAG, etc.
• Fortran, C, and C bindings
• Fortran specification released late 1997
• C and C specification released late 1998
• See http//www.openmp.org for complete details

5
Scope of OpenMP

• Shared-memory explicit parallelism
• No automatic parallelism
• Some research into extending to
  distributed-memory environments
• Directives, environment variables, runtime
  library routines

6
Cray Tasking Syntax

• sentinel directive argument , directive
• sentinel
• cdir or !dir (cmic and !mic are obsolete)
• Fixed format source
• normal Fortran column rules
• Free format source
• sentinel begins anywhere on a line, but must be
  the first text
• sentinel followed by anything other than space
  implies continuation
• Multiple directives are comma-delimited
• if a directive has required arguments, no other
  directives allowed

7
OpenMP Syntax

• prefix directive clause, clause
• Fixed format source
• prefix !OMP, COMP, or OMP
• normal Fortran source rules
• Free format source
• prefix !OMP
• sentinel begins anywhere on a line, but must be
  the first text
• normal Fortran source rules

8
Directive Correspondence

• Autotasking
• DOALL
• GUARD
• ENDGUARD
• PARALLEL
• ENDPARALLEL
• DOPARALLEL
• ENDDO
• CASE
• ENDCASE

• OpenMP
• PARALLEL DO
• CRITICAL
• END CRITICAL
• PARALLEL
• END PARALLEL
• DO
• END DO
• SECTIONS
• END SECTIONS

9
Simple Parallel Loop

• !MIC DOALL PRIVATE(I) SHARED(X,Y,Z)
• !OMP PARALLEL DO
• DO I 1, N
• Z(I)LOG(SIN(X(I))2COS(Y(I))4)
• END DO
• !OMP END PARALLEL DO

10
Directive Correspondence

• Autotasking
• DOALL
• GUARD
• ENDGUARD
• PARALLEL
• ENDPARALLEL
• DOPARALLEL
• ENDDO
• CASE
• ENDCASE

• OpenMP
• PARALLEL DO
• CRITICAL
• END CRITICAL
• PARALLEL
• END PARALLEL
• DO
• END DO
• SECTIONS
• END SECTIONS

11
Parallel Loop With Critical Section

• !MIC DOALL PRIVATE(I) SHARED(X,Y,Z,T)
• !OMP PARALLEL DO
• DO I 1, N
• Z(I) LOG(SIN(X(I))2 COS(Y(I))4)
• !MIC GUARD
• !OMP CRITICAL
• T T Z(I)
• !MIC ENDGUARD
• !OMP ENDCRITICAL
• END DO
• !OMP END PARALLEL DO

12
Directive Correspondence

• Autotasking
• DOALL
• GUARD
• ENDGUARD
• PARALLEL
• ENDPARALLEL
• DOPARALLEL
• ENDDO
• CASE
• ENDCASE

• OpenMP
• PARALLEL DO
• CRITICAL
• END CRITICAL
• PARALLEL
• END PARALLEL
• DO
• END DO
• SECTIONS
• END SECTIONS

13
Parallel Regions

• !MIC PARALLEL PRIVATE(X) SHARED(Y)
• !OMP PARALLEL PRIVATE(X)
• !OMP DO
• X 3.0
• !MIC GUARD
• !OMP CRITICAL
• Y Y 1.0
• !MIC ENDGUARD
• !OMP ENDCRITICAL
• !MIC ENDPARALLEL
• !OMP END DO
• !OMP END PARALLEL

14
Directive Correspondence

• Autotasking
• DOALL
• GUARD
• ENDGUARD
• PARALLEL
• ENDPARALLEL
• DOPARALLEL
• ENDDO
• CASE
• ENDCASE

• OpenMP
• PARALLEL DO
• CRITICAL
• END CRITICAL
• PARALLEL
• END PARALLEL
• DO
• END DO
• SECTIONS
• END SECTIONS

15
Generalized Parallel Loop

• SUM 0.0
• !MIC PARALLEL PRIVATE(XSUM,I) SHARED(SUM,A,N)
• XSUM 0.0
• !MIC DOPARALLEL
• DO I 1, N
• XSUM XSUM A(I)
• ENDDO
• !MIC GUARD
• SUM SUM XSUM
• !MIC ENDGUARD
• !MIC ENDDO
• !MIC ENDPARALLEL

16
Generalized Parallel Loop (cont.)

• SUM 0.0
• !OMP PARALLEL PRIVATE(XSUM)
• XSUM 0.0
• !OMP DO
• DO I 1, N
• XSUM XSUM A(I)
• ENDDO
• !OMP CRITICAL
• SUM SUM XSUM
• !OMP END CRITICAL
• !OMP END DO
• !OMP END PARALLEL

17
Directive Correspondence

• Autotasking
• DOALL
• GUARD
• ENDGUARD
• PARALLEL
• ENDPARALLEL
• DOPARALLEL
• ENDDO
• CASE
• ENDCASE

• OpenMP
• PARALLEL DO
• CRITICAL
• END CRITICAL
• PARALLEL
• END PARALLEL
• DO
• END DO
• SECTIONS
• END SECTIONS

18
Parallel Blocks

• !MIC PARALLEL
• !OMP PARALLEL
• !MIC CASE
• !OMP SECTIONS
• !OMP SECTION
• CALL SUBA
• !MIC CASE
• !OMP SECTION
• CALL SUBB
• !MIC ENDCASE
• !OMP END SECTIONS
• !MIC ENDPARALLEL
• !OMP END SECTIONS

19
Mixing OpenMP and MPI

• INCLUDE mpi.h'
• CALL MPI_INIT(IERR)
• CALL MPI_COMM_RANK(MPI_COMM_WORLD,MYID,IERR)
  
• CALL MPI_COMM_SIZE(MPI_COMM_WORLD,NP,IERR)
• PRINT ,'PROCESS ',MYID,' OF ',NP,' IS
  ALIVE'
• !OMP PARALLEL PRIVATE(NTHREADS, TID)
• TID OMP_GET_THREAD_NUM()
• PRINT ,'HITHREAD',TID,' PROCESS',MYID
• IF (TID .EQ. 0) THEN
• NTHREADS OMP_GET_NUM_THREADS()
• PRINT ,'NTHREADS',NTHREADS,MYID',MYID
• END IF
• !OMP END PARALLEL
• CALL MPI_FINALIZE(IERR)
• END

20
Mixing OpenMP and MPI (cont.)

• mpxlf_r smp.f -qsmp
• hi End of Compilation 1
• 1501-510 Compilation successful for file smp.f.
• setenv XLSMPOPTS "parthds2"
• ./a.out -nodes 2
• Process 1 of 2 is alive
• Process 0 of 2 is alive
• HITHREAD0 PROCESS0
• NTHREADS2 MYID0
• HITHREAD1 PROCESS0
• HITHREAD0 PROCESS1
• NTHREADS2 MYID1
• HITHREAD1 PROCESS1