Advanced MPI Lab

1
Advanced MPI Lab

• MPI I/O Exercises

2
Getting Started

• Get a training account from the instructor
• Login (using ssh) to ranger.tacc.utexas.edu

3
MPI I/O Exercises

• Objectives Get familiar with MPI I/O
• Setting up an MPI I/O problem requires
  substantial overhead
• Set up communicators
• Set up MPI datatypes for structures on disk and
  in memory
• Set up fileviews.
• Lab time is finite, so well start with an
  example, and just make some tweaks.

4
I/O Exercises

• Copy the sample program to your directory
• dstanzi/IOLab/ghost_rows.c
• (FORTRAN version coming soon)
• Also, copy the sample job script
• dstanzi/IOLab/ghost_job
• If you havent run jobs on Ranger before, you may
  want to peruse the user guide
• http//services.tacc.utexas.edu/index.php/ranger-u
  ser-guide

5
I/O Exercises

• This program operates on a 2x3 grid of 6 MPI
  tasks.
• Each task has a 10x10 block of the final result
  array
• Each local block is a 12x12 array, the data in
  the middle, with 1 ghost value all the way around
  the border.
• The total 20x30 array is written on disk in a
  single write_all() command, grabbing the values
  from the middle of the 12x12 array, and skipping
  all the ghost values.
• The trick is setting up two subarray datatypes,
  one for how the data exists on disk (filetype, no
  ghost rows), and one for how it exists in memory
  (memtype).
• The code illustrates best practice when your
  dataset is too large for the memory of any one
  node.

6
I/O Exercises

• Once youve taken a look through the code, make
  the following modifications
• Change from one ghost value to two ghost values
  (8x8 data block).
• Display the resulting file to check your answers
• Modify the code to operate on a 2x4 grid of tasks
  instead of a 2x3 grid (change the total array
  size).
• Hint Youll need to change the MY_NSLOTS
  variable in the job script as well as the code
• For the ambitious
• Read the data back in to the 8 tasks. Rather
  than read in an 8x8 block, have each task read a
  4x16 block (same total datasize, different
  distribution).

7
Compiling and Running the Code

• This code will work best with mvapich2 and the
  intel compiler. Run these 3 commands before
  compiling to change your environment
• login3 module del mvapich
• login3 module swap pgi intel
• login3 module load mvapich2
• You can compile using the command
• mpicc ghost_rows.c
• Run using
• qsub ghost_job