Comments on Co-Array Fortran

1
Comments on Co-Array Fortran

• Robert W. Numrich
• Minnesota Supercomputing Institute
• University of Minnesota, Minneapolis

2
Philosophy Behind the CAF Model

• A minimum number of new features that look and
  feel like Fortran.
• The rules for co-dimensions are the same as those
  for normal dimensions with a few exceptions.
• The CAF model is purely local.
• The compiler performs normal optimization between
  synchronization points.
• Compiler is not required to perform, is not
  expected to perform, is deliberately prevented
  from performing, global optimization.
• Programmer is responsible for explicit data
  distribution, explicit communication, and
  explicit synchronization.
• Programmer is responsible for memory consistency.

3
The Essential Features of CAF

• Co-array syntax
• Co-dimensions (How images are related to each
  other)
• Data communication (How data moves between
  images)
• Synchronization
• Full barrier
• Pair-wise handshake
• Dynamic Memory management
• Allocatable co-arrays
• Allocatable/pointer components of co-array
  derived types

4
Recommendations

• Delete all of Section 8.5
• Substitute SYNC(myPal)
• Delete everything on I/O except
• Only image 1 reads stdin
• Records from different images not mixed to stdout
• Delete all the stuff on collectives
• Substitute the functions glbSum(x), glbMin(x),
  glbMax(x)
• Argument is not necessarily a co-array.

5
Ragged Arrays

• The most important, the most powerful, feature of
  the CAF model.
• Allows each image to have data structures with
  different sizes and different locations on each
  image.
• No other model can handle this.
• Fortran with CAF extensions handles it in a very
  natural way.
• Allocatable/pointer components of co-array
  derived types
• type x
  real,allocatable,target z()
• real,pointer ptr() type(x)
  y?
• end type x
  allocate(z(someRule(this_image)))
• 
  yptr gt z
• 
  sync
• 
  ypptr 25.0
• Most difficult feature to implement on crummy
  hardware and/or crummy operating systems.
• Good systems should be rewarded for getting this
  right.
• We should not allow bad systems to drag
  everything down to the lowest common denominator.

6
Memory Consistency

• The programmer is responsible for maintaining
  memory consistency, not the compiler.
• The rules must be very simple and very clear.
• With just SYNC, it is very simple
• The addition of NOTIFY/QUERY makes it not so
  simple
• Segment boundary statements
• Compiler may optimize between segment boundaries
  but not across segment boundaries
• Processor must make co-arrays visible to all
  images across segment boundaries
• The programmer must make sure that one and only
  one image defines a co-array variable at the
  same time
• The programmer must make sure that no image tries
  to reference a co-array variable at the same
  time as another image is trying to define it.

7
Input/Output

• Whats the minimum needed?
• stdin/stdout are special cases
• Always connected to all images
• Only image 1 can read stdin
• System must not mix records to stdout from
  different images
• Shared files
• Allow each image to open to same unit
• Direct access only
• Allowing sequential access requires changes to
  backspace, rewind, etc.
• open(unitu,access?,)
• open(unitu,accessconnectList,)
• Do we need teams?
• Do we need to sync?

8
Collectives

• Avoid language bloat
• Not everything in the MPI Library needs to be
  reproduced in CAF as intrinsic procedures
• What do we really need and want?
• What should the interface be?
• CAF is intended as a low-level language
• Collectives are easy to write in CAF for any
  specific procedure
• Throwing a long list of new intrinsic procedures
  over the wall may discourage vendors from
  adopting CAF
• If anything, supply intrinsic functions
• glbSum(x), glbMin(x),
  glbMax(x)
• argument x need not be a
  co-array
• Propose a supporting library for CAF.

9
VOLATILE Co-arrays?

• What do we want the VOLATILE attribute to mean
  for co-arrays?
• Inhibit optimization?
• Always read from memory?
• Always flush to memory?
• Can VOLATILE make spin-loops work without the
  need for an artificial sync-memory() from the
  programmer?
• Statements with co-arrays are segment boundary
  statements

10
SYNC/BARRIER

• One new statement SYNC
• Implies full BARRIER for all images
• No arguments
• No SYNC_IMAGES
• Synchronization between subsets of images can be
  done with NOTIFY/QUERY

11
NOTIFY/QUERY

• Why do we want NOTIFY/QUERY?
• Split-phase sync
• Subset sync
• Master-slave work distribution
• Should they match in pairs?
• Should we expose which notify matches which
  query?
• Maybe we really want EVENTS?
• EVENT_POST(tag)
• EVENT_WAIT(tag)
• EVENT_CLEAR(tag)