Design and Implementation of GASNet Collectives

1
Design and Implementation of GASNet Collectives
Paul H. Hargrove PHHargrove_at_lbl.gov
LBNL September 9, 2004
2
Why GASNet?

• Language independent
• Used by both the Titanium and UPC projects
• Network independent
• Portable (low-performance) networks
• UDP MPI
• High-performance networks
• Quadrics/elan Myrinet /gm
• InfiniBand/vapi SP Switch/LAPI
• Dolphin/SCI X-1/SHMEM
• Altix/SHMEM
• SMP

3
Overview of the Design

• Fully non-blocking design and implementation
• Control over initiation/completion semantics
• Inspired by UPC collective specification
• Does not require global knowledge of addresses
• Design requirement for Titanium support
• Aggregation hints
• Design includes room for extensions
• Subset collectives (I.e. teams)
• Unequal contributions

4
Overview of the Design

• Data-movement collectives
• Broadcast
• Exchange (a.k.a. upc_all_gather_all)
• Scatter
• Gather
• Transpose
• Computational collectives (pending)
• Reduction (commutative and non-commutative)
• Scan (inclusive and exclusive)

5
Overview of the Implementation

• Portable reference implementation
• In terms of the network-independent functionality
• Object-oriented model
• Multiple algorithms for each operation
• Compile-time hooks for network-specific tuning
• Replacement of internal object implementations
• Addition of algorithms (e.g. using hardware
  support)
• Tuning of parameters
• Runtime hooks for network-specific tuning
• Selection from among existing algorithms

6
Performance of 8-byte Broadcast
7
Performance of 8-byte Broadcast