SMP Based Cluster

1
SMP Based Cluster
Inter-node Communication
Network
2
MPI Intra-node Communication Design

• Small messages are transferred directly through
  the receive buffers (RBxy)
• Large messages are transferred indirectly where
  data is transferred through the send buffer pool
  (SBPx) and a control message is put into the
  receive buffer
• Benefits
• Lock free
• Efficient cache utilization
• Efficient memory usage

3
Performance on NUMA Cluster

• Bandwidth is improved by up to 50
• Benefits mainly come from the reduced L2 cache
  miss rate

4
Performance on Multi-core NUMA Cluster

• CMP latency is improved by up to 12
• SMP latency is improved by up to 30
• Bandwidth is improved by up to 25