Software Overhead in Messaging Layers

1
Software Overhead in Messaging Layers

• Pitch Patarasuk

2
Messaging Layer

• Bridge between the hardware functionality and the
  user communication requirement
• Network hardware feature
• Arbitrary delivery order
• Finite buffering
• Limited fault handling
• User communication requirement
• In-order delivery
• End-to-end flow control
• Reliable transmission

3
Messaging Layer (cont.)
4
Software overhead

• Main communication cost hardware cost
  software cost
• Software cost dominates hardware cost
• 50-70 of the software cost are the direct gap
  between the network features and user
  requirements

5
Software overhead in messaging layers

• Analyze the costs of communication functionality
  and network features
• What overhead might be reduced if the underlining
  network provide higher level of service

6
CM-5 active message layer (CMAM)

• CM-5
• Parallel machine up to a few thousand nodes
  connected with an incomplete fat tree topology
• Active message
• Communication mechanism intended to expose full
  hardware flexibility and performance of modern
  networks
• Each message contain an address of a user-level
  handler which is executed on message arrival with
  the message body as an argument
• The handler extract data from the network and
  integrate it into the ongoing computation
• CMAM vs Send/Recv
• User direct access to network, no OS involve, the
  data go directly to the user space computation

7
Software overhead cost analysis

• Consider implementation of 3 protocols
• Single-packet delivery
• Finite sequence, multi-packet delivery
• Indefinite sequence, multi-packet delivery
• Use instruction counts for measurement

8
Single-packet delivery
Description Source Destination
Call/Return 3 10
NI Setup - -
Write to NI 2 -
Read from NI - 3
Check NI status 7 12
Control Flow 3 2
20 27
9
Finite sequence, multi-packet delivery
10
Indefinite sequence, multi-packet delivery
11
Message size 16 words
12
Message size 1024 words
13
Messaging layer with high-level network feature

• Given that CMAM is considered to be very
  efficient, there are 2 choices to reduce software
  overhead
• Lower user requirement
• Raise level of service provided by the network
• Compressionless Routing (CR)
• Order-preserving transmission
• Deadlock freedom independent of packet acceptance
  guarantees
• Fault-tolerant transmission at packet level

14
Single-packet delivery

• Has the same cost as the previous CMAM case
• However, it is guaranteed to be fault-free, no
  deadlock or buffer overflow

15
Finite sequence, multi-packet delivery
16
Indefinite sequence, multi-packet delivery
17
Message size 16 words
18
Message size 1024 words
19
Discussion

• Larger packet sizes
• Reduce overhead, especially for
  indefinite-sequence protocol
• Improved network interfaces and DMA hardware
• Network interface
• only make basic cost faster, but not protocol
  cost in messaging layer
• Make it more important for messaging layer to be
  effective
• DMA
• only reduce cost in moving large amounts of data

20
Discussion (Cont.)

• Implication of network design
• Improving routing performance may increase
  software cost, e.g. out-of-order delivery
• Reducing communication features
• Fault-tolerance, in-order-delivery
• Put burden on parallel software programmer
• Use instruction counts for measurement instead of
  latency
• Latency is hard to measure in a portable fashion

21
Conclusion

• Software overhead is much larger than routing
  time and 50-70 are messaging layer overhead
• Software overhead in messaging layer is the cost
  of the gap between the network feature and the
  user requirement
• This cost can be reduced to zero if the
  underlining network provides functionality that
  the user requires.

22
Questions?