High Performance Messaging on Workstations

1
High Performance Messaging on Workstations

• Illinois Fast Messages (FM) for Myrinet

Presented By Scott Thomas
2
Introduction

• Workstations clusters
• Advantages
• Lower Cost
• Larger Software Base
• Greater Accessibility
• Disadvantages
• Protocol Overhead
• Buffer Management
• Operating System Overhead

3
Background

• Previous methods
• Mapping network device interface into user space
  directly via load/store operations
• Myrinet Network Features
• Uses LANai network coprocessor (5 MIPS)
• 3 DMA engines (in, out, host)
• Workstation Features
• Memory bus is fast, I/O Bus is slow!

4
(No Transcript)
5
Critical Issues

• Critical Issues
• Division of labor between the host and network
  coprocessor
• Assign as much functionality to the host as
  possible
• Management of the input/output bus
• Use DMA as much as possible
• Buffer management
• Minimal amount of queues

6
The Fast Messages Approach

• Illinois Fast Messages 1.0 methods
• FM_send_4() - Send a four word message
• FM_send() - Send a longer message
• FM_extract() - Process a received message

7
FM Implementation

• FM 1.0
• Host Program
• LANai Control Program (LCP)
• Message Transmission
• 1. Get data to the LANai
• 2. Put the data on the line
• 3. Remove data from the line at receiver
• 4. Transport data to host memory

8
Network Coprocessor Program

• Baseline vs. Streamed

9
SBus Management

• Methods of Interaction
• all-DMA
• outgoing host copies data into DMA, give LANai
  pointers, triggers send
• incoming uses pointer to write data into host
  memory
• hybrid
• outgoing host copies data into LANais memory,
  triggers send
• incoming DMAs messages into host memory

10
SBus Management
11
Buffer Management / Flow Control

• 4 queues
• LANai send
• LANai receive
• host receive
• host reject
• Return-to-sender flow control
• Receiver returns packet to sender if no memory
  available

12
Comparative Performance

• Adding small features can cause huge performance
  hits.

13
Comparative Performance
14
Conclusion

• Improvements
• improved I/O bus performance for non-DMA
  operations
• faster network interface processor

15
(No Transcript)