InfiniBand Range Extenders

1
Wide Area InfiniBand
What it is, and why it is
Obsidian Strategics Inc. David Southwell dsouthwel
l_at_obsidianstrategics.com
ESCC/ Internet2 Winter 2008 Joint techs Jan 22
Honolulu, HI
2
InfiniBand 101
Open standard for high bandwidth, low latency,
low CPU overhead, highly scalable
interconnect. 1X, 4X, 8X or 12X channels at 2.5,
5.0 or 10.0 Gbits/s per link (less 20 for 8B/10B
encoding). 1X SDR 2Gbits/s, 12X QDR
96Gbits/s (silicon Q1'08) - today, mostly 8 or
16Gbits/s deployed. Cut-through switches 60ns
latency, scaling to thousands of ports. Open
stacks for Linux (Windows) Open Fabrics
Alliance Low CPU loading due to Remote Direct
Memory Access (RDMA)? Lossless flow control
using buffer-credit scheme very efficient but
limits link lengths!
Host Channel Adapter A
Host Channel Adapter B
Across IB link
Update credit
From user-space memory
Into user-space memory
3
Equipment for range-extended InfiniBand
Wide Area
- SDR 4X IB dual GbE payloads - Global reach
over 10G WANs OC-192 or 10Gbase-LW/LR
SONET/ ATM/ GFP-F/ IPv6 POS - 2-port IB switch
or router modes
Campus Area
- 10/40km dark fiber (LR/ER optics)? - 2-port IB
switch mode only
Secure Wide Area (functional prototype stage)?
- Global reach over 10GbE WANs - 2-port IB switch
or router modes - Line-rate AES-192 crypto
4
Applications