InfiniBand

1
InfiniBand

• André Larocque (1811828)
• Francis Lacroix (1826581)
• Thursday March 14th, 2002
• This Report was prepared for Professor L.
  Orozco-Barbosa in partial fulfillment of the
  requirements for the course ELG/CEG 4183

2
Agenda

• Introduction
• Current I/O Technologies
• Limitations of Current I/O Technologies
• Previous Solutions
• InfiniBands Role
• InfiniBand Architecture
• InfiniBand Benefits
• Current Status of InfiniBand
• The Future of InfiniBand
• Conclusion

3
Introduction

• Without I/O, computing just cant happen. But
  over the years traditional PCI bus, SCSI bus and
  ATA I/O connectors have become almost too slow
  for todays computing need.1
• New I/O technology developed by the InfiniBand
  Trade Association which was originally formed by
  Compaq, Dell, Hewlett-Packard, IBM, Intel,
  Microsoft and Sun Microsystems

4
Introduction

• New I/O technology developed is named InfiniBand.
• Original design is the result of merging Future
  I/O and Next Generation I/O.
• Successor to the Peripheral Component
  Interconnect (PCI) technologies.
• Currently concentrates on server I/O.

5
Current I/O Technologies
6
Limitations of Current I/O Technologies

• Existing PCs and server systems are hampered
  with an inflexible and relatively slow internal
  data flow system, including todays 64-bit, 66
  MHz PCI bus2
• Lack of development for the PCI technology has
  caused several problems to surface, specially in
  the server domain.

7
Limitations of Current I/O Technologies
8
Limitations of Current I/O Technologies

• In servers, RAS (Reliability, Availability
  Serviceability) is extremely important.
• Reliability
• Ability of a system (in our case server) to
  perform its required functions under stated
  conditions for a certain amount of time.
• Shared-memory architecture which prevents
  separation of different I/O controllers address
  space.
• Parallel nature of the PCI bus prevents the
  system from detecting and isolating a
  malfunctioning adapter.

9
Limitations of Current I/O Technologies

• Availability
• The degree to which a system is operational when
  required for use.
• Usually achieved through redundancy.
• Shared parallel nature of the PCI bus cannot
  support these redundant connections.
• Serviceability
• The degree at which components can be added,
  removed or exchanged in the system.
• There are no stable and reliable hot-pluggable
  mechanisms for PCI Cards/Adapters.

10
Previous Solutions

• Two main solutions
• Future I/O
• Next Generation I/O
• Remedies
• Specialized Web Appliances
• New PCI Technologies
• Integrating Gb Ethernet into the Chipset
• Hardware acceleration for TCP

11
InfiniBands Role

• Same purpose as PCI connect servers (CPU and
  memory) to external storage and LANs.
• Will coexist with, and then completely replace
  the PCI architecture.

12
InfiniBand Architecture

HCA Host channel adapter
Manages interactions between the server and
external sources, including other HCAs, network
devices, and peripherals.
13
InfiniBand Architecture

IBA Fabric

• Hardware and software that makes data movement
  possible.

• Consists of switches, integrated circuits and the
  programming

14
InfiniBand Architecture

I/O Units

• Connects servers to storage and network
  resources, and other peripherals.

• Can be shared by more than one server.

15
InfiniBand Architecture

TCA Target Channel Adapter

• Common IBA I/O Units IBA-Fibre Channel,
  IBA-SCSI, and IBA-Ethernet

16
InfiniBand Architecture

• InfiniBand fabric manager adds and removes nodes
• InfiniBand fabric manager assigns local
  identifiers to end nodes
• InfiniBand fabric manager establishes routing
  between end nodes

17
InfiniBand Architecture

Server
18
InfiniBand Architecture
19
InfiniBand Architecture

• Channel communication model, simpliflying
  device-driver design.
• This channel-based communication allows direct,
  protected access to host memory.
• Independant access to system memory cannot
  corrupt system memory and cause a system crash.
• Redundant physical paths between end nodes and
  switches amplify reliability of system.

20
InfiniBand Architecture

• Seems like a full-fledged network
• Described as an I/O network
• Bus itself is now viewed as a switch
• InfiniBand uses Internet Protocol Version 6
  (IPv6) with its 128-bit address, allowing an
  almost limitless amount of device expansion

21
InfiniBand Benefits

• Scalable performance Each link has a minimum
  connection bandwidth of 500MBps up to 6GBps for a
  single link.
• Increased reliability redundancy in connections.
• Increased availability redundancy in connections
  allows new routes to be established should a
  device or link fail.
• Increased serviceability IBA I/O units are
  designed to be hot-pluggable.

22
Current Status of InfiniBand

• "As the deployment of InfiniBand solutions draws
  near, it becomes more and more important that IT
  managers learn how InfiniBand architecture will
  dramatically alter data centers"3
• IT managers are learning about InfiniBand
• 30 member companies will demonstrate InfiniBand
  products this April at a conference.

23
The Future of InfiniBand
A few article headlines

• Experts predict rosy future for InfiniBand4
• Intel's Pappas says InfiniBand is the future5
• InfiniBand I/O standard will enhance storage
  networking6
• its high level acceptance seems already assured
  by endorsements from Compaq, Dell, Hewlett
  Packard, Microsoft and Sun Microsystems. The
  technology appears to have both technological and
  economic merits.6

24
QA
25
References

• searchStorage.com
• http//searchstorage.techtarget.com/
• CrossRoads InfiniBand Whitepaper
  http//www.crossroads.com/downloads/tempfiles/001_
  WP_InfiniBand.pdf
• Network World Fusion
• http//www.nwfusion.com/
• InfiniBand Trade Association
• http//www.infinibandta.org/
• Intel InfiniBand Presentations
• http//www.intel.com/technology/infiniband/presen
  tations.htm

26
Questions

• Q What does RAS stand for?
• A Reliability, Availability Serviceability
• Q Which market is currently targeted by the
  InfiniBand Trade Association?
• A The server market
• Q At which Ethernet speed does the PCI I/O
  become a bottleneck?
• A 1 Gb

27
Questions

• Q How is the reliability increased by the
  InfiniBand Architecture?
• A By having multiple links between each end
  nodes and switches (redundancy).
• Q What does HCA stand for?
• A Host Channel Adapter.