ECE 6160: Advanced Computer Networks InfiniBand - PowerPoint PPT Presentation

1 / 36
About This Presentation
Title:

ECE 6160: Advanced Computer Networks InfiniBand

Description:

ECE6160:Advanced Computer Networks. 6. Why InfiniBand? Ease of connect ... IBA does not assign specific meaning to Class of Service. Service Level (SL) is ... – PowerPoint PPT presentation

Number of Views:32
Avg rating:3.0/5.0
Slides: 37
Provided by: xubi
Category:

less

Transcript and Presenter's Notes

Title: ECE 6160: Advanced Computer Networks InfiniBand


1
ECE 6160 Advanced Computer NetworksInfiniBand
  • Instructor Dr. Xubin (Ben) He
  • Email Hexb_at_tntech.edu
  • Tel 931-372-3462

Note Some of these slides are adapted from
InfiniBand Trade Association (http//www.infiniban
dta.org).
2
Prev
  • Networked Storage
  • NAS, SAN
  • iSCSI

3
I/O Systems
interrupts
Processor
Cache
Memory - I/O Bus
Main Memory
I/O Controller
I/O Controller
I/O Controller
Graphics
Disk
Disk
Network
4
InfiniBand Architecture (IBA) Focus
  • 0 Minimized cabling
  • 0 Intra-box / Inter-box solution
  • 0 Simplified Switches
  • 0 Server application architecture support
  • 0 RAS features necessary for 24x7 systems

5
What is Infiniband?
  • Infiniband Architecture(IBA) defines a System
    Area Network (SAN)
  • IBA SAN is a communications and management
    infrastructure for I/O and IPC
  • IBA defines a switched communications fabric
  • high bandwidth and low latency
  • protected, remotely managed environment.
  • IBA hardware off-loads from the CPU much of the
    I/O communications operation.

6
Why InfiniBand?
  • Ease of connect
  • 0One fabric connection for all host I/O
  • IPC, Storage I/O, Network I/O
  • 0Improved upgrade/growth strategy
  • Scalability
  • 0Thousands of nodes per subnet / multiple
    subnets
  • 0Cost-effective intra-subnet switches
  • 0Multiple link widths for high bandwidth
  • Performance
  • 0CPU-offloaded hardware support for message
    queuing, memory protection, fabric protocol
    processing

7
Why InfiniBand? (Contd.)
  • Reliability, Availability, Serviceability
  • 0Redundant paths and/or fabrics
  • 0In-band management
  • Connector, backplane, baseboard, chassis, power
  • 0Layered error management architecture
  • Flexibility
  • 0Topology independent
  • 0Multiple link widths
  • 0Multiple units of transfer
  • 0Auto-negotiation
  • 0Multiple form factors / standardized connectors

8
IB Architecture Model
9
IB Architecture Layers
10
Physical Layer Physical protocol overview
11
Link Characteristics
12
Work queue
13
Host Channel Adapter
14
Target Channel Adapter
15
Switch
16
Router
17
Addressing
  • For subnet transactions within a cluster group,
    IB uses 64-bit addressing. (LID)
  • For external transactions between IB subnets, it
    uses 128-bit IPv6 addressing. (GID)
  • Easily integrate into mainstream IP networking
    via IP routers.

18
Link Layer
  • Responsible for the orderly reception and
    transmission of packets including
  • 0Link initialization and control
  • 0Provide Virtual Lanes (VLs)
  • 0Transmit packets
  • Control flow
  • Arbitration between Virtual Lanes (VLs)
  • 0Receive packets
  • Error checking

19
Virtual Lanes (VLs)
  • Communications over the IB fabric are performed
    through VLs.
  • A VL represents a pair of receive and transmit
    buffers on a port, with each VL providing its own
    flow control
  • Each port/link may support up to 15 VLs for data
    and one for management.
  • The data streams originating from an HCA are
    mapped to VLs and are muxed into a single stream
    over the link, and demuxed at the receiving side.

20
Virtual Lane
21
Class of service
  • Class of Service is intended for use in traffic
    isolation or providing differentiated services
  • IBA does not assign specific meaning to Class of
    Service
  • Service Level (SL) is
  • 0used to identify service class of a flow
  • 0a property of the connection or datagram
  • 0a 4-bit field of the packet
  • SL defines VL to use on each hop

22
Link Layer Flow Control
  • Each VL has separate buffering and flow control
  • Credit based flow control is used
  • 0Credits sent on per VL basis
  • 0Receiver issues credit based on packets taken
    from incoming VL buffer
  • 0Credits sent to sender
  • 0Sender manages flow control

23
Transport Layer
  • The five Transport Services
  • 0Reliable Connection
  • 0Reliable Datagram
  • 0Unreliable Connection
  • 0Unreliable Datagram
  • 0Raw Datagram
  • The Transport Functions
  • 0Send
  • 0RDMA Read Write
  • 0Atomic
  • not all functions apply to all the services
  • send WQE indicates the function type to use

24
Message
25
Data Packets
26
Path MTU
27
Transport services one purpose
28
IBA reliable transport
29
IBA unreliable service
30
Raw Datagrams
31
Service Reliable Connection
32
Service Unreliable Connection
33
Service Reliable Datagram
34
Service Unreliable Datagram
35
Error Detection
36
Summary
  • InfiniBand replaces the traditional server bus
    with a high-speed switch.
  • Communication paths over InfiniBand links are
    established by VLs
  • Local IB grouping (subnet) communications are
    performed via data link switching.
  • Intersubnet communication is based on IPv6
    addressing and network routing.
  • Intelligent storage switches may provide
    any-to-any storage connectivity.
  • Overall, IB provides a solution for both internal
    and external connection of server nodes.
Write a Comment
User Comments (0)
About PowerShow.com