Solaris IPoIB (ibd) Implementation - PowerPoint PPT Presentation

About This Presentation
Title:

Solaris IPoIB (ibd) Implementation

Description:

If transmitter guarantees GID0 usage, SM assisted reverse lookup possible. RARP unsupported ... client's MAC address for 3rd party lookup. IPoIB Configuration ... – PowerPoint PPT presentation

Number of Views:208
Avg rating:3.0/5.0
Slides: 9
Provided by: ietf
Learn more at: https://www.ietf.org
Category:

less

Transcript and Presenter's Notes

Title: Solaris IPoIB (ibd) Implementation


1
Solaris IPoIB (ibd) Implementation
  • Kanoj Sarcar
  • Sr Staff Engineer
  • Sun Microsystems
  • (presented by Bill Strahm)

2
Architecture Overview
  • DLPI driver
  • Only supported for GID0
  • Interop (TopSpin, openib)
  • Software structure
  • IP SDP uDAPL
  • ibd ...
  • IBTF / IBMF
  • HCA driver
  • Performance

3
Tools changes (20 byte MAC)
  • New SIOCXARP ioctls and struct xarpreq
  • arp(1M) changes
  • PPPD proxyarp etc
  • Undocumented helper function ala ether_ntoa,
    ether_aton
  • extern char _link_ntoa(const unsigned char ,
    char , int, int)
  • extern unsigned char _link_aton(const char ,
    int )
  • ifconfig(1M), snoop(1M)
  • RARP
  • /etc/ethers, NIS assume 6 bytes!

4
Tools changes (IPoIB specific)
  • DHCP agent
  • Usage of BROADCAST bit
  • Setting IB hardware type in DHCP packet in htype
  • Client id format
  • Code Len Type lt----------------
    Client-Identifier --------------gt
  • ---------------------------------------
    ---------------....----
  • 61 21 00 00 (4 octets)
    16 octet GID
  • ---------------------------------------
    ---------------....----
  • DHCP Server
  • No explicit ARP cache setting with DHCP chaddr
  • Snoop(1M) parses IPoIB, disallows MAC filters

5
Tools changes (IPoIB specific)Part II
  • ifconfig(1M)
  • hme0 flags1000843ltUP,BROADCAST,RUNNING,MULTICAST
    ,IPv4gt mtu 1500 index 2
  • inet 10.6.98.56 netmask ff000000
    broadcast 10.255.255.255
  • ether 03ba244df
  • ibd1 flags1000843ltUP,BROADCAST,RUNNING,MULTICAST
    ,IPv4gt mtu 2044 index 4
  • inet 192.168.100.100 netmask ffffff00
    broadcast 192.168.100.255
  • ipib 004a000000123402c919
    765711
  • SNMP (net-snmp)
  • ifSpeed consistent with draft-ietf-ipoib-ibif-mib-
    07.txt
  • ifPhysAddress as MAC address (to match ARP)
  • octet/packet counts only for ipoib network
    interface

6
IPoIB oddities
  • DLPI receiver indicates src addr unavailable
    for unicast packets
  • If transmitter guarantees GID0 usage, SM assisted
    reverse lookup possible
  • RARP unsupported
  • Non persistant MAC
  • Server requires client's MAC address for 3rd
    party lookup

7
IPoIB Configuration
  • One time discovery
  • Reconfiguration boot or IO discovery command
  • Create possible ipoib instance per port/Pkey
  • SM must be up
  • Interface initialization
  • IP plumb triggers MCG probe as validity check
  • SM must be up, fabric admin must create IP MCG
  • Decisions
  • End nodes do not create IP MCG
  • End nodes do not have to know IB parameters
  • Issues
  • IP multicast group membership requests
  • Can not wait, can not spool!
  • Solaris IP stack joins 224.0.0.1 at interface init

8
Interesting Driver Details
  • QPN/CQ/WQE/PathRecord
  • Tx path copy vs registration
  • GID/QPN to Pathrecord (hash/cache)
  • Disallow (DLPI) change MAC address
  • Async thread for (blocking) SM communication
  • IB link up/down handling
  • ULP notification for IPMP failover
  • MCG create/delete trap handling
  • 2 MCG membership lists (full/sendonly, non)
  • MCG pathrecord and membership implication
Write a Comment
User Comments (0)
About PowerShow.com