iSER on SCTP

1
iSER on SCTP IB

• draft-hufferd-ips-iser-sctp-ib-00.txt
• Generalizations to iSER specification
• John Hufferd
• Mike Ko
• Yaron Haviv

2
Abstract

• The iSCSI Extensions for RDMA document currently
  specifies the RDMA data transfer capability for
  iSCSI over iWARP/TCP
• This new document generalizes the iSER document
  to permit it to be used with other RDMA capable
  protocols such as
• iWARP/SCTP
• InfiniBand
• etc.
• It also describes what should be defined in the
  InfiniBand Trade Association and what things are
  appropriate for specification in the IETF

3
Motivation

• Current wordage in iSER is only applicable to
  iSER on iWARP/TCP
• It was felt that the protocol should be made
  generic for RDMA LLPs, and that includes
• iWARP/SCTP
• InfiniBand
• Etc.
• Makes an iSCSI based protocol apply across more
  Networks, and eliminates some of the management
  and discovery protocols that would otherwise be
  needed
• For example, would not have to fix the short
  comings of SRP (SCSI RDMA Protocol) on IB

4
Goals

• To specify changes/adjustments that should be
  considered to the wordage in the iSER document to
  make it more General
• These changes should not modify the basic
  operation of iSCSI/iSER when operating on
  iWARP/TCP
• Some of the terminology needed to be clarified as
  to the applicability of the terms to the actual
  LLP used

5
Examples of changes (1)

• The term iWARP protocol suite is replaced by
  RDMA-Capable Protocol
• The term iWARP layer will be replaced by
  RDMA-capable protocol layer
• Wherever the iWARP term is specific to the TCP
  implementation, it will be replaced with
  iWARP/TCP
• The term RNIC will be replaced with
  RDMA-Capable Controller
• The clause such as an RNIC will be added as
  needed

6
Examples of changes (2)

• The Steering Tag (STag) term will have its
  definition extended such at the IB Local
  Steering Tag (L-Key) and the Remote Steering
  Tag (R-Key) are included in the STag definition
  by way of example
• The Definitions for IRD and ORD terms will have
  their definition extended such that the IB
  Responder Resources, and Initiator Depth are
  included in the definitions by way of example
• The Term RDMA-capable protocol (RCP) will be
  defined and used when ever any RDMA wire protocol
  or RDMA protocol stack is applicable
• RDMAP should be used only when it explicitly
  refers the iWARP protocol (TCP or SCTP)

7
Examples of changes (3)

• The Term RDMAP Stream will be replaced by the
  term RCP Stream and defined as
• RCP Stream - A single bidirectional association
  between the peer RDMA-capable protocol layers on
  two Nodes over a single transport-level stream.
• For TCP or SCTP, an RCP Stream is also known as
  an RDMAP Stream.
• For iSER/TCP, the association is created when the
  connection transitions to iSER-assisted mode
  following a successful Login Phase during which
  iSER support is negotiated.
• Needed since SCTP and IB start their RCP stream
  mode at connection time

8
Examples of changes (4)

• The term RCP Message should be defined and used
  as a replacement for the term RDMAP Message the
  definition will be
• RCP Message The sequence of packets of the
  RDMA-capable protocol which represent a single
  RDMA operation or a part of RDMA Read Operation.
  For TCP or SCTP, an RCP Message is also known as
  an RDMAP Message
• When discussing the iSER Hello and HelloReply
  Messages the term "iSER Message" will be used
• Instead of RDMAP Message
• This distinction is needed in order to
  accommodate LLPs that have native message
  delivery capability, such as SCTP or IB

9
Example of Changes (5)

• We permit the iSCSI layer (if appropriate) to use
  the RCP message mode capability immediately after
  connection establishment before enabling
  iSER-assisted mode
• Appropriate for SCTP or IB
• In this case the iSER Hello and HelloReply
  Messages are not the first RCP Messages, but they
  are the first iSER Messages
• Added a discussion of connection establishment
  along with the use of the RCP messaging
  protocol, for exchanging Login Request and Login
  Response Messages
• Examples are discussed that are appropriate for
  SCTP and IB, along with the transitioning of the
  connection to iSER mode

10
Examples of changes (6)

• The discussion of Security specifies that all non
  IP protocols will define their own requirements
  for IPsec
• However the iSCSI requirements for IPsec are
  still required
• Wherever an iSER Message enters an IP environment
  from a non IP one (such as IB)
• The iSCSI/iSER requirement for IPsec on IP based
  protocols such as TCP and SCTP
• Will continue to require IPsec as a must implement

11
Generic example of iSCSI/iSER layering in Full
Feature Mode
12
IB Informative Section

• Information on how iSER would be used in an IB
  network is included in the Draft
• Not intended to be included in the iSER draft
  unless included in an informational Appendix
• Various Network topologies are shown
• Host Side IB Network, including Gateways
• Storage Side that includes iSER/IB
• IB iSER discovery process is described
• Use of IP over IB (IPoIB)
• SendTargets
• SLP
• iSNS
• Conversion of IP address into IB GID via ARP
  processes
• Discussion of what the IBTA needs to define

13
Host Side iSER/IB Topology
14
Storage Side with iSER/IB
15
Info What the IBTA needs to define

• Means for permitting a Host to establish an
  iSCSI/iSER connection with a peer InfiniBand
  end-node
• Indicating when that end node does not support
  iSER
• So the Host would be able to fall back to
  iSCSI/TCP over IPoIB
• Means for permitting the Host to establish
  connections with
• IB iSER connections on Storage Controllers
• or
• IB iSER connected Gateways in preference to IPoIB
  connected Gateways/Bridges
• or
• Connections to Target Storage Controllers that
  accept iSCSI via IPoIB
• How to operate in an environment where ZBTO, and
  SendInvSE are optional
• iSER ServiceID
• How the ServiceID can be added to the IP port
  number during the connection process

16
Info Key Implementation areas that do NOT
require IBTA specification

• How implementations determine which iSCSI/iSER
  portal group to use
• Basing the decision on new information that may
  be placed in the iSCSI discovery information is
• Not required
• Simple trial selection is acceptable
• How implementations determine how to best handle
  the concept of MC/S as it deals with multiple IB
  AddressesPorts per Portal Group

17
Preferred Change to Discovery Data

• Useful to have a connection type associated with
  the Portal Group Tag
• Will permit the most appropriate connections to
  be made without needless connection tries and
  failures
• Useful for iSER/iWARP/TCP
• Useful for iSER/iWARP/SCTP
• Useful for iSER/IB
• Proposed Syntax
• IP AddressPort, PG, Type

Portal Group Type IANA Portal Group Type
Value iSCSI 0, or blank iSER/iWARP/TCP
1 iSER/iWARP/SCTP 2 iSER/iWARP/IB 3
18
Recommendations

• Have Mike Ko update the iSER wordage as indicated
  by this draft
• Include the IB sections as Informational
  Appendices in iSER draft
• Send Discovery Data changes to the authors of
  iSCSI, iSNS, SLP for iSCSI