iSCSI/RDMA:%20Overview%20of%20DA%20and%20iSER

1
iSCSI/RDMAOverview of DA and iSER

• Mallikarjun Chadalapaka
• HP

2
Agenda

• Motivation for iSCSI/RDMA
• iSER and DA?
• DA Approach, why, principles
• iSER big picture
• SCSI Read
• SCSI Write
• Next steps

3
Motivation for iSCSI/RDMA

• Motivation
• Take advantage of the RDMA protocol suite RNIC
  hardware.
• When RDMA technology advances, so does iSCSI.
• Zero-copy, direct data placement can be achieved
  with generic protocols.
• Opportunity for iSCSI protocol to offload some
  complexity
• iWARP offers CRC-level reliability on TCP/IP.
• iSCSI will be run on RNICs anyway, lets make it
  optimal.
• High-level goal Run most efficiently on
  generic RNICs.

4
So what did we do?

• 2 discrete steps
• Distill the data movement needs of iSCSI to
  propose an architecture for any Datamover.
• DA (Datamover Architecture for iSCSI).
• Based on the architecture, instantiate a
  Datamover protocol on RDMAP.
• iSER (iSCSI Extensions for RDMA).

5
Why DA?

• Principle iSCSI is assisted by a Datamover
  protocol.
• But by how much?
• Are sessions offloaded?
• How about login?
• How about discovery?
• DA draws a line through the iSCSI protocol
  functionality
• Data movement below, everything else above.
• Everything below the line is assisted by a
  Datamover.
• The line is defined by Operational Primitives.

6
Datamover Architecture
Datamover Layer, Protocol, Interface
iSCSI layer
iSCSI layer
Datamover Interface (DI)
Datamover layer
Datamover layer
Datamover protocol
Datamover-dependent
TCP layer
TCP layer
7
DA Design principles

• Not a wire protocol spec.
• A functional model of interactions within an
  iSCSI node.
• Agnostic about hardware or software
  implementations.
• Independent of iSCSI revision specifics, or of
  any Datamover protocol.
• Motivated by RDMA but designed not just for iWARP.

8
iSER Big picture
A TCP-based implementation is shown as an example.
SCSI

• Layered on RDMAP protocol.
• iSCSI still requires direct TCP access for
  bootstrapping (login).
• Transition to RDMA mode along with FFP.
• Same iSCSI well-known port
• Same discovery.

iSCSI
Datamover Interface
1
2
iSER
iWARP Verbs
RDMAP
DDP
Generic RDMA over TCP/IP
iWARP protocol suite
MPA
TCP
RNIC
9
iSCSI/iSER Port to Port (FFP)
Initiator
Target
Initiator
Target

SCSI target port
SCSI initiator port
SCSI target port
SCSI initiator port



I_T Nexus
I_T Nexus
SCSI
SCSI
I_T Nexus
I_T Nexus
iSCSI Session
iSCSI Session
iSCSI Connection
iSCSI Connection
iSCSI Connection
iSCSI
iSCSI Connection
iSCSI Session
iSCSI Session
iSCSI Connection
iSCSI Connection
iSCSI
iSCSI Connection
iSCSI Connection
TCP connection
TCP connection
DI
iSCSI/iSER connection
iSCSI/iSER connection
iSCSI/iSER connection
iSCSI/iSER connection
iSER
Verbs
RDMAP Stream
RDMAP Stream
RDMAP Stream
RDMAP
RDMAP Stream
DDP Stream
DDP Stream
DDP Stream
DDP Stream
DDP MPA
TCP connection
TCP connection
10
SCSI Read
Initiator
Target
Command encapsulated in a Send Type Message
(Advertises initiators STag for the SCSI buffer)
SCSI Command PDU
Command Request
SCSI Data-in PDU
RDMA Write Message
Read data back to the host
Data transfer (all associated RDMA Writes
complete)
SCSI Response PDU
SCSI Response PDU Encapsulated in a Send Type
Message
Command Complete
11
SCSI Write
Initiator
Target
Command encapsulated in a Send Type Message
(Advertises initiators STag for the SCSI buffer)
SCSI Command PDU
Command Request
R2T PDU
RDMA Read Request Message
Write data from the host
Data transfer (all associated RDMA Reads complete)
SCSI Response PDU
Response encapsulated in a Send Type Message
Command Complete
12
Dependencies Non-dependencies

• DA is dependent only on iSCSI.
• iSER is dependent on DA iSCSI.
• iSER is dependent only on the RDMAP protocol
  among the iWARP suite.
• iSER DA are not dependent on the RDMA Verbs.

13
Now Next steps

• I-Ds submitted in July 2003.
• Now in their -02 revisions.
• http//www.ietf.org/internet-drafts/draft-chadalap
  aka-iwarp-da-02.txt
• http//www.ietf.org/internet-drafts/draft-ko-iwarp
  -iser-02.txt
• Discussion on the IPS (and some on RDDP)
  reflector.
• Proposal is to make DA and iSER drafts as the
  official IPS WG drafts, to eventually become
  standards-track RFCs.