Technology Trends

1
Technology Trends

• IP Storage Networking

2
Agenda

• IP Storage Networking
• History
• Protocols
• Initiators
• IP Storage Networking Solutions

3
IP Storage Networking

• Analyst Perspective"It's not a NAS versus SAN
  worldit's both. iSCSI is a way to utilize
  existing, inexpensive infrastructure to carry
  block storage. There's no question that iSCSI
  enables networked storage for the masses.
• Steve Duplessie, Founder, Enterprise Storage
  Group
• Manufacturer Perspective"Its important to
  understand that iSCSI is not about replacing
  Fibre Channel. However, customers do want to
  extend the benefits of storage networking to
  parts of their infrastructure where SAN
  technology has not been economically feasible.
  Thats where the majority of their data is. And
  thats where iSCSI comes in.
• - Dave Hitz, Founder and EVP, NetApp

4
IP Storage Networking History

• NAS was first
• File-oriented services providing cross-platform
  access
• Not appropriate for applications that require
  block-level access
• Databases such as Oracle and MS Exchange
• Fibre Channel
• developed to solve problems of traditional
  channel protocols (SCSI, ESCON)
• Expensive and requires specialized expertise

5
IP Storage Networking History

• IP-based Storage Networking developed to
• Leverage the ubiquity of Ethernet and IP
• Reduce the cost and complexity of FC
• Foundation technologies
• Ethernet IEEE 802.3
• IP IETF through RFCs
• SCSI NCITS T10 Committee (T11 for FC)

6
IP Storage Networking History

• False-start in 2000
• IBM and Cisco pushed it
• IBM had the i200
• Why was it delayed?
• Fibre Channel turned out to be very good
• A requirement for IPSec was added to the
  specification and it took some time to finish
• Protocols developed by IP Storage Forum of IETF
• FCIP, iFCP, iSCSI
• All 3 ratified in Jan/Feb 2003
• All used to transport SCSI protocol commands

7
FCIP

• Fibre Channel over IP
• First on the scene but interest declining
• Allows SAN-to-SAN connectivity by providing a FC
  tunnel over an IP network (LAN/WAN)
• TCP Protocol is required
• LAN/WAN is invisible to SAN
• SAN is invisible to LAN/WAN
• Problems in LAN/WAN may trigger FC error handling
  routines
• Time-Out is a problem and getting the TOV values
  right is difficult
• FCIP devices responsible for re-establishing
  broken links

Makes it easy to implement
8
FCIPtunneling between SANs
Server
Server
Server
Server
Server
Server
Tunnel Session
IP WAN
FCIP Device
FCIP Device
FC Switch
FC Switch
FC Tape
FC Tape
FC Disk
FC Disk

• CNT is a primary provider
• Can be implemented as
• Stand-alone system
• IP Router Blade
• FC Switch Port

Source IP SANs by Tom Clark, Pg 127
9
iFCP

• Gateway to gateway protocol for providing FC
  Services to FC devices over an IP network
• Can be used in place of FC fabrics
• Why it was not embraced by FC Switch Vendors
• Devices plug into F-Ports in iFCP switches
• Uses TCP and Translates FC Address (24-bit) into
  IP Address
• IP network problems only affect the 2 devices
  communicating as opposed to the entire fabric as
  with FCIP
• Nishan (McData) developed one of the few
  commercially successful products based on iFCP
• Check out
• http//www.snia.org/tech_activities/ip_storage/iFC
  P_user_overview.pdf

10
iFCP a fabric example
Fibre Channel Traffic
Fibre Channel Device Address
Fibre Channel Device Domain
Control Data
IP Address Mapping
IP Fabric
iFCP Switch
iFCP Switch
iFCP Frames
IP Network
iFCP Gateway Region
iFCP Gateway Region
Source IP SANs by Tom Clark, Pg 127
11
mFCP

• UDP/IP variant of iFCP
• Does not use flow-controls of the TCP layer
• Equivalent to FC Class 3 Service
• Increased performance over iFCP
• Possible to use mFCP within the Data Center
  (where network is clean) and iFCP between Data
  Centers
• Contributed as Open Source by Nishan Systems

12
iSCSI

• Eliminates FC from the storage networking
  equation
• End-nodes implement SCSI for block data transfer
  over IP networks
• Eliminates intelligent switches
• End-nodes responsible for connection
  establishment and integrity
• Many features built into protocol to accommodate
  IP networks
• Error handling, security
• Very few extensions to the standard SCSI protocol

13
iSCSI Protocol Stack
Initiator
Target
SCSI
SCSI
SCSI Application Protocol
iSCSI
iSCSI
iSCSI Protocol
Data Sync
Data Sync
TCP
TCP
TCP/IP Protocol
Lower Level Function (IPSec)
Lower Level Function (IPSec)
IP
IP
Link
Link
IP Network
Source IP SANs by Tom Clark, Pg 141
14
iSCSI Naming Conventions

• Nodes on the network are known by its IP
  Address/TCP Port Combo
• Network Portal
• Nodes also have a human-readable name iSCSI
  Node Name
• fqn.com.ramjack.bigarray.research.30221
• can have an alias
• WWN convention accommodated by IEEE extended
  unique identifier (eiu)
• eiu.0300732A32590D26

15
iSNS

• Internet Storage Name Server
• Used by iSCSI and iFCP
• Combination of FC and IP discovery techniques
• FC SNS in switches, zoning, state change
  notifications (SCN)
• IP DNS, SLP LDAP
• Discovery Domain Sets
• Similar to FC Zoning
• Can also keep and exchange public and private
  keys used by nodes (X.509)
• Implemented in Centralized iSNS servers, IP
  Storage Switches and Target Devices
• Nishan supplied iSNS protocol as Open Source
• Currently in final stages of ratification by IETF

16
IP Storage Networking Security Quality of
Service Implementations

• Security
• Discovery Domains
• Equivalent to Soft Zoning in FC
• Access Control Lists
• LUN Masking
• VLAN Tagging
• IPSec, CHAP, SRP, Kerberos, SPKM
• Quality of Service
• Traffic Prioritization
• Type of Service (TOS IP header field)
• Differentiated Service (DiffServ)
• Resource Reservation Protocol (RSVP)
• Multiprotocol Label Switching (MPLS)

17
iSCSI InitiatorsSoftware-based

• Best for lightly loaded servers generating lt40MBs
• Supplied by OS
• W2K, WXP, W2K3, NetWare, Redhat
• Microsofts support in June 2003 accelerated
  acceptance
• Supplied by Cisco for Solaris, AIX, HP-UX
• PYX Technologies supplies advanced iSCSI software
  stack with Level 2 Error Recovery

18
iSCSI InitiatorsSoftware-based

• Uses standard NIC
• Makes implementation practically free on the
  Server
• Downside
• iSCSI and TCP processing done by Host CPU
• Can be significant
• Requires workaround for Windows Dynamic Disks

19
iSCSI InitiatorsSoftware performance
considerations

• Cisco Tests
• 1) MS Exchange with 4000 users
• Generated 20MBs server to storage thruput (no
  problem!)
• 2) OLTP/DW environment with 150 users
• Performed the same as Fibre Channel
• Sizing
• Network access 1MBs thruput uses 8Mhz processor
• Storage access 1MBs thruput uses 5Mhz processor
• Example Backup server moving from network to
  tape drive at 30MBs each way
• Requires 390Mhz
• Tape 30 5 150Mhz
• Network 30 8 240Mhz
• To achieve full duplex Gig-E, consumes full 2Ghz
  processor

20
iSCSI InitiatorsHardware-based

• Best for strained servers
• Offloads TCP/IP processing to Adapter
• TCP Offload Engine (TOE) - TNIC
• General purpose not necessarily optimized for
  Storage
• Microsoft does not yet approve
• Offloads iSCSI and TCP/IP processing to Adapters
• Storage NIC (SNIC)
• Microsoft iSCSI-Ready Branding program
• Most adapter vendors building TNICs and SNICs
• Capable of providing wire-speeds when other
  bottlenecks are eliminated

21
iSCSI InitiatorsHardware Performance
considerations

• Adaptec Tests
• Generated 130MBs, full duplex using 64kb I/O
• Consumed only 5 CPU on 1Ghz processor
• iReady Tests
• Used IOMeter test suite on Pentium II with 1500
  byte I/O
• Hardware Initiator 30000 IOPS _at_ 75 CPU
• Software Initiator 12000 IOPS _at_ 90 CPU

22
iSCSI InitiatorsSummary
Host
Application
Operating System
SCSI
iSCSI
Storage NIC (SNIC)
TCP/IP
TCP Offload Engine (TOE) TNIC
Ethernet
Standard NIC
Software Initiators
Hardware Initiators
23
IP Storage NetworkingSolutions

• iSCSI Storage Router
• Bridge iSCSI to Fibre Channel
• No virtualization of the storage
• Best in environments with an existing FC SAN
• HP StorageWorks SR2122
• 2 iSCSI 2 FC ports
• Cisco SN5420 SN5428
• 2 iSCSI 8 FC ports

iSCSI Server W2K
iSCSI Server Solaris
iSCSI Server Linux
iSCSI Ethernet
Ethernet Switch
Storage Router
Fibre Channel
FC Tape
FC Disk
24
IP Storage NetworkingSolutions

• iSCSI Storage Bridge
• Bridge iSCSI to SCSI
• Good to get SCSI tape drives into IP SAN
• Has to bridge SCSI-2 (tape) to SCSI-3 (iSCSI)
• ATTO IPBridge 2500 C/R/D
• Mix of iSCSI and SCSI ports

iSCSI Server W2K
iSCSI Server Solaris
NAS Server
iSCSI or NDMP Ethernet
Ethernet Switch
Storage Bridge
SCSI
SCSI Tape
SCSI Disk
25
IP Storage NetworkingSolutions

• iSCSI Storage Concentrator
• Bridge plus Storage Provisioning appliance
• Serves volumes to iSCSI servers from SCSI or FC
  Disk Arrays
• Uses
• Allows stranded servers to participate in SAN
• provides a way to reuse older JBOD or other Disk
  Arrays
• StoneFlys i3000 Storage Concentrator
• Mix of iSCSI, SCSI and FC ports

iSCSI Server W2K
iSCSI Server Solaris
iSCSI Server Linux
iSCSI Ethernet
Ethernet Switch
Logical Disk
Storage Concentrator
SCSI or FC
SCSI or FC Disk
Physical Disk
26
IP Storage NetworkingSolutions

• iSCSI Storage Switch
• Ive tried my hardest, but I just cannot figure
  out what this thing is
• Its a combo
• Ethernet switch
• Fibre channel switch
• Bridge and/or Router
• Storage Virtualization engine
• SANRAD V-Switch 3000

27
IP Storage NetworkingSolutions

• iSCSI Storage Array
• Just like todays SCSI or FC Arrays
• 14 SATA drives
• Dual iSCSI controllers (active-passive)
• Software for
• Provisioning, SnapShot, Replication, etc.
• Good for net new storage implementations
• EqualLogic PeerStorage Array

iSCSI Server W2K
iSCSI Server Netware
iSCSI Server Linux
Ethernet Switch
iSCSI Ethernet
iSCSI Disk
28
IP Storage NetworkingSolutions

• iSCSI Tape Library
• Limited deployment to date
• SpectraLogic first to market
• SpectraLogic 2K Tape Library

From http//www.spectralogic.com/index.cfm?fuseac
tionproducts.showContentAndChildrenCatID153
29
IP Storage NetworkingSolutions
iSCSI Server W2K
iSCSI Server Netware
Standard File Server

• iSCSI and NAS
• NetApp supports iSCSI as a free download for
  their F800 and FAS900
• Allows for block-level support
• Databases
• Oracle and Exchange

Ethernet Switch
Ethernet
iSCSI Block- Level traffic
IP File - Level traffic
NAS Filer
SCSI or FC
SCSI or FC Disk
30
IP Storage NetworkingSolutions

• IP on Fibre Channel Switches/Directors
• Cisco
• IP Storage Services Module on MDS 9000 Series
• 8-port card supports both iSCSI and FCIP
  simultaneously on each Gigabit Ethernet port
• McData
• Acquired Nishan and is marketing their edge
  devices for iFCP
• Did not find any plans to integrate into
  directors/switches
• Brocade
• Developing Multiprotocol fabric routing services
  that will allow for iSCSI to FC bridging and FCIP
• CNT
• Using Edge Router for FCIP

31
IP Storage NetworkingSolutions

• iSCSI on Enterprise-class Arrays
• HDS has announced iSCSI blade for 9900V series
  arrays
• Will provide for TrueCopy over IP
• Partnering with McData
• EMC already provides iSCSI connectivity across
  their Symmetrix DMX line
• http//www.emc.com/news/press_releases/view.jsp?id
  1744
• IBM had the TotalStorage IP Storage 200i until it
  killed it in 2002
• Teaming with Cisco

32
IP Storage NetworkingSolutions

• iSCSI for BladeServers
• Ideal solution for this type of environment
• since often, each blade has only an ethernet port
  and a small, on-board hard drive.
• Sun has blessed the EqualLogic PeerStorage Array
  for the B1600 Blade Platform
• Allows for booting from IP SAN
• Without iSCSI, BladeServers relegated to using
  NAS
• At least Suns was..

33
iSCSI Best Practices

• Start with non-mission critical application
• Put iSCSI at the edge and FC in the core
• For an enterprise-class environment
• Create a separate IP SAN from regular LAN
• For Small/Medium Businesses, can skip FC for
  Storage Consolidation and use iSCSI
• Decide early who is responsible for IP SAN

34
iSCSI Resources

• http//www.iscsistorage.com
• http//www.ietf.org/html.charters/ips-charter.html
  
• http//www.snia.org/tech_activities/ip_storage
• http//www.iol.unh.edu/consortiums/iscsi