Title: Storage WideArea Networks SWANs
1Storage Wide-Area Networks(SWANs)
- Randy H. Katz
- Computer Science Division
- Electrical Engineering and Computer Science
Department - University of California, Berkeley
- Berkeley, CA 94720-1776
2Storage Networks
- Primary goal is to share storage among computers
in complex, heterogeneous environments, with PCs,
workstations, file servers, and mainframes - Storage can be
- Direct attached (e.g., host bus adapter/HBA)
- Network attached (via a file server)
- Channel attached (primarily fibre channel, but
also IBM SSA)
3Storage Networks
E.g, Hitachi, EMC, IBM Storage Arrays
Fibre Channel Loops Dual Ported Disks And
Controllers
Remote Storage Manager
DI
DI
DI
DI
Cache
Workstation
Cache
Mainframe
Crossbar Interconnect
Cache
Fibre Channel Or ESCON
Cache
HI
HI
HI
HI
Multiple Host Interconnections
4Storage Networks
- Major development in 1990s storage networks
- Native FC is a point-to-point or
loop/string-oriented (arbitrated) method of
interconnection 1 Gbps, extend up to 10 km - Emergence of FC fabrics FC switches arranged
hierarchically to enable connectivity between any
host and any storage device (e.g., Brocade
Networks dominates this product space) - Standardize FC protocol stack lives on top of
such fabrics device naming, transport, CoS, etc. - SCSI-3 protocol over FC fabrics
- Gigabit ethernet now emerging in SAN environment
5Basic Attached Storage
- Device attached by SCSI HBA or channel interface
- Host manages the file-to-block mapping
6Network-Attached Storage (NAS)aka Network File
Service
Host
NetworkInterface (NI)
Network File Server
Host
LAN
File Name, Offset, Length
OS
NetworkInterface (NI)
Host
NetworkInterface (NI)
- Mapping from File to Block done in
network-attached File Server, not host
7Network-Attached Secure Devices (NASD)
Gibson_at_CMU Research project on device embedded
protocol stack, authentication
Host
NetworkInterface (NI)
Network File Server
Host
LAN
File Name, Offset, Length
Disk, Cylinder, Track, Sector
OS
NetworkInterface (NI)
Host
NetworkInterface (NI)
Network-attached Secure Device (NASD)
8Storage PlatformsStorage Virtualization
Main Frame
LUN Logical Unit Logical disk mapping onto
underlying physical disks on logical block to
physical block basis
LUN, Offset, Length
ChannelInterface
Main Frame
Disk Storage Subsystem
Work Station
LUN To PHY
9NAS
NAS distinguished by an exported Network File
System interface over a standard Local Area
Network-based transport
File Server
Host
NetworkInterface (NI)
Host
LAN
File Server
File Name, Offset, Length
NetworkInterface (NI)
Host
NetworkInterface (NI)
File Server
10NAS SAN
SAN distinguished by a block-oriented
interfaceUsually implemented across a
channel-oriented fabric
Main Frame
PHY Device, Cyl, Trk, Sector
File Server
Disk Storage Subsystem
Host
NetworkInterface (NI)
Channel Interface
CI
SAN
LUN, Offset, Length
Host
LAN
File Server
Tape Storage Subsystem
LUN, Offset, Length
File Name, Offset, Length
CI
CI
NetworkInterface (NI)
CI
Host
Optical Disk Storage Subsystem
NetworkInterface (NI)
File Server
Main Frame
11NAS SAN SWAN
Now extend the NAS or the SAN over a
wide-area network transport NOTE wide-area SAN
is new idea
12Shared StorageReference Model
Application
File/Record Subsystem
Host
Service Subsystem Discovery, Monitoring Resource
Mgmt, Configuration Security, Billing Redundancy
Mgmt, Back-up High Availability,
Fail-over Capacity Planning
Block Aggregation
Storage Domain
SAN
Device
Block Subsystem
13SAN Reference Model
Block-oriented SAN
Application
FS
File
Host-based
SAN
SAN-based
Block
Device-based
DA
14SAN Reference Model
Application
NAS Storage
Host
Host
LAN
FS
File
NAS
Host
SAN
Block
Device
15SAN Reference Model
Application
Heterogeneous Storage Environment
Host
Host
Host
Host s/w raid
LAN
FS
File
NAS
NAS Head
Host
SAN
SAN
Block
Device
DA
16Seven Layer Stack
IP NFS, CIFS FTP, SNMP, TFTP, Telnet, FCP,
SCSI-3 TCP, UDP IP LAN, MAN, WAN Phy
Sw GigaE NFS, CIFS FTP, SNMP, TFTP, Telnet, FCP,
SCSI-3 TCP, UDP IP Mac Client/Control Phy
FC SCSI-3 VI IP FC-4Protocol I/FFC-3
Encrypt/AuthenticationFC-2 Framing, FC,Class
of Service FC-1 Encoding,Link Control FC-0 Phy
7 6 5 4 3 2 1
Application Presentation Session Transport Network
Data Link Physical
17Fibre Channel Protocol Stack
- FC-0 gigabit physical layer
- FC-1 data encoding and link layer control
- FC-2 segmentation/reassembly of data frames,
flow control, class of service - FC-3 common services, e.g., encryption
- FC-4 upper layer protocol upon which SCSI-3 or
IP can run - Contrast with Gigabit Ethernet
- Ethernet framing, VLAN tagging, frame
prioritization (8 levels), link aggregation, 1.25
gbps - IP Gigabit Ethernet emerging for SANs
18SCSI-3 Terminology
Target
Initiator
Device Server
Request
LUNs
Application Client
SCSI Client-Server Model
Response
Delivery Subsystem (e.g., Fibre Channel orSerial
SCSI over IP)
19FC over IP (FCIP)
Tape Storage Subsystem
Tape Storage Subsystem
Server
Server
WAN
IP Network
FC Over IP
FC Switch
FC Switch
FC Over IP
Server
Server
JBOD
JBOD
Tunnel Session
- IETF IP Storage (IPS) working group
- Recall FC fabrics developed in context of machine
room/building-scale interconnect (e.g., no
congestion control!) - FC time outs in wide-area? Flow control
interaction? QoS? - How does bridging actually work for FC e2e
management?
20Internet FC Protocol (iFCP)
- Gateway to gateway protocol, sessions rather than
tunnels - TCP for congestion control, error detection,
recovery - Plug FC devices directly into iFCP switches
- Session and naming semantics
21iFCP Services
- 24-bit N-Port Address ltDomain, Area, Portgt
- iSNS Internet Storage Name Servicediscovery and
management protocol for IP storage networks
(IPNSP) - Protocol specification includes address
translation feature to allow remote storage
devices to be assigned a local, FC fabric
compliant address - Local commands executed locally on the fabric
- Remote commands executed on top of TCP
connections - Error Detection/Time Outs
- Security
22Other Protocols
- Metro Fibre Channel Protocol (mFCP)
- FCP over IP using UDP rather than TCP (link layer
flow control and pacing) - Internet SCSI (iSCSI)
- IP to the storage device
- Serial SCSI block data transfer over IP (SCSI
Access Method Command SetSAM) - IPSec, command/data ordering, steering to
application memory
23iSNS
- Discovery Process
- Device registration
- WWN or iSCSI names
- Zoning/discovery domains
- iSNS objects
- Portals
- Storage Port
- Storage Nodes
24Storage Applications
- Data Centers
- Server clustering
- Storage centralization, consolidation, management
- LAN-free back-up
- IP Storage for Remote Applications
- Remote back-up
- Remote mirroring
- Disaster recovery
- Content distribution
25Rhapsody Networks, Inc.
- Storage Application Director
- Alteon box for storage networks
- Peek into storage packets traversing fabric
(deep frame classification) and invoke codefor
encryption/decryption, mirroring, LUN mappings,
etc. - Per port software processing and cut-through
fabric routing - Data copy engine, table lookup engine, in-transit
I/O mods, data escrow/trap to software for
complex errors or event processing - Intelligent queue management