An End-to-End Approach to Globally Scalable Network Storage

1
An End-to-End Approach to Globally Scalable
Network Storage

• Presented in cs294-4 P2P Systems
• by
• Sailesh Krishnamurthy
• 15 October 2003

2
Logistical Networking

• Models sync/async aspects of communication
• Single fabric that unifies
• Data Storage
• Data Transportation
• Internet scalability goals
• Claim end-to-end design principles vital

3
Background SAN vs NAS

• Current trends in storage networking
• SAN - Storage Area Network
• NAS - Network Attached Storage

4
More on NAS v. SAN

• NAS
• Wires TCP/IP
• Protocol NFS, CIFS

• SAN
• Wires Fiber Channel
• Protocol Encapsulated SCSI

Where is the File System ?
5
Traditional networks

• General goals
• Minimize delay
• Minimize probability of corruption
• Maximize probability of delivery
• Assumptions in traditional storage nets
• When storage is closely coupled, delay and
  probability of corruption can be low while
  availability is high.

6
SANs cannot scale to SWANs

• In the SWAN, resources can be intermittently
  unavailable
• So we need e2e strategies
• Simple retries
• Redundant data accesses spread across nw
• High-latency archival backups

7
Correctness in the wild

• SANs are controlled environments and
  correctness is not an issue
• In the SWAN, data storage may not be reliable.
• Data accuracy must be checked by producers and
  consumers - at endpoints

8
SWAN Security

• SWANs are not physically localized
• SAN security assumptions dont hold
• Again, e2e approaches are required
• DoS is a gotcha for SWANs
• Cant be prevented with e2e strategies
• Imitate techniques for handling DoS in IP

9
Unbounded Size/Duration

• Since single store may not have all the resources
  all the time, the endpoint has to manage
  distribution of data
• Unbounded duration allocation hurts resource
  sharing.
• Should this be managed at endpoints ?

10
Logistical Networking

• Storage Networking
• IP networks interconnection fabric of storage
  pool
• Logistical Networking
• Storage part of the networking infrastructure
• Shared resource fabric exposing storage resources
• Similar to how internet exposes bandwidth
  resources
• Storage Stack
• Bottom-up, layered e2e design approach
• Internet Backplane Protocol (IBP)

11
Storage Stack
Applications
exNode tools and services
exNode A data structure for aggregating network storage
IBP Allocation and management of storage on network storage depots
Local Access
Physical
12
IBP - Internet Backplane Protocol

• First layer of stack thats globally accessible
• Abstracts access layer resources (file/block
  storage services)
• Expose underlying storage resources to maximize
  freedom at higher levels
• Implement only indispensable common functions
• Enable scalable internet style resource sharing
• Mask peculiarities of access layer resource
• Abstract service based on data blocks that are
  managed as byte arrays

13
IP vs IBP

• IP vs link layer
• Agg. of link layer packets masks packet size
  limits
• Simple fault detection - faulty datagrams dropped
• Global addressing masks diffs b/w LANs
• IP Property
• Any participant of a routed IP n/w can use any
  link layer connection

• IBP byte array indep.
• Agg. access layer blocks masks fixed block size
• Simple fault detection - drop faulty byte arrays
• Global addressing (IP) maks diff b/w acc layer
• IBP Property
• Any participant of an IBP n/w can use any access
  layer storage resource

14
Issues with IBP

• DoS vulnerability is much worse
• In IP
• DoS attacks require constant sending of data
• Does not profit the attacker in any way
• In IBP
• Once data block is allocated it remains used
• Using remote storage does benefit the attacker
• Strong semantics (reliability) of traditional
  storage/SAN are difficult to implement in the SWAN

15
IBP Solutions

• Time-limited storage allocations
• When a lease expires, the storage can be reused
  for some other user
• Soft storage semantics in IBP
• IBP is a best-effort service
• Allocated storage can be revoked at any time

Storage Management Data Transfer Depot Management
IBP_Allocate, IBP_Manage IBP_Store,IBP_Load, IBP_Copy,IBP_mcopy IBP_Status
16
exNode - Flexible Aggregation of Network Storage

• Implement abstractions w/ strong properties
• Higher layer construct
• Aggregates primitive IBP byte-arrays
• Need to maintain state that represents the agg.
• exNode aggregates IBP byte-arrays as the Unix
  inode aggs. disk blocks

17
e2e services for storage

• exNode can hold additional metadata for services
• Redundancy
• Framing of data into segments w/ checksums
• exNode is analogous to the state of a TCP
  connection, data on disk analogue of a TCP stream

18
Relation to p2p systems

• Paper compares with Napster/Gnutella
• In file sharing all allocations are at endpoints
  .. leads to large data transfers
• Appropriate comparison is Oceanstore ?
• My view
• exNode infrastructure is a way to create storage
  services from smaller blocks
• Can be useful in an Oceanstore-like setting
• Can alleviate some SAN shortcomings ?