DAFS Techwave 2002

1
E128 Implementing ASE on Network Appliance
Filers Using the Direct Access File System
Stephen Daniel Database Performance
Architect Network Appliance Steve.Daniel_at_netapp.co
m
2
Storage Options

• Storage for large databases blocks or files?
• Advantages of Files
• Easier to manage
• Create
• Spread across multiple volumes
• Grow
• Copy

3
Storage Options

• Files Provide Excellent Storage Virtualization

Servers
SAN
Common Storage Pool
4
Storage Options

• Storage for large databases blocks or files?
• Advantage of Block Storage (raw partitions)
• Speed

5
Manageability of Files with the Performance of
Blocks

• Why are files slow?
• Local File systems
• Code path length
• Network File Systems
• Code path length
• Data copies
• Data copying is required by traditional network
  protocols

6
Manageability of Files with the Performance of
Blocks

• Components of a Solution
• Manageability of files
• Use an external storage system
• Efficient sharing of data
• Virtualization of the storage pool
• Offload file system code path

7
Manageability of Files with the Performance of
Blocks

• Components of a Solution
• Performance of Blocks
• Use new RDMA-capable hardware
• Allows zero-copy networking
• Offloads network protocol stack
• New hardware paradigm, RDMA, requires a new
  software stack

8
Remote DMA Technology

• Remote DMA (RDMA) Technology

LAN or SAN
Data flows from user memory to storage system
buffers without any data copies
9
Using RDMA VI Interconnect

• VI Messaging Protocol
• Protocol created and promoted by Intel
• Provides end-to-end management of connections
  based on RDMA capability
• Typically used for cluster interconnects

10
Direct Access File System

• What is DAFS
• DAFS is a protocol for a network file system
• Provides a rich API for managing complex I/O
  streams
• Similar in function to NFS version 4
• DAFS is designed around RDMA technologies
• Layered on top of messaging protocol, such as VI

11
DAFS Industry Initiative

• DAFS Collaborative
• 87 companies 9 primary spokespersons
• DAFS Prototypes developed by 4 companies, 3
  universities
• DAFS Protocol Spec v1.0 published September 2001
• DAFS API Spec v1.0 published November 2001
• Industry initiative transitioning to next stage
• IETF (protocol)
• DAT Collaborative (RDMA transport APIs)
• DAFS Implementers Forum in SNIA

12
DAFS Clients

• Three Styles of DAFS Client
• Full network file system fDAFS
• Integrated into a kernel, just like NFS
• Can provide an efficient direct I/O path
• Applications must use standard system calls

13
DAFS Clients

• Three Styles of DAFS Client
• User-level I/O library uDAFS
• Full implementation of the DAFS API defined by
  the DAFS collaborative
• Reduced dependency on system calls
• Dramatically reduced CPU cost of I/O compared to
  any other form of kernel-mediated I/O, including
  raw disk partitions

14
DAFS Clients

• Three Styles of DAFS Client
• Kernel-level device driver dDAFS
• Functionally a disk device driver
• A DAFS file appears as a raw partition in /dev
• Provides best time-to-market DAFS client

15
DAFS Products

• Current DAFS Technology Choices
• DAFS / VI / IP / Ethernet
• DAFS / Infiniband
• The examples in this talk
• DAFS / VI / IP / Ethernet / Solaris 8

16
DAFS Software Architecture
Runs on the Emulex NIC
17
DAFS Hardware Architecture
Database Server
Parallel paths for performance and failover
Conventional gigabit Ethernet with IP protocol
Multi-protocol capable storage devices
18
DAFS Performance Testing
Database Server CPU msec/op
113
Direct-attached storage w/ volume manager (VxVM)
89
Direct-attached storage w/ local FS (ufs)
76
Raw access to direct-attached storage
70
dDAFS kernel device driver (w/ VI/IP HBA)
28
User-level uDAFS client (w/ VI/IP HBA)

• Sun E3500 (400MHz) w/ Solaris 2.8
• Mixed read/write workload
• 4kB transfers asynchronous I/O

19
TPC-C Performance Testing

• Products Used
• Sybase ASE 12.5.0.1
• Fujitsu-Siemens Prime Power 850 with 16 CPUs
• Solaris version 8
• Network Appliance DDA 1.0 dDAFS driver
• Emulex GN9000 VI HBAs
• Network Appliance F820 storage systems

20
TPC-C Performance Testing

• Manageability of Files
• Managed the database using NFS
• Build
• Copy
• Resize

21
TPC-C Performance Testing

• Manageability of Files
• Network Appliance data management
• Snapshots to protect data
• Recover from crashes
• Undo configuration changes

22
TPC-C Performance Testing

• Performance of Blocks
• Ran the database using dDAFS
• To convert from NFS to dDAFS
• Shutdown ASE
• Rename the NFS files
• ddafs open the NFS files as ddafs devices
• Bring up ASE

23
TPC-C Performance Testing
Most comparable result...
24
TPC-C Performance Testing
25
TPC-C Results

• The Numbers
• 112,286.46 tpmC
• 13.44 / tpmC
• Published March 13, 2002
• Available August 31, 2002

26
TPC-C Results

• Records (as of publication date)
• Best price/performance on UNIX System
• Best performance/CPU-MHz on SPARC
• NetApp Features enabled during TPC-C
• All storage was RAID protected
• 5 Snapshots were active

27
Network Appliance

• Network Appliance enables enterprises to
  simplify, share, scale and manage their
  information infrastructure.