Testing the Zambeel Aztera

1
Testing the Zambeel Aztera

• Chris Brew
• Fermilab
• CD/CSS/SCS
• Caveat This is very much a work in progress. The
  results presented are from jobs run in the last
  two days and have not been fully checked for
  validity.

2
Aim

• To test the overall performance of the Zambeel
  Aztera Network Attached Storage System
• To develop suite of tests that can be used for
  comparisons between different NAS devices

3
What is it?

• From a user point of view its just a large
  black box network attached NFS server serving
  out a 2.8TB filesystem
• Behind the scenes it consists of multiple
  compute nodes and storage nodes. Compute
  nodes interface between the storage and the
  clients making multiple disks across multiple
  nodes appear as one (or more) filesystems. Load
  balancers share the load amongst the compute
  nodes. Very redundant - every component is
  duplicated, data is mirrored and striped across
  strorage nodes.

4
Tests

• Manageability Tests
• Ease of setup
• Ease to Reconfigure
• Performance Tests
• Max Throughput Read and Write
• Max Throughput Reading a Single File
• Simultaneous Reads and Writes
• Creation, Listing and Deletion of Large Numbers
  of Small Files
• Data Integrity
• Failure Tests
• Fail various parts of the system and see what
  happens

5
Tools

• Reader/Writer
• Fermi Built tools that write and read a file
• IOZone
• Standard Benchmark tool with many options. Used
  here for the data integrity tests.
• Bonnie
• Another standard benchmark tool. Used for its
  File Creation/Deletion test.

6
Why Three Tools?

• All have different Strengths/Weaknesses
• Reader/Writer Very Simple and Easy to script
  around
• IOZone Many Many modes
• Bonnie Only one with file creation/deletion test

7
Performance Tests So Far

• All tests use the new hotdog nodes (Dual 1.6 GHz,
  1GB Memory, 100Mb/s Network)
• Completed Tests
• Max Throughput (n clients with 1 or 2 processes
  writing/reading 1GB files. Rate for all files
  completed before the first node to complete 5
  files are averaged on each node and summed across
  the nodes)
• Single File Throughput (same as previous read
  test but all nodes/processes read the same file)
• Client comparison (Measure the single client
  read/write speeds for different clients)

8
Maximum Throughput
9
Tuning Changes Throughput
10
Different Clients/Different Performance
11
Reading a Single File
12
Failure Testing

• Whilst the system was up and clients were reading
  and writing data, removed and replaced one by
  one
• Compute Module
• Storage Module
• Load Balancer
• Performance was obviously degraded but the system
  stayed up and serving its clients

13
Conclusions

• For 50 Clients the system gives a total
  throughput of 120Mb/s for writes and 220Mb/s
  for reads
• Reliability seems to be high. It can survive any
  single component failure
• With these tools we have the basis of a test
  suite and procedure for comparing the different
  storage technologies that are becoming available