Exploring Parallel IO for Multithreaded Applications using Posix Threads

1
Exploring Parallel I/O for Multi-threaded
Applications using Posix Threads

• Kshitij Mehta and Edgar Gabriel

2
Motivation

• I/O often a bottleneck in compute-intensive
  applications
• I/O approaches in multi-threaded applications
• one thread writes/reads data
• Bad performance
• Each thread writes/reads data to/from its own
  file
• Requires a post-processing step
• Each thread writes/reads data from a common file
• Bad performance due to contention

3
Goal of this study

• Goal of this study explore alternative
  interfaces and implementations that
• Overcome the limitations of the two approaches
  presented on the previous slide
• Allow a group of threads to perform I/O
  operations on the same file efficiently

4
Data Distribution

• Typical Data distributions
• Contiguous in memory, contiguous in file
• Contiguous in memory, noncontiguous in file
• Noncontiguous in memory, contiguous in file
• Noncontiguous in memory, noncontiguous in file

5
Disk Striping

• Stripe factor no. of storage servers
• Stripe size size of each data chunk on a server

6
I/O interfaces

• Implicit file pointer management vs. explicit
  file offset interfaces
• read()/write()
• pread()/pwrite()
• List I/O interfaces
• More work can be done per function call
• readv()/writev()
• Individual vs. Collective I/O
• Collective I/O allows for optimizations

7
Collective I/O interface options

• Option 1 all threads pass in exactly the same
  arguments for
• ltbuffer pointer, data size, file offset
  (optional)gt
• Advantage library is aware that it can use
  multiple threads for execution
• Option 2 allow each thread to provide a
  different ltbuffer pointer, data size, offsetgt
  argument
• Library can check whether data from different
  threads are contiguous in memory

8
Optimizations - Merging of buffers
9
Internal optimization

• Parallel file systems (PVFS2, Lustre, GPFS etc.)
  have a large number of I/O servers ( e.g. disks)
• Each (large) file is striped across multiple
  disks
• Some file systems allow direct I/O to storage
  servers
• Data is written in chunks of size stripe size
• Understand the relationship between size of data
  in an I/O call and stripe size

10
UH_Pthread_Coll_IO (I)

• Library of C functions
• Threads read/write data to the same file
• Perform merging of buffers
• Categories of interfaces
• mem_merge, all_merge
• individual bufptr interface, list I/O interface
• mem_merge contiguity in memory
• all_merge contiguity in both memory and file

11
Algorithm
12
Tests

• Tests performed on a shark node having two 2.2
  GHz quad core AMD Opteron processors (8 cores
  total)
• 8 GB main memory
• Test parameters
• Number of threads 1, 2, 4, 8
• Segment size 128K, 256K, 512K, 1MB
• Stripe size 8KB, 64KB, 1MB, 2MB
• Total file size 16GB

13
Results
14
Conclusion and Improvements

• Library interfaces perform as well as the best
  benchmark
• Necessary to extend current implementation to
  further study the effect of stripe size and size
  of a data chunk being written
• Parallel I/O for shared memory platforms an
  interesting field of research

15
Thank You

• Questions ?