Active Buffering

1
Active Buffering

• Reduce the visible cost of parallel I/O by
  increasing parallelism
• by using idle resources to spread the I/O costs
  across the entire computation
• in a
• platform-independent,
• self-tuning (buffer space, request pattern),
• user-transparent manner

Computation
I/O
Computation
I/O
Computation
I/O
Computation

2

Active Buffering
local memory access
inter-processor communication
disk I/O
wide-area transfer

• Problem diminishing throughput, parallelism, and
  scalability along path
• Solution Buffer as much as possible as high as
  possible, resume computation
• Status In production use in CSARs Rocpanda
  tech transfer underway from PANDA to ROMIO
  ICS02,IPDPS02,IPDPS03,PDSECA03

3
Aggregate GEN2 Write Thruput

• LLNL IBM SP (ASCI Frost)
• 1 I/O server per 16-way SMP node
• Write in HDF

4
The rest of these slides are only backup for DK

5
Active Buffering vs. Async. I/O
6
AB Server/Thread State Machine
data to receive
enough buffer space
prepare
receive a block
recv.
write
out of buffer space
request
write a block
fetch
idle- listen
init.
recv.
recv.
alloc. buffers
got write
idle, no
exit
req.
all data
message
data to
received
fetch
no
fetch a block
data to
write done
data
no request
write
idle
fetch write all
busy- listen
to fetch
write done
exit
msg.
7
ABT and ROMIO

• Unchanged user interfaces
• High-level and file-system independent
• Serverless IPDPS 03
• Ongoing transfer to ROMIO
• Reads via buffer flushes or direct from buffers

ADIO
NFS
HFS
NTFS
PFS
PVFS
XFS
UFS
MIG.