OptiGrid Tests

1

• OptiGrid Tests
• Venkatram Vishwanath, Tom Defanti, Alan Verlo,
  Jason Leigh, Linda Winkler, Praveen KJ, Thomas
  Hutton, John Ratcliffe, Lance Long, Greg Hidley,
  Pat Hallihan, Qian Liu

2
TeraGrid and CAVEWave

• Teragrid
• RTT 56.3 ms
• IP -gt Ethernet -gt SONET
• OC-192 LINE RATE 9.95G
• OC-192 DATA RATE 9.58G
• Packet Over SONET Efficiency 0.986 (1500byte)
• MPLS Efficiency 0.997 (1500byte)
• CAVEWave
• RTT 78.2 ms
• IP-gt Ethernet -gt DWDM

3
(No Transcript)
4
Maximum Throughput using IPERF - UDP
CAVEWave 10 UDP streams at 975Mbps 0 Packet
Loss Effective Throughput 9.75Gbps(San Diego
to Chicago) Effective Throughput
9.73Gbps(Chicago to San Diego)
TeraWave 10 UDP streams at 960Mbps 1 Packet
Loss Effective Throughput 9.45Gbps(San Diego
to Chicago) Effective Throughput
9.43Gbps(Chicago to San Diego)
5
Unidirectional Reliable Memory to Memory
Throughput (LambdaStream - LS)
CAVEWave 10 LambdaStream Streams Effective
Throughput 9.23Gbps(San Diego to
Chicago) Effective Throughput 9.19Gbps(Chicago
to San Diego)
TeraWave 10 LambdaStream Streams Effective
Throughput 9.16Gbps(San Diego to
Chicago) Effective Throughput 9.14Gbps(Chicago
to San Diego)
6
Bi-directional Reliable Mem to Mem Throughput
(LambdaStream - LS)
CAVEWave 10 LambdaStream Streams from Chicago to
San Diego 10 LambdaStream Streams from San
Diego to Chicago Effective Throughput 18.19
Gbps
TeraWave 10 LambdaStream Streams from Chicago to
San Diego 10 LambdaStream Streams from San
Diego to Chicago Effective Throughput 18.06
Gbps
7
Disk Configuration and speeds

• Yorda / Nico (Chicago)
• Read 620 Mbps
• Write 480 Mbps
• Scylla ( Chicago)
• Read 500 Mbps
• Write 350 Mbps
• Vellum (San Diego)
• Read 650 Mbps
• Write 530 Mbps

8
Disk to Memory tests with LambdaRAM/ LambdaStream
CAVEWave16 senders to 16 receivers (point to
point ) Effective Throughput 9.21Gbps(San
Diego to Chicago) Effective Throughput
9.08Gbps(Chicago to San Diego)
TeraWave 16 senders to 16 receivers (point to
point ) Effective Throughput 9.15Gbps(San
Diego to Chicago) Effective Throughput
9.03Gbps(Chicago to San Diego)
Dataset 220GB Satellite Imagery of Chicago
courtesy USGS. Each file is 5000 x 5000 RGB image
with a size of 75MB i.e 3000 files
9
Disk to Disk tests with LambdaRAM/ LambdaStream
TeraWave 20 senders to 20 receivers (point to
point ) Effective Throughput 9.02 Gbps(San
Diego to Chicago) 451 Mbps disk to disk transfer
per stream Effective Throughput 9.22
Gbps(Chicago to San Diego) 461 Mbps disk to
disk transfer per stream
CAVEWave20 senders to 20 receivers (point to
point ) Effective Throughput 9.01 Gbps(San
Diego to Chicago) 450.5 Mbps disk to disk
transfer per stream Effective Throughput 9.30
Gbps(Chicago to San Diego) 465 Mbps disk to
disk transfer per stream
Dataset 220GB Satellite Imagery of Chicago
courtesy USGS. Each file is 5000 x 5000 RGB image
with a size of 75MB i.e 3000 files
10
TCP vs LambdaStream (LS) Performance

• Single Stream TCP
• Chicago to San Diego
• 20 Mbps with no tuning
• 300 Mbps with 5MB
• 940 Mbps with 15MB
• San Diego to Chicago
• 940 Mbps
• Does not work for Multiple Streams.

• Single Stream LS
• Chicago to San Diego
• 928 Mbps
• (Untuned performance)
• San Diego to Chicago
• 928 Mbps
• (Untuned performance)

11
TeraGrid Utilization during tests
Weekly' Graph (30 Minute Average)
Max  In 9129.3 Mb/s (91.3) Average  In 674.8
Mb/s (6.7) Max  Out 9275.7.8 Mb/s (92.7)
Average  Out 583.9 Mb/s (5.83) (Tests coincided
with NTTs tests(8-10 hours/day) which did not
use much BW. Max BW used by NTT 1Gbps.
12
CAVEWave Utilization during tests
Weekly' Graph (30 Minute Average)
Max  In 9295.0 Mb/s (92.9) Average  In
1078.8 Mb/s (10.8) Max  Out 9124.8 Mb/s (91.2)
Average  Out 1615.9 Mb/s (16.2) (Tests
coincided with NTTs tests(8-10 hours/day) which
did not use much BW. Max BW used by NTT 1Gbps)