ApGrid Demonstration

1
ApGrid Demonstration

• Putchong Uthayopas, Sugree Phatanapherom,
• Parallel Research Group, Department of Computer
  Engineering
• Faculty of Engineering, Kasetsart University
• Bangkok, Thailand.

2
Introduction

• Motivation
• Explore technical problem in the deployment of
  grid system in Asia pacific level using resources
  provided by ApGrid test-bed
• Develop initial ad hoc infrastructure that allows
  researcher to start using ApGrid test bed
• Explore some innovative ideas and applications of
  the international grid
• Stimulate the collaboration among Asia Pacific
  Grid researchers
• Participants
• HKU (China)
• AIST (Japan)
• KU, KMITNB, NECTEC (Thailand)

3
About the ApGrid Demo

• Hardware
• 14 nodes 42 processors in 3 countries and 5
  organizations are used.
• SUN, Alpha, x86 (Intel and AMD)
• Software
• Globus Toolkit 2.0
• Job Manager
• Condor, PBS, and SQMS
• Signed Certificate by AIST are used
• /CJP/OAIST GTRC/CNltfull namegt/Emaillte-mailgt
• /CJP/OAIST GTRC/CNhost/ltfqdngt
• /CJP/OAIST GTRC/CNldap/ltfqdngt

4
Software Infrastructure
Applications (CFD, Rendering, Monitoring)
Globus 2
AIST
HKU
ThaiGrid
KU
KMITNB
NECTEC
5
MDS
mds.apgrid.org
ApGrid
Thai
AIST
HKU
NECTEC
KOUME (4/4)
Grid (4/4)
RDC4 (1/1)
PALM (1/1)
Alpha (1/2)
AMATA (1/28)
KMITNB
(Node/CPU)
PRG (2/2)
KU
6
Demonstration Applications

• Grid Movie Rendering
• Computation
• Matrix Multiplication
• Equation solver using Gauss Elimination
• Computational Fluid Dynamic (Heat Transfer)

7
Grid Movie Rendering
stage
exe
/
io
dispatch
stage output
stage inputs
povray
animation
povray
source
output frames
output frames
8
Computation Applications
9
DEMO !
10
Bandwidth Measurement

• Using Globus to help running bandwidth
  measurement
• Algorithm
• Start Iperf server on one Grid Point
• Start Iperf client on all grid point and measure
  the bandwidth to iper server
• Repeat the measurement for all point

11
Bandwidth
ALPHA AMATA HKU KOUME PALM PRG RDC4
ALPHA 876.9 80.1 0.021 1.680 1.927 74.0 8.911
AMATA 65.6 1782.3 0.014 1.595 1.925 89.0 8.926
HKU 0.132 0.023 515.2 0.679 0.017 0.039 0.034
KOUME 1.153 1.375 1.114 2271.9 0.003 1.339 1.324
PALM 1.967 1.966 0 0.356 858.0 1.962 0
PRG 69.9 88.4 0 1.468 1.926 329.2 8.875
RDC4 0 0 0 0 0 0 2036.9
12
Proposed Distributed Bandwidth Measurement
Infrastructure
13
Task Execution Model and Parameters

• Tqg - Grid queuing latency time
• Depend on Policy, Number of Grid level jobs
• Tsl Launcher staging time
• Tql - Local queuing latency time
• Tsexec Executable staging time
• Tsinput Input staging time
• Texec - Execution time
• Tsoutput Output staging time

14
Application Characteristics
Grid Movies Rendering
Matrix Multiplication, Gauss Elimination
CFD (Heat Conduction)
15
Comparison of Staging Time and Overall Time
16
(No Transcript)
17
Results

• MRB (Minimum required Bandwidthe)
• Minimum Bandwidth that the communication overhead
  time is less than execution time of a problem
• Depend on Bandwidth and remote computing power
• Can be used to decide which application is
  practical to run on the grid
• Example
• MRB between KU and HKU for each test
  application
• Povray - 128 Kbps
• MMul - 8 Mbps
• Gauss - 4 Mbps
• Heat - 16 Mbps

18
Recommendation 1

• Minimize traffic
• Scheduler
• Dispatch grid jobs at low-traffic time to get the
  maximum bandwidth
• Traffic aware scheduler
• Continuous traffic monitoring is important
• Launcher
• Stage output back at low-traffic time
• Co-scheduling the input/executable/output staging
  task among multiple jobs to fully utilized
  network bandwidth

19
Recommendation 2

• Minimize Execution Staging Latency
• Schedule the pre-staging of executable
• Reuse the executable code
• Break application into dynamics link module and
  pre-stage DLL module

20
Problems

• MDS
• Unstable (2.1)
• Firewall
• gatekeeper 2119
• giis/gris 2135
• gsiftp 2811
• other tcp range

gass server
gass client
establish
send request
disconnect
listenforreply
establish
reply
disconnect
HKU
40000-40050
KU
2001-10000
NECTEC
????
21
Problems

• Bandwidth Variation
• KU-HKU
• Maximum 1 Mbps
• Minimum 1 Kbps or less
• AIST-KMITNB
• AIST-gtKMITNB 2.5 Kbps
• KMITNB-gtAIST 365 Kbps

22
Summary

• This demo is a crucial step in
• Establishing a workable infrastructure that
  utilize ApGrid Resources
• Explore the challenging issues on the real system
• Grid is working but appropriate for application
  with high computation/communication ratio

23
Next Step

• Establish the Monitoring Infrastructure and a
  monitoring portal that allows human and software
  entities on Apgrid to access the traffic
  information.
• ApGrid FAQ
• Testing more applications on more site
• Drug Design, Chemical Reaction ( Gamess)
• Find a way to increase bandwidth between
  participating test site

24
Acknowledgement

• Yoshio Tanaka, AIST
• Choli Wang,Roy C. Ho, HKU
• Sissades Tongsima,Kittinarong Laongvaree, NECTEC
• Vara Varavidthaya, KMITNB

25
The End