Presentation and Analysis of Grid Application Performance Data

1
Presentation and Analysis of Grid Application
Performance Data
Using GRM, Mercury and R-GMAfor monitoring grid
applications

• Norbert Podhorszki and Peter Kacsuk
• Laboratory of Parallel and Distributed Systems
• MTA SZTAKI
• Hungary

2
Grid Application Monitoring

• Our target of research is
• Performance monitoring (not job status!) of
• Parallel applications (MPI, PVM, P-GRADE)
• Running on one grid resource (cluster,
  supercomputer)
• Large volume of trace information
• On-line transfer of trace to the user

3
Grid Application Monitoring

• What do we need for this type of application
  monitoring?
• Instrumentation
• Trace collector
• Mechanism to transfer trace data from the
  application processes to the monitoring tool /
  the user
• Information system for finding the application
  and the monitor components

4
GRM and PROVE
Our starting point
5
GRM
Local Host
Main MonitorMM
PROVE
Site 2
Site 1
Host 1
Host 2
Host 1
Local MonitorLM
Local MonitorLM
Local MonitorLM
Application Process
Application Process
Appl. Process
Appl. Process
6
Problems with GRM

• Firewalls block communication between Local and
  Main Monitors
• Need for some proxy mechanism (site-wide service
  for monitoring)
• Launch of Local monitors
• We do not know where the application starts
• We are not able to launch LM there as well
• Used forking out LM from application process

7
Mercury monitor

• Developed within the GridLab project
• Modular framework (wrt. sensor, metrics,
  communication protocol)
• Supports both resource status and application
  monitoring
• Efficient implementation to be fast

8
Mercury monitor
9
Mercury monitor

• What is missing for application monitoring?
• Instrumentation trace collector
• Independent Mercury installations on different
  grid resources (services) ? we need to find them
  ? information system is needed

10
R-GMA

• R-GMA is the Relational Monitoring and
  Information System within the EU-DataGrid project
• Gives the illusion of a virtual relational
  database of the information available about grid
  services, resources and jobs
• SQL is the information publishing and query
  language

11
R-GMA
12
R-GMA
13
R-GMA
14
R-GMA

• What is missing for application monitoring?
• Instrumentation trace collector
• Java servlet technology, transfer of data in XML,
  parsing of SQL strings all the time ? not
  efficient for transfer of large amount of data ?
  mechanism for transfer is needed

15
Grid Application Monitoring

• GRM
• Instrumentation library and trace collector
• Mercury monitor
• Transfer trace data from the application
  processes to GRM
• R-GMA
• Information system for finding the application
  and the monitor components

16
GRM Mercury R-GMA
17
Mercury monitor

• Here we use it to deliver trace data from the
  application to the user efficiently.
• Uses TCP Socket communication
• Data in XDR-like format and could be optimised
  for TCP transmission
• Two sw. hops between application and GRM local
  and main monitors
• One hw. hop host of main monitor

18
R-GMA

• To find the application by the user/GRM
• Where is it running? -gt machineX.siteY
• What is its global job id? -gt GID
• To find the monitor to be connected
• What is the address of Mercury monitor running at
  siteY?
• To find the application by the monitor
• What processes (PIDs) belong to application GID?
• In EDG, global job ID is not a problem, as it is
  known both by the application processes and the
  user.

19
GRM Mercury R-GMA
20
R-GMA

• Combining the two tasks into one for efficiency
• Publication of application information and
• Publication of URL of Mercury monitor in
• one relational table (jobID, MercuryURL)
• An Archiver is used in R-GMA to store the
  advertisements

21
GRM

• Job wrapper script
• Publish process info into R-GMA
• Global job ID
• address of corresponding Mercury monitor
• Instrumentation library
• Connect and publish trace to Mercury monitor
• GRM
• Look for application asking R-GMA
• Connect to Mercury monitor and query for trace

22
Procedure of monitoring

• 1. Instrument the application with GRM
  instrumentation calls
• by hand currently, for P-GRADE applications
  automatically
• 2. Submit your job into the grid
• resource broker gives the jobID
• 3. Wrapper script somewhere publishes job info in
  R-GMA
• 4. R-GMA archives the information
• 5. Start GRM giving jobID as parameter
• 6. GRM asks R-GMA about application jobID
• 7. R-GMA responds with answer (jobID, msURL)
• 8. GRM subscribes at Mercury using the msURL for
  application jobID
• 9. You are happy

23
Status (GRMMercury)
24
Success of Mercury

• It is used in GridLab project
• It will be used in the Hungarian ClusterGrid
  project
• 99 clusters, 2000 machines
• It becomes part of DataGrid release 2
• for application monitoring

25
Status (GRMMercuryR-GMA)

• Works fine in our institute
• GRM Mercury was the difficult point (large
  volume of data)
• R-GMA is easy to use as an information system
• Currently under installation on EDG testbed
• difficulty configuration of Mercury on different
  grid resources automatically

26
Future work

• Intrusion replace socket communication between
  LM and application process for shared-memory
  buffering (as in original GRM)
• Scalability data reduction at LMs and MM.

27
Conclusion

• GRM, connected to Mercury and R-GMA, can be used
  to collect trace about parallel applications
  running in the grid.
• Good example (the first) for the integration of
  different results in different grid projects

28

• Thank you