Presentation and Analysis of Grid Performance Data - PowerPoint PPT Presentation

About This Presentation
Title:

Presentation and Analysis of Grid Performance Data

Description:

Hungarian Academy of Sciences. Pulse. On-line analysis and visualisation tool for ... GRM is a tool for performance monitoring of parallel applications, connected to ... – PowerPoint PPT presentation

Number of Views:19
Avg rating:3.0/5.0
Slides: 33
Provided by: szt4
Category:

less

Transcript and Presenter's Notes

Title: Presentation and Analysis of Grid Performance Data


1
Presentation and Analysis of Grid Performance
Data
  • Norbert Podhorszki and Peter Kacsuk
  • Laboratory of Parallel and Distributed Systems
  • MTA SZTAKI
  • Hungary

2
Contents
  • Relational Grid Information and Monitoring
    System(R-GMA)
  • R-GMA Browser
  • Browsing status information
  • Pulse
  • Analysis and presentation of performance
    information of grid services and components
  • GRM/PROVE
  • Performance monitoring of applications

3
R-GMA
  • R-GMA is the Relational Monitoring and
    Information System within the EU-DataGrid project
  • Gives the illusion of a virtual relational
    database of the information available about grid
    services, resources and jobs
  • SQL is the information publishing and query
    language

4
R-GMA
5
R-GMA
6
R-GMA
7
Pulse
  • On-line analysis and visualisation tool for
  • monitoring data of resources and services and
    applications.

8
Pulse
  • Data source components ? preprocessing
    components ? view components their
    connections are defined in XML
  • Separates data model from visualisation modules
  • based on Model-View-Controller paradigm

9
Pulse
  • Data model meta-data (name, type, unit, range)
    and interface to the data source.
  • Controller preprocessing, analysis components
  • View displaying components

10
Example CPU Load
11
Example CPU Load
12
Example CPU Load
13
Data model
  • Data channel
  • Name
  • Data type
  • Unit
  • Range of valid data
  • LoadChannelMetaData(
  • nameload1,
  • typefloat,
  • unit,
  • range0.0-100.00.01 )

14
Input components
  • Input component (sensor) an interface to some
    kind of external data source (file, service, an
    executable, etc.)
  • A sensor
  • Reads data from the external source
  • Translates data into a channel based
    representation
  • Publishes complex information in several channels
  • Provides one piece of information at a time
  • Buffered sensor
  • Buffering capability to a sensor

15
Presentation components
  • To show the results to the user
  • Tabular view, Histogram plotter
  • Presentation components subscribe to the sensor's
    notification service -gtthey are notified about
    the status changes in the sensor -gtdisplay
    updates are controlled by the sensors (incoming
    data)

16
Analysis components
  • Essentially
  • Take data from other components (sensors)
  • Do something with the data
  • Forward it to other components (plotter)
  • E.g.
  • Channel selector (for histogram plotter)
  • Merge data from several sensors
  • Compute statistical values on data

17
Configuration of Pulse
  • XML configuration to define
  • Components
  • Connections between the components

18
Pulse conclusion
  • Pulse can be used to build an analysis chain to
    process dynamic information and present results
    in a graphical form
  • Flexible data representation
  • Already it can be used as a tool for the R-GMA
    grid monitoring and information system

19
Application monitoring
  • Performance monitoring (not job status!) of
  • Parallel applications (MPI, PVM, P-GRADE)
  • Large volume of trace information
  • On-line transfer of trace to the user

20
GRM at EuroPar2001
Local Host
Main MonitorMM
Site 1
Site MonitorSM
Site MonitorSM
Site 2
Host 1
Host 2
Host 1
Local MonitorLM
Local MonitorLM
Local MonitorLM
Application Process
Application Process
Appl. Process
Appl. Process
21
Tools
  • GRM
  • Instrumentation library and trace collector
  • Mercury monitor
  • Transfer trace data from the application
    processes to GRM
  • R-GMA
  • Information system for finding the application
    and the monitor components

22
Mercury monitor in the GridLab project
23
GRMMercuryR-GMA
24
Mercury monitor
  • To deliver trace data from the application to the
    user efficiently.
  • Uses TCP Socket communication
  • Data in XDR format and could be optimised for TCP
    transmission
  • Two sw. hops between application and GRM local
    and main monitors
  • One hw. hop host of main monitor

25
R-GMA
  • To find the application by the user/GRM
  • Where is it running? -gt machineX.siteY
  • What is its global job id? -gt GID
  • To find the monitor to be connected
  • What is the address of Mercury monitor running at
    siteY?
  • To find the application by the monitor
  • What processes (PIDs) belong to application GID?

26
GRM
  • Instrumentation library
  • Connect and publish trace to Mercury monitor
  • GRM and PROVE
  • Look for application asking R-GMA
  • Connect to Mercury monitor and query for trace
  • Visualise trace information with PROVE

27
Current status
28
Future work
  • Intrusion replace socket communication between
    LM and application process for shared-memory
    buffering (as in original GRM)
  • Scalability data reduction at LMs and MM.

29
Conclusion
  • Different monitoring information can be presented
    with different techniques
  • Web browser is used to search among available
    information and static (or slowly changing)
    behaviour of the grid
  • Pulse can be used to build specific analysis
    chains to process dynamic information about the
    services and resources and to create graphical
    plots
  • GRM is a tool for performance monitoring of
    parallel applications, connected to Mercury and
    R-GMA to work for grid.

30
  • Thank you

31
R-GMA Schema Browser
32
R-GMA Schema Browser
Write a Comment
User Comments (0)
About PowerShow.com