jGMA: A Reference Implementation of the Grid Monitoring Architecture

1
jGMA A Reference Implementation of the Grid
Monitoring Architecture

• Mat Grove
• Distributed Systems Group
• University of Portsmouth
• http//dsg.port.ac.uk/mjeg/

2
Outline

• Introduction and Motivation.
• The Grid Monitoring Architecture (GMA).
• The jGMA Implementation.
• Summary and future work.
• jGMA Demo.

3
Introduction

• GridRM is
• A generic resource monitoring system basically
  a framework for gathering data from heterogeneous
  data sources,
• A scalable means of monitoring resources
  distributed over the Grid,
• Written in Java, using commons standards
• Such as SQL, JDBC, Java Servlets, GLUE.
• GridRM is made up of local and global layers the
  latter basically is the means of transporting
  GridRM data between grid sites.
• GridRM needed a standards-based transport layer
  to bind together these its sites at the global
  layer.
• A GMA-based transport appeared to be the best
  candidate.

And and the the a
4
GridRM (gridrm.org)
5
The Grid Monitoring Architecture

• The GGFs GMA document is their suggested
  informational means for monitoring within the
  Grid
• Note Informational documents are used to inform
  the community about a useful idea or set of
  ideas.
• The GMA document (2001) recommends
• A publish, subscribe and bind architecture,
• An implementation that is
• Secure and scalable,
• Has a low-latency and capable of high data rates,
• Imposes minimal impact on the infrastructure.
• GMA compliancy means implementing approximately
  20 features.

And and the the a
6
GMA Components

• Registry

• Publish/Search

• Publish/Search

• Bind

• Producer

• Consumer

• Events

7
Some GMA Implementations

• In the first instance we wanted to take up an
  off-the-shelf GMA package, several out there.
• Embedded (part of a larger s/w package)
• e.g. AutoPilot and Globus MDS
• MDS can be extracted into a standalone
  infrastructure with a little effort.
• Standalone
• pyGMA (Berkeley National Lab), and R-GMA
  (DataGrid/LCG).
• None of these implementations met GridRMs
  requirements
• Easy to integrate within GridRM (Java-based),
• Simple installation with minimal dependencies,
• No need to fork parent software tree,
• GMA compliance.

And and the the a
8
jGMA Components

• jGMA has the following components
• A registry,
• A producer and consumer (shared code-base),
• A PC Servlet (provides WAN access between
  producers and consumers).
• jGMA features
• Written in Java, so takes advantage of usual
  features and libraries,
• Small API (lt20 calls),
• Released as single jar of currently 28Kbytes.
• Limited dependencies
• Tomcat and Java.

9
jGMA Implementation
10
jGMA Optimisations

• The jGMA Implementation
• Non-blocking communications
• Local communications use Java sockets
• RMI,
• Serialising objects,
• Packed binary arrays,
• WAN communication use HTTP
• HTTP POST,
• Encapsulating binary data.
• Performance Tests
• Latency (Over Ethernet 1.5ms, Localhost 0.8ms),
• Throughput (1200 messages per second),
• Scalability (tested stability under heavy loads).

11
Consumer pseudo code.
Consumer() Connectornew JgmaConnector(holly.d
sg.port.ac.uk) Connector.register(pinger)
Registered() Connector.registryQuery(select
from producers) RegistryEvent(result)
for each result as producer
Connector.sendMessage(producer, ping)
12
Producer pseudo code.
Producer() Connectornew JgmaCOnnector(holly.d
sg.port.ac.uk) Connector.register(ponger)
MessageEvent(message) Connector.replyMessage(me
ssage, pong)
13
Example jGMA Addressing.

• LAN address
• WAN address

• socket//148.197.150.356099/producer/cam

• http//dsg.port.ac.uk8080/jGMA/PC?8723623

14
Summary

• Summary
• jGMA is a reference implementation of GMA written
  in Java,
• jGMA is a easy to use, simple, scalable and a
  robust system,
• Our early tests show that jGMA has good
  performance and scalability.
• Some issues need dealing with
• Throttling.

15
Future Work

• Future developments
• Implement a persistent registry
• File (ASCII text),
• R-DBMS (i.e. MySQL),
• XML (i.e. Xindice).
• Completing the leasing infrastructure,
• Virtual Registry,
• Monitoring via PC Servlet,
• Hooks for debugging,
• Being deployed in GridRM,
• Look at working with other GMA developers to get
  an agreement on a standardised wire API.

16
Demo
17
Download

• Available jGMA-0.3.7-beta
• New version for October
• http//dsg.port.ac.uk/projects/jGMA

18
Links

• Project Web page
• http//dsg.port.ac.uk/projects/jGMA/
• The DSG Web page
• http//dsg.port.ac.uk/
• GridRM
• http//gridrm.org/
• Related links
• GGF http//www.gridforum.org/
• R-GMA http//www.r-gma.org/

19

• Xcx
• vxv
• Xcv
• xcv
• Sdf
• Sd
• F
• S
• Dfs
• Df

• Xcx
• vxv
• Xcv
• xcv

20

• Xcx
• vxv
• Xcv
• xcv
• Sdf
• Sd
• F
• S
• Dfs
• Df

• Xcx
• vxv
• Xcv
• xcv

21

• Xcx
• vxv
• Xcv
• xcv
• Sdf
• Sd
• F
• S
• Dfs
• Df

• Xcx
• vxv
• Xcv
• xcv

22

• Xcx
• vxv
• Xcv
• xcv
• Sdf
• Sd
• F
• S
• Dfs
• Df

• Xcx
• vxv
• Xcv
• xcv

23

• Xcx
• vxv
• Xcv
• xcv
• Sdf
• Sd
• F
• S
• Dfs
• Df

• Xcx
• vxv
• Xcv
• xcv