A Scalable Virtual Registry Service for jGMA

1
A Scalable Virtual Registry Service for jGMA
Matthew Grove CCGRID 2005 - WIPMay 2005
2
Outline

• Introduction and motivation,
• The jGMA architecture,
• The Virtual Registry (VR),
• An implementation of the VR using IRC,
• Future work,
• Summary.

3
Introduction

• jGMA is a pure Java reference implementation of
  the GGF's Grid Monitoring Architecture, initially
  designed to be a standards-based messaging
  layer to bind together the global layer of a
  project called GridRM,
• jGMA requirements
• Compliant to the GMA specification,
• Supports both non-blocking and blocking events,
• Work locally over a LAN or a wide-area, such as
  the Internet,
• Fast, and have a minimal impact on its hosts,
• Scalable,
• An integrated registry service,
• Able to work through firewalls.

4
Design Philosophy

• Reuse existing software components, if possible,
  rather than reinvent functionality,
• Small and well defined API,
• Minimal number of other installation
  dependencies,
• Simple to install, configure and use,
• Provide a pluggable infrastructure with a choice
  to extend core functionality with a further
  more-sophisticated component.

5
jGMA Implementation

• jGMA has the following components
• A producer and consumer (shared code-base),
• A Mediator which provides WAN access to producers
  and consumers and hooks for a distributed Virtual
  Registry, and debugging/monitoring.
• jGMA features
• Written in Java, so takes advantage of usual
  features and libraries,
• Small API (lt15 calls),
• Released as single jar of currently 28Kbytes,
• Limited dependencies
• Java.

6
jGMA Architecture
7
The Mediator

• The Mediator replaces the PC and Registry
  servlets.

8
The jGMA Library and API

• A choice local registries, two have been
  implemented Java data structure and MySQL.

9
Virtual Registry Requirements

• To complete the implementation jGMA requires a
  registry component which must
• Be scalable,
• Store sufficient information to be GMA compliant,
• Be secure, and prevent unauthorised access to the
  data,
• Need minimal configuration,
• Ideally have no single point of failure,
• Be robust and tolerant to failures,
• Efficient query routing between VRs.

10
Virtual Registry Services

• Core Services
• Boot-strapping Joining the jGMA network with
  minimal hardwiring,
• Communications Efficient routing of queries
  between VR peers,
• Caching Keeping a temporary local copy of some
  information to reduce the amount of
  communications between peers.
• The API is designed to allow different
  implementations of these core services to be
  mixed and matched.

11
IRC VR Service

• The IRC VR service is an attempt at meeting the
  criteria of the jGMA VR by building a service
  which uses the mature Internet Relay Chat
  protocol,
• IRC networks have servers connected via a graph
  topology (attempting to efficiently route
  messages and provide fault tolerance), some
  networks manage thousands of users.
• Fault tolerant boot-strapping via DNS,
• Various degrees of security can be provided by
  combinations of
• Leveraging functionality built into the IRC
  protocol (locking the channel, private messages),
• Using features of the IRC daemons such as
  encrypted routing for IRC messages,
• Running your own IRC daemons rather than using a
  public network.

12
IRC VR Service Continued

• Each Mediator has an IRC Bot which connects to
  a chosen IRC network
• The Bot joins a channel and announces itself to
  other Bots,
• The Bot publishes search queries to the IRC
  channel,
• Other Bots receive the queries and pass them onto
  the Registry backend,
• The Bots reply to the queries via a private
  message over IRC.
• All messages are routed by the IRC network.

13
IRC VR Schematic
14
IRC VR Issues / Questions

• Currently IRC channels are used to partition the
  VR by project,
• There needs to be other ways to reduce the impact
  of a flood query on the network
• What are the limits on individual channels, is
  there a need to have multiple channels and route
  messages between them?
• Using IRC requires an extra port be accessible to
  the mediator (out going port 6667).

15
Current Status

• LAN (socket) and WAN (HTTP) layers complete,
• The initial implementation of the IRC VR is
  complete and functional,
• Demo applications and benchmarking has thoroughly
  tested the jGMA software,
• Permanent on-line demo
• http//dsg.port.ac.uk/projects/jGMA/demos/web/

16
Webcam Browser Demo

• http//dsg.port.ac.uk/projects/jGMA/demos/web/

17
Future Work

• Short term
• Implement a generic caching service for the VR,
• Write a simple P2P VR, which can be used in
  place of the IRC VR,
• Add hooks for debugging,
• Benchmark jGMA against Naradabroker.
• Longer term
• Provide a updated binary release of jGMA,
• Develop an application or library that uses
  jGMA.

18
Summary

• jGMA is functional despite a few missing
  features,
• Current work is focusing on the implementation of
  the VR,
• The design of the jGMA architecture is complete.

19
Acknowledgements

• University of Portsmouth,
• Members of the DSG
• Mark Baker,
• Hong Ong,
• Garry Smith,
• Mr Boe-akis,
• Aamir Shafi,
• Helen Xaing.

20
Links

• Project Web page
• http//dsg.port.ac.uk/projects/jGMA/
• The DSG Web page
• http//dsg.port.ac.uk/