WP3%204/9/2002

1
WP34/9/2002
Steve Fisher / RAL lts.m.fisher_at_rl.ac.ukgt
2
Summary

• Quality
• Current status 1.2
• R-GMA in release 2.0
• Recent Requirements
• Work this week
• Future work

3
Quality

• Use XP (not the Windows one)
• So code does not belong to the individual but we
  are all responsible for the code
• Ego-less programming
• Some pair-programming
• 100 Unit tests for R-GMA
• Insist that all unit tests are run before work is
  checked in
• Unit tests are being written as problems appear
• Follow approved coding style (most of the time)
• Log4j is used uniformly throughout the code
• Will be active in QAG

4
Status in 1.2

• MDS seems to be working with the hierarchy
  but
• It relies on some patches
• MDS 2.2 is claimed to fix many bugs
• We will take a look at MDS 2.2 and see if it can
  indeed be introduced easily
• Support of 1.2
• All categories of bugs go to me to dispatch to
  the most appropriate person or persons

5
Release 1.2
RB
6
Release 2.0
Multi-valued attributes make it not totally
trivial
We will also leave the GRISs for the RB
7
GOUT
Archiver
Consumer (CE)
Consumer (SE)
Consumer
LDAP
DataBaseProducer
RDBMS
Consumer (..)
Clean up
8
R-GMA in release 2.0

• Mediator able to merge information from several
  producers
• GRM/PROVE for parallel applications
• Authentication (from WP2)
• Ad-hoc authorisation sufficient for LB to ensure
  that users can only find out about their own jobs
• Snapshot producer
• Canonical producer
• Java, C, C, Python and Perl APIs
• Pulse
• GUI to browse R-GMA data

Dates will be defined when we have a release
schedule to follow
9
R-GMA in release 2.0

• GLUE (affects GIN and GOUT)
• Plan to allow users to publish their own tables
• We may not offer this immediately because we
  won't have sufficient security in place.
• We will in any case provide a general table
• UserTable
• Identification
• String
• Real
• Integer
• TimeStamp
• Publish with WHERE Identification xxx
• Easy installation and configuration
• For developers
• Installers
• Users

10
Release 2.0 Internals

• GRRP-like soft state registration
• Uniform exception handling
• To ensure that useful messages and stack traces
  are preserved.
• This includes communication between Servlet and
  API
• Replication
• Distribute the schema and the registry
• For performance
• For reliability

11
Recent Requirements

• Common request for not losing any information
• Lossless communication
• Demands for resilience
• Keep on producing even when unable to contact
  the registry
• Only want latest information (by primary key)
• Will provide a snapshot facility
• Canonical producer

12
Work this week

• Quality
• R-GMA at SZTAKI
• GRM / PROVE
• Pulse
• Security
• New directions for R-GMA
• Fit in with OGSA
• Registry and Schema
• Replication for performance and reliability
• Time stamps
• Performance
• Theory
• Of R-GMA
• Plans

13
New directions for R-GMA

• We have produced quite a lot of code
• Now need to do some refactoring
• Will include redesign of APIs
• Distinguish more clearly between using a consumer
  via the mediator and making a direct connection
  to one or more producers
• Clarify the capabilities of different kinds of
  producer (including the Canonical Producer)

14
Primary Producers

• Database producer
• supports queries over fixed set of tuples
• static / history queries
• Stream producer
• supports queries over changing set of tuples
• continuous queries
• Snapshot producer
• supports snapshot queries
• offers up-to-date values for each primary key
• Canonical producer
• being developed by CrossGrid

15
Communication Modes

• Stream Producers offer two communication modes
  for continuous queries
• lossless (infinite buffer size)
• lossy

16
Registry and Schema

• We plan deployment for release 2.0 with 3
  registries and 3 schemas (per VO?)
• The schemas will be exact copies of each other
• Need to ensure that there is exactly one
  definition of a table
• The registries will not always be identical but
  will catch up
• Can afford to have a small delay in making
  producers and consumers known. No problem with
  producers and consumers which are still in the
  registry but dont exist.

17
Future Work

• OGSIfication
• We need to do this very soon in order to make
  sure that we dont fall behind
• Consider how to handle time better in queries
• Proper authorisation scheme
• More mediator functionality
• Joins
• Can do it now by direct connection to a
  DataBaseProducer no use of mediator
• Might make it into release 2.0