Information and Monitoring

1
Information and Monitoring

• Rob Byrom / WP3
• r.byrom_at_rl.ac.uk

2
Plan of Talk

• Final EDG Release
• Whos using R-GMA?
• Migrating to EGEE
• Working within ARDA
• R-GMA Future Work

3
Final EDG Release

• Robust
• Significant improvements to robustness
  mid-December
• Information Catalogue ran continuously for 1
  month without restart
• Performance
• Significant improvements to performance
• Carried out extensive testing using CMS
  simulation test framework
• Performance figures published daily available
  via our web site
• http//hepunx.rl.ac.uk/edg/wp3/performance/index.h
  tml
• Authentication
• Implemented but currently disabled on app testbed
• Nagios
• Use alert system to identify problems on the
  various testbeds
• https//heplnx30.pp.rl.ac.uk/nagios/
• Ganglia Integration
• Ganglia data published via R-GMA (called Ranglia)

4
Whos using R-GMA?

• BaBar/Southern Tier 2
• Ranglia
• CMS
• Boss
• David Colling, Peter Hobson et al
• D0
• Data challenge
• Jeff Templon - Nikhef
• LCG
• Publishing PBS accounting data for LCG
• Network Monitoring
• WP7

5
Migrating to EGEE

• WP3 to form the Information Collection Cluster
• JRA1 - Objectives
• Provide robust, supportable middleware components
• Select, re-engineer and integrate identified
  components
• Evolve towards Services Oriented Architecture
• Adopt emerging WS-RF standards
• Support multiple platforms
• Define defect handling processes and
  responsibilities
• Selection of Middleware based on requirements
  of
• The applications (Bio HEP)
• The Operations

6
EGEE Milestones and Deliverables for the first
year
Month Deliverables Milestones Item Lead Partner
M03 MJRA1.1 Tools for middleware engineering and integration deployed CERN
M03 DJRA1.1 (Document) Architecture and Planning (Release 1) CERN
M03 MJRA1.2 Software cluster development and testing infrastructure available CERN
M05 MJRA1.3 Integration and testing infrastructure in place including test plans (Release 1) CERN
M05 DJRA1.2 (Document) Design of grid services (Release 1) CERN
M09 MJRA1.4 Software for the Release Candidate 1 CERN
M10 MJRA1.5 Integrated Release Candidate 1 enters testing and validation period (Release 1) CERN
M12 DJRA1.3 (Software) Software and associated documentation (Release 1) CERN
7
Working within ARDA

• Working in ARDA middleware task force
• Presented by Miron Livny at ARDA workshop
• Promoting an integrated approach to info services
• Hope document will be available soon
• First Web Service prototype for R-GMA
• Within two weeks

8
R-GMA Future Work

• Functionality
• Registry Schema Replication
• Enhanced Mediator
• Authorization
• Multiple VO Support
• Re-engineering
• Packaging
• Web Services

9
Registry Schema Replication

• Registry Replication
• Each logical Registry has multiple physical
  copies
• Transmit new and deleted records and a checksum
• Consumer Producer use any Registry instance
• Fail over mechanism
• Prototype complete
• Schema Replication
• Mechanism to vote for a master schema
• Synchronise all Schemas with the master

10
Enhanced Mediator

• Mediator brokers the correct Producer/s for a
  Consumer query
• Meta-data to be made available to Consumers via
  RGMAWarnings
• Support for republisher hierarchies
• Ability to deal with more complex predicates

11
Authorization

• Reviewing requirements
• Considering the best way to express the required
  authorisation semantics
• Looking into GACL syntax

12
Multiple VOs

• Currently one virtual Registry Schema per grid
• Used by all VOs
• Moving to one virtual Registry Schema per VO
• To be added as part of the EGEE re-engineering
• Information to multiple VOs need only be
  published once

13
Re-Engineering

• EGEE provides good opportunity to improve upon
  existing design
• Producing a documented design
• Agreed UML diagrams
• Will refactor to this design
• Looking into JMS
• To enable a pluggable streaming interface between
  Producers and Consumers

14
Quality

• Emphasis on better quality control from
  experiences with EDG
• System testing
• Currently using CMS testing framework with Nagios
  alerts
• Need to identify bugs/problems early on
• Improving approach to code design
  implementation
• Using UML to document new ideas
• More unit testing to improve code coverage

15
Packaging

• Remove dependency on a particular platform
• Provide distribution in tar/zip format

16
Web Services

• Currently we have Schema, Registry and Consumer
  Web Services
• Only Producers left to be converted (a few days
  work)
• Provide wrappers for compatibility with our
  current APIs
• Using eu-egee name space
• Continue to follow emerging standards
• WS-RF
• WS-Notification

17
Summary

• R-GMA robustness improved considerably
• R-GMA being used by BaBar/Southern Tier 2, CMS,
  D0, LCG
• First release for EGEE due in December 2004
• Working within ARDA to provide Information
  Monitoring Service
• Web service prototype well under way