CLRC eScience Centre

1
CLRC e-Science Centre
A Web Portal for 4-D Climate Databases Kerstin
Kleese -van Dam k.kleese_at_dl.ac.uk
2
ESTEDI within CLRC (1)
CLRC has been involved in Grid developments since
they reached Europe (e.g. DataGrid and WOS in
1999)
Internal progress of e-Science work Spring 2000
- Internal Co-ordination Group set up October
2000 - Internal committee set up, start of
prototype developments - funding 300K April 2001
- CLRC e-Science Centre founded with 5M over 3
years government funding July 2001 - UK Grid
Support Centre launched (government funded)
3
ESTEDI within CLRC (2)
CLRC aims to integrate all its facilities
(experiment - data - computing), make them grid
aware and link them with others.
Through the growing importance of e-Science the
position of the ESTEDI project has changed
considerably over the last 18 month.
ESTEDI is now part of the CLRC e-Science
Programme and RasDaMan is seen as an important
building block in the Data Management strategy.
The initial testbed for RasDaMan is Earth Science.
4
Architecture for Grid access to Scientific Data
at CLRC
5
The Vision for CLRC - Services
Problem Solving Environments
CLRC Authentication
Computing Applications
Experimental Facilities
HPCPortal
Remote systems
Local systems
Exemplar for GRID deployment in wider research
community
6
Integrated Solution for Earth Science
Data Storage
DataPortal
RasDaMan
Disk
Tape
BADC Catalogue
SRB
HPCPortal
7
General CLRC DataPortal Architecture
CLRC DataPortal Server
XML wrapper
XML wrapper
Local metadata
Common metadata catalogue database
Local data
Facility 1
8
Server Architecture
USER
Key
User input interpreter
User output generator
Internal
http
pre-set XSL Script
Query Generator
module
Response Generator
XML Schema
XML Parser
External agent
Central metadata repository
Wrapper for other Catalogues
XML File
XML File
Ascii file
9
Architecture for integrating existing Catalogues
DataPortal Server
Request file(s)
XML Wrapper
Response Generator
SQL input translator
XML output generator
Local Metadata Catalogue
RasDaMan
SRB
External agent
10
Integration of SRB
RasDaMan
DataPortal
DB2, Oracle, Illustra, ObjectStore - HPSS,
Unitree - Unix, ftp
11
Metadata used for the ESTEDI Testbed
12
Metadata Formats for the Earth Science Testbed
CLRC DataPortal Server
XML wrapper
XML wrapper
CERA
CLRC General Scientific Metadata Format
Local data
BADC
13
Diversity Users Searches
Discovery
Excavation
Experimenter
Data curator
Wider science community
General community
Specialist user
14
General Scientific Metadata
A generic metadata model for all scientific
applications with Specialisation for each domain

Can answer questions across domains Can answer
questions about specific domains
15
CLRC DataPortal - Scientific Metadata Model
Metadata Object
Topic
Study Description
Access Conditions
Data Description
Data Location
Related Material
16
CLRC Metadata example
lt?xml version"1.0" encoding"UTF-8"?gt lt!DOCTYPE
CLRCMetadata SYSTEM "clrcmetadata.dtd"gt ltCLRCMetad
atagtltMetadataRecord metadataID"N000001"gt ltTopicgt
ltDisciplinegtChemistrylt/Disciplinegt ltSubjectgtC
rystal Structurelt/Subjectgt ltSubjectgtCopperlt/Subj
ectgt... ltExperimentgt ltStudyNamegtCrystal
Structure Copper Palladium complex 150K
... ltInvestigatorgtltNamegtltSurnamegtPorter...ltInsti
tutiongtUniversity of Peebles ... ltFundinggtEPSRC
... ltTimePeriodgtltStartDategtltDategt21/04/1999.
ltPurposegtltAbstractgt To study the structure of
Copper and Palladium co-ordination complexes at a
150K. ltDataManagergtltNamegtltSurnamegtTeat... ltIn
strumentgtSRS Station 9.8, BRUKER AXS SMART
1K... ltConditiongt...Wavelength...ltUnitsgtAngstr
om...ltParamValuegt0.6890... ltConditiongtCrystal-t
o-detector distanceltUnitsgtcm...ltParamValuegt5.00...
ltAccessConditionsgtThe user has to be one of
Prof. F. Porter.
17
Architecture of the Insertion Process
GRIB file
BADC/CERA
Metadata
SRB
Data
Insertion
More detailed Metadata Data itself
RasDaMan
18
Existing CERA Metadata Model
19
Integration of Cera and RasDaMan
Each GRIB file is assigned one entry in the CERA
repository The new table RAS_INFO contains the
overall temporal and spatial coverage of each.
RasDaMan MDD (one entry per parameter 1
RasDaMan collection 1MDD) The new table
RAS_CONNECT assigns each CERA entry ( GRIB
message) the name of the corresponding RasDaMan
collection that contains the MDD with the binary
data An auxiliary table RAS_ALTITUDES is used for
creating an index for the different levels of
altitude from the GRIB data.
20
User interface for the Earth Science Testbed
21
User Query Hierarchy
Key
Internal ANSI or RAS SQL
External
RasDaMan more detailed Metadata, selection,
manipulation
Local Catalogue e.g. BADC - CERA Ontology,
detailed Metadata, logical link to physical files
module
module
External agent
Checkout
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
More Information under
CLRC -
http//www.clrc.ac.uk CLRC e-Science Centre -
http//www.e-science.clrc.ac.uk UK Grid Support
Centre - http//www.grid-support.ac.uk/ ESTEDI
Project - http//www.estedi.org/ e
mail k.kleese_at_dl.ac.uk