Grid data storage

1
Grid data storage
2
What is the Grid?
Storage resource
Compute resource
Grid middleware
Compute resource
Storage resource
3
Issues under research

• Authentication
• Method (passwords, certificates)
• Infrastructure
• Access
• Job submission and batch processing (Condor)
• Log on to machine (gsissh)
• Service-oriented (web services, grid services)

4
Services

• Look up DNA sequence with this ID
• Look up similar sequences to this DNA sequence

• Predict what genes are involved in this sequence
• Predict what proteins might be produced from
  this sequence

5
Workflows
6
Workflow results

• Raw data
• Metadata
• Information about the data itself (types)
• Information about how the data was produced
  (when, by what service, by whom)

7
Results storage

• As files on hard disk
• Leave results management to the users
• MyGrid Information Repository (MIR)
• Accessed using SOAP web-service calls
• Specifically designed for storage and query of
  workflow results
• As files with metadata on Storage Resource Broker
  (SRB)
• Accessed by SRB clients over tcp
• Use SRB facilities for authentication and
  filesystem querying

8
SRB file system paradigm

• Looks like a normal file system

• Metadata per file/directory (FILE_NAME,
  OWNED_BY, UDSMD0, UDSMD1)

• Powerful query mechanism (list me all files
  produced on the 13th October by user sre which
  are workflow results)

9
SRB architecture
10
SRB _at_ MRL

• MCAT and SRB Server running on geiger
• MCAT running on Postgres
• JARGON api and Scommands are free to download
• Ill set up an account for you