The eMinerals integrated compute and data grid for molecular simulations

1

• The eMinerals integrated compute and data grid
  for molecular simulations
• Martin Dove
• Department of Earth Sciences, University of
  Cambridge
• and
• National Institute for Environmental eScience

2
Outline of presentation

• Introduction to the eMinerals project
• Integrated compute and data grid
• Demonstration

3
Molecular environmental issues
Radioactive waste disposal
Pollution molecules and atoms on mineral surfaces
Crystal dissolution and weathering
Crystal growth and scale inhibition
4
Molecular environmental issues
Radioactive waste disposal
Pollution molecules and atoms on mineral surfaces
Crystal dissolution and weathering
Crystal growth and scale inhibition
5
Applications
Simulations of dissolution of quartz, SiO2, in
water as part of study of weathering processes
6
Example Pollutant molecules on mineral surfaces

7
eMinerals team
8
Elements of escience
Computing grids
Data grids
Collaborative grids
9
eMinerals integrated compute/data minigrid
10
eMinerals minigrid compute component
11
eMinerals minigrid data and SRB component
12
eMinerals compute grid

• Compute grid built around shared resources
• 4 Intel commodity clusters (total 88 processors)
• IBM pSeries parallel computer (24 processors)
• Apple Xserve cluster (4 nodes, 8 GB/node)
• 25-node Condor pool in Cambridge (12 GB/node)
• 930-node Condor pool in UCL
• Access to compute resources via GSI / X.509
  digital certificates
• Data managed through the storage resource broker,
  with 4 TB data vaults

13
Integrated compute/data grid

• Access to compute resources via Globus (GT 2
  functionality) job run commands
• Authentication via GSI / X.509 digital
  certificates
• Limited access via gsissh
• Job submission via Condor-G and Condor DAGman
• Relatively easy interface to the grid
• Data files are handled via the SRB

14
Job submission to the minigrid

• Simple workflow
• User places data files onto the SRB, possibly
  also the application executable
• The job is transferred to one of the compute
  resources
• The files are downloaded from the SRB using the
  Sget command
• Once the job has executed, the new output files
  are uploaded to the SRB using the Sput command

15
Job submission to the minigrid
Universe globus Globusscheduler
ltminigrid resourcegt/jobmanager-ltjobmanagergt Execut
able ltname of executable binary or
scriptgt Notification NEVER GlobusRSL
(argumentsnone)(job_typesingle)(stdinltfilenamegt
) Sdir ltsome directory in the
SRBgt Sget ltlist of input file names, or
for wildcardgt Sput ltlist of output
file names, or for wildcardgt Output
ltstandard output file namegt transfer_output
False Log ltname of log filegt Error
ltname of standard error filegt Queue
16
eMinerals minigrid demonstration

• Show the data files previously placed onto the
  SRB
• Execute a job on the Lake resource
• Final results obtained from the SRB

17
eMinerals minigrid demonstration
Upper Mantle consists of Mg2SiO4 There are two
stable phases, olivine and spinel
18
Summary

• The eMinerals project has built a minigrid
  infrastructure
• Heterogenous hardware
• Mix of Condor, PBS, Sun Grid Engine, IBM
  LoadLeveller
• Storage Resource Broker
• Access via Globus GSI, using Condor-G
• Data transfers handled via the SRB
• We have implemented our own condor_submit wrapper
  for integrated minigrid