EUROGRID - CCGRID2002

1
UNICORE Towards production quality grid
environment

Piotr Bala N. Copernicus
University Torun (PL) ICM Warsaw University (PL)
Funding by European Commission Grant No.
IST-1999-20247 (EUROGRID) and IST-2001-32257
(GRIP)
2
EUROGRID Vision

• Build a European Grid infrastructure
• that gives users
• a seamless, secure access to
• High Performance Computing resources
• and that advances computational science
• in Europe

3
GRIP Project

• Extend EUROGRID to cooperativity with globus
  middleware.
• Provide UNICORE users with access to resources
  avaliable trough globus.

4
EUROGRID Infrastructure and Components

• Based on UNICORE system
• Develop additional GRID components
• efficient data transfer
• ASP infrastructure
• resource broker
• application coupling
• interactive access
• Integration of new components by Intel (former
  Pallas) and Fecit

5
UNICORE architecture
UNICORE Client
UNICORE Gateway B
Insecure Internet
UNICORE Gateway A
UNICORE Server (NJS)
UNICORE Server (NJS)
TSI
TSI
NQE (Cray T3E)
PBS (Linux cluster)
6
UNICORE Security

• Based on the PKA
• Industrial standard X509
• Secure communication
• Gateway, NJS cetificates
• Gateway, NJS check user certificate
• Multiuple CA accepted
• User certificates
• User certificate stored in client
• Public key stored in UUDB database at each site
• PKA mapped to user account (XLOGIN)
• more than one certificate can map to xlogin
• multiple CA allowed
• Multiple certificates allowed

7
UNICORE

• Stable and reliable
• Allows access to production systems
• OpenSource
• Commercial Support Available (UNICORE Pro)
• Interoperability with Globus (GT2, GT3)
• (see R. Menday talk tomorrow)

8
UNICORE

• Stable and reliable
• Allows access to production systems
• OpenSource
• Commercial Support Available (UNICORE Pro)
• Interoperability with Globus (GT2, GT3)
• (see R. Menday talk tomorrow)
• Pratial support of OGSA standard
• (see next talk)

9
EUROGRID resources

• European HPC GRID testbed
• Agreement on security standards, certification,
  access policies etc.

10
UNICORE Applications

• Job files for user applications
• script tasks
• command tasks
• file transfer tasks (client-vsite, vsite-vsite)
• Plug-in for job and input preparation
• User can run the same script at all VSITEs
• Introduction of IDB entries for site dependent
  installation
• GROMOS96/pkg/gromos96 export GROMOS96
• GAUSS_EXEDIR/pkg/gaussian/g98 export
  GAUSS_EXEDIR
• AMBERHOME/pkg/amber6 export AMBERHOME
• CHARMM/pkg/charmm/c27b2/exec/t3e/charmm export
  CHARMM

11
BioGRID

• UNICORE Client (4.1.5 or UNICORE Pro 1.0.7) unix
  and Windows
• Easy installation (www.unicore.org,
  biogrid.icm.edu.pl)
• Example jobs
• Gaussian98, TBDFT, Amber, Gromos
• Plugins
• Gaussian98
• AMBER
• CPMD (V. Huber FZJ)
• BLASS (S. Haubold, TU Dresden)
• PDB Search
• Sequence Search
• Visualization Plugin
• PluginLoader
• List Jobs

12
UNICORE Client

• Single application
• Job preparation
• Job monitoring

13
UNICORE Client
14
UNICORE Client Gaussian98 job preparation
15
UNICORE Client edit resources for job
16
Gaussian98 in EUROGRID
17
Gaussian98 plugin

• Input preparation
• reads existing input, recognizes keywords
• Molecule coordinate editor
• formats
• XYZ
• Z-matrix
• text

18
Gaussian98 plugin

• CPU time estimate
• based on known algorithm scaling O(N4)
• Checks program avaliability

19
Gaussian98 plugin

• Prepares and transfers files for visualization

20
Gaussian98 plugin

• Output

21
Gaussian Plugin J. Pytlinski, A. Nowinski

• Visualization panel

22
Gaussian Plugin J. Pytlinski, A. Nowinski
23
Gaussian Plugin J. Pytlinski, A. Nowinski
24
Gaussian Plugin J. Pytlinski, A. Nowinski
25
Amber 6.0 plugin L. Skorwider

• Input preparation
• extended input preparation wizard
• covers all most popular options for NVT, NVE, NPT
  simulations
• NMR simulations are not covered yet
• Extended user help
• short help available for each button
• Keyword list and keyword locator
• user is navigated to the proper panel
• JavaHelp entries
• Plugin checks Amber availability
• SOFTWARE_RESOURCES entries

26
Amber plugin

• Input preparation

27
Amber plugin

• Input can be edited by user
• Errors are marked in red

28
Amber plugin

• Panel for each group of options
• Default options
• Help window

29
Amber plugin

• Find parameter button

30
Visualizer Plugin K. Benedyczak
31
Visualizer Plugin K. Benedyczak
32
CPMD plugin
V. Huber (FZJ)

• CPMD input

33
CPMD plugin

• CPMD wizard

34
PDB search plugin
M. Wronski

• Prepares database query
• Protein Data Bank (PDB) database
• Sequence database
• Databases available through web interface
• Plugin uses HTTP protocol to generate query
• Database mirrored at various locations
• Output visualization
• PDB structure can be visualized in the plugin
  using external software
• JMV (Java Molecular Viewer)
• K. Shulten group Chicago
• Requires Java3D
• JMol
• Sourceforge
• RasMol/RasWin

35
PDB search plugin

• Lite search

36
PDB search plugin

• Advanced search

37
PDB search plugin

• User can use different mirrors
• The best one is selected
• (fastest answer)

38
PDB search plugin

• Search results

39
PDB search plugin

• Visualization with external packages

40
PDB search plugin

• Rasmol
• JMV
• Jmol
• User-defined

41
Sequence search plugin
M. Wronski

• Search interface

42
Sequence search plugin

• Datailed view

43
Sequence search plugin

• Simple visualizatio based on BIOJava

44
Blast Plugin S. Haubold (TU Dresden)

• Input preparation

45
List All Jobs Plugin
K. Benedyczak

• List all users jobs
• Scans USITEs and VSITEs
• Lists Job information
• Information on running jobs
• Alternative mechanism based on the additional
  buttons which can be defined in the Job window
  can be used.
• Direct connection to JMC pannel

46
List All Jobs Plugin
47
File Filter Plugin
J. Wypychowski

• Access to the work directory of the running job
• Filter status file
• Status file
• Intermediate files (eg. coordinates)
• Display results in the Client
• Task running as separate job
• Immediate queue
• User has to be able to run two jobs

48
File filter plugin

• Pick up job from JMC
• Define file to process
• Submit job

49
File filter plugin

• Tail of the Amber status file

50
DBAccess Plugin
M. Wronski

• Access to the SQL databases
• MySQL
• Postgress
• ...
• Results can be saved in the file
• XML output

51
Generic plugin - basic idea J. Wypychowski

• Plugin is not a higher level language program
  No Java
• Applications typically require a set of
  parameters from a basic set of types
  (integer, string, float)
• Constraints for the input data must be available
• UNICORE tasks and interaction must be simple
• Interface should generate itself on the fly
• Text based configuration
  (XML)
• Independent from UNICORE internal representation

52
GenericPlugin - implementation

• Plugin within plugin generic plugin generator
• XML based configuration with strict grammar
  definition
• Small set of basic parameter types
• Auto-generated interface
• Constraints and parameters are scriptable with
  JavaScript
• Simplified UNICORE tasks and data
• Common API to create application plugins
• VSite based application plugin storage

53
GenericPlugin in UNICORE
54
Script Plugin in XML

• 01 lt?xml version"1.0"?gt
• 02 lt!DOCTYPE application SYSTEM "plugin.dtd"gt
• 03 ltapplication name"script"gt
• 04 ltresource type"context"gtshlt/resourcegt
• 05 ltsection name"main" /gt
• 06 ltparameter name"contents"
  section"main"gt lttype_string big_area"yes"/gt
  lt/parametergt
• 07 ltjob namejobgt
• 08 ltscript namescriptgt
  ltcontextgtshlt/contextgt
• 09 ltcontentsgt ltvariable
  namecontents/gt lt/contentsgt
• 11 lt/scriptgt
• 12 lt/jobgt
• 13 lt/applicationgt

55
DeviceSteer Krzysztof Benedyczak

• Access to the external devices
• Thermometer connected through serial port to TSI
  server
• Deamon for the device running on the target
  system
• DeviceSteer Plugin

56
DeviceSteer
57
DeviceSteer
58
BioGRID

• Operate a GRID for biomolecular simulations
• Develop interfaces to existing biological
  andchemical codes
• Web site biogrid.icm.edu.pl

59
Web site

• Information on
• software
• info
• home page
• local info
• VSITEs
• USITEs

60
Web site

• Step by step user guides

61
biogrid.icm.edu.pl www.eurogrid.org www.grid-inter
operability.org
Piotr Bala J. Pytlinski, L. Skorwider, M.
Wronski, K. Benedyczak, J. Wypychowski, A.
Nowinski, M. Nazaruk, K. Wawruch ICM Warsaw
University (PL)
N. Copernicus University
Torun (PL)
Funding by European Commission Grant No.
IST-1999-20247 (EUROGRID) and IST-2001-32257
(GRIP)
62
EUROGRID Workshop

• Eurogrid Workshop (29.10.2003)
  Cracow Grid Workshop 2003 Kraków, 27-29.10.2003
• UNICORE Tutorial 28.10.2003 - evening
• Room for 30-40 people
• Lectures 29.10.2003 morning
• UNICORE overview - K.-D. Ortel, Intel
• Meteo-GRID Performing Local Weather Forecast
  Using GRID Computing - K.-J. Lenz, DWD
• Globus/Unicore interoperability - R. Menday, FZJ
• Plugin development - K. Benedyczak, M.
  Wronski ICM/UMK
• Unicore in OpenMolgrid project - M. Romberg, FZJ