Grid Information Service

1
Grid Information Service Globus MDS

• ? ? ?
• Supercomputing Research Department
• Supercomputing Center, KISTI

2
Contents

• Grid Information Service
• Grid Information Service in Globus toolkitTM
• MDS configuration

3
Grid Information Service- Introduction- General
infrastructure- Service architecture
4
Introduction

• Goal
• Seamless integration with diverse and
  heterogeneous resources so as to realize Grid
  computing
• Configuration and adaptation in heterogeneous,
  dynamic environments
• Maintain metadata related with resources
  efficiently
• Information Hardware, software, and status of
  the system
• Operation Add, Delete, Update, Search
• Main functions
• Resource discovery
• Resource monitoring
• Resource lookup

5
Introduction(cont)

• Requirements
• Scalability
• Scalable, efficient access to dynamic data
• Performance
• Access to static and dynamic information
• Access to multiple information sources
• Robustness
• Decentralized maintenance
• Uniformity
• Uniform, flexible access to information
• Security
• Access control

6
General infrastructure

• Services
• Resource information service
• Uniform means of querying resources for their
  current configuration, capabilities, and status
• Answer queries about a particular resource by
  directing the query to an information provider
• Grid Resource Information Service(GRIS) in
  Metacomputing Directory Service(MDS)
• Aggregate directory service
• Means of combining arbitrary GRIS services to
  provide aggregate view that can be explored or
  searched
• Provide the collective-level searching function
• Grid Index Information Service(GIIS) in MDS
• Protocols
• Enquiry(GRIP) and registration(GRRP) protocol

7
Service architecture
8
2. Grid Information Service in Globus toolkit-
Metacomputing Directory Service(MDS)
9
MDS

• Grid information service in Globus toolkitTM
• LDAP based information service infrastructure
• Use OpenLDAP software(http//www.openldap.org)
• Currently, version 2.1
• Components
• Data representation and API
• Defined by LDAP v3.0
• Information model
• Schema definition
• Implementation
• Performance, multiple data sources, and
  scalability

10
MDS(cont)

• Protocols
• Grid Resource Information Protocol(GRIP)
• Support both discovery and inquiry
• Adopt standard LDAP protocol
• Data representation
• Query language
• Wire protocol
• Grid Resource Registration Protocol(GRRP)
• Support notification and invitation
• Soft-state registration protocol
• Fault resilient
• Adopt LDAP as transport
• GRRP messages mapped onto LDAP add operations

11
MDS(cont)

• Services
• Grid Resource Information Service(GRIS)
• Implemented as an OpenLDAP server backend that
  can be customized by plugging in specific
  information sources
• Use OpenLDAP Generic modules API
• Additional modules
• Filtering results returned
• by an information provider
• Caching mechanism using TTL

12
MDS(cont)

• Grid Index Information Service(GIIS)
• Provide the hierarchical structure based on LDAP
  Directory Information Tree(DIT)
• Similarly, use OpenLDAP extension mechanism
• Pluggable index construction
• Pluggable search handling
• Generic GRRP handling by the front end
• Chaining and caching
• Redirection(planned)
• Use LDAP v3 referral

13
MDS(cont)
14
MDS(cont)

• Core schema
• Used by core GRIS information providers
• platform type, host OS, system load, memory, file
  system, etc.
• Root entry of all MDS data types
• Mds(OID 1.3.6.1.4.1.3536.2.6) object class type
• OID assignment 1.3.6.1.4.1.3536.2.6.
• Naming assignment Name prefix MDS
• e.g. MdsHost, MdsDevice, MdsComputer, etc.
• User defined schema
• Used by customized information providers
• host statistics, network status, storage or I/O,
  Application info., etc.
• Naming and OID assignment PEN from IANA

15
MDS(cont)

• Security
• Authentication
• GSI single sign-on and public-key security
  mechanism
• Verify credentials
• Mutual authentication
• OpenLDAP SASL mechanism
• Authorization
• Access control policies
• Fine-grained access control by GSI name
• LDAP access control mechanism

16
3. MDS configuration- Overall procedures-
Testbed scenario- MDS configuration files
17
Overall procedures

• Step 1 Design MDS hierarchy and define
  necessary GIS services
• Step 2 Determine the Distinguished Name(DN) for
  GIS services
• e.g., dckisti, dckr, oGrid
• Step 3 Install, setup and deploy Globus toolkit
  on each server
• globus-install, globus-setup, globus-local-deploy

18
Overall procedures(cont)

• Step 4 Modify a few configuration files
  manually
• Configuration file for daemon processes
• deploy_dir/etc/globus-daemons.conf
• Configuration file for each daemon process
• deploy_dir/etc/grid-info-resource-register.conf
• deploy_dir/etc/grid-info-resource-slapd.conf
• Configuration file for access control
• deploy_dir/etc/grid-info-site.conf
• Step 5 Test and Running
• e.g., MDS tools or LDAP browser

19
How to configure GIS

• Configuration for root-node
• Daemon processes for 2 LDAP servers(GRIS, GIIS)
• Registration process for GRIS
• Configuration for intermediate-node
• Daemon processes for 2 LDAP servers(GRIS, GIIS)
• Registration process for GRIS and GIIS
• Configuration for leaf-node
• Setup daemon process for a LDAP server(GRIS)
• Registration process for GIIS

20
Testbed scenario

• Configuration
• Hardware
• 4 4 node linux cluster for computational nodes
• 2 linux pc for stand-alone GIIS server
• Software
• Globus toolkit 1.1.4 with GIIS patch
• Setup parameters
• Port number
• GIIS service 2167
• GRIS service 2135

21
MDS hierarchy for the scenario
22
Configuration files for root-node GIS

• Hostname mds.hpcnet.ne.kr
• globus-daemons.conf
• grid-info-site-xx.slapd.conf

- libexecdir/slapd -p 2167 -d 1 -f
sysconfdir/grid-info-site-xx.slapd.conf -
sbindir/grid-info-soft-register -log
localstatedir/grid-info-system.log -f
sysconfdir/grid-info-resource-register.conf --
libexecdir/slapd -p 2135 -d 1 -f
sysconfdir/grid-info-resource-slapd.conf
database shell suffix
"oGrid" suffix "dckr, oGrid" search
/opt/globus/libexec/grid-info-site-backend
add /opt/globus/libexec/grid-info-site
-backend
23
Configuration files for root-node GIS

• grid-info-resource-register.conf
• grid-info-resource-slapd.conf

dn serviceMDS Resource, hnmds.hpcnet.ne.kr,
serviceMDS Registration, dckr, oGrid regtype
mdsreg reghn mds.hpcnet.ne.kr regport
2167 regperiod 300 type ldap hn
mds.hpcnet.ne.kr port 2135 rootdn
hnmds.hpcnet.ne.kr, dckr, oGrid ttl
600 timeout 60 mode cachedump cachettl 30
database shell suffix "dckr,
oGrid" suffix "oGrid" search
/opt/globus/libexec/grid-info-resource-search
24
Configuration files for root-node GIS

• grid-info-site.conf

dn conf regdir localstatedir/grid-info-site-r
egdir cachedir localstatedir/grid-info-site-ca
che defaultttl 300 maxttl 2880 minttl
120 defaulttimelimit 10 maxtimelimit
300 mintimelimit 5 this entry is for
access-control only. it uses all the global
configuration data set above. dn serviceMDS
Resource, hn, serviceMDS Registration, dckr,
oGrid
25
Configuration files for intermediate-node GIS

• Hostname sdd36.hpcnet.ne.kr
• globus-daemons.conf
• grid-info-site-xx.slapd.conf

- sbindir/grid-info-soft-register -log
localstatedir/grid-info-system.log -f
sysconfdir/grid-info-xx-register.conf --
libexecdir/slapd -p 2167 -d 1 -f
sysconfdir/grid-info-site-xx.slapd.conf -
sbindir/grid-info-soft-register -log
localstatedir/grid-info-system.log -f
sysconfdir/grid-info-resource-register.conf --
libexecdir/slapd -p 2135 -d 1 -f
sysconfdir/grid-info-resource-slapd.conf
referral ldap//mds.hpcnet.ne.kr2167/dckr
,oGrid database shell suffix
"oGrid" suffix "dckisti, dckr,
oGrid" search /opt/globus/libexec/grid-i
nfo-site-backend add
/opt/globus/libexec/grid-info-site-backend
26
Configuration files for intermediate-node GIS

• grid-info-xx-register.conf

dn serviceMDS Resource, hnsdd36.hpcnet.ne.kr,
serviceMDS Registration, dckisti, dckr,
oGrid regtype mdsreg reghn mds.hpcnet.ne.kr reg
port 2167 regperiod 300 type ldap hn
sdd36.hpcnet.ne.kr port 2167 rootdn dckisti,
dckr, oGrid ttl 600 timeout 60 mode
cachedump cachettl 30
27
Configuration files for intermediate-node GIS

• grid-info-resource-register.conf
• grid-info-resource-slapd.conf

dn serviceMDS Resource, hnsdd36.hpcnet.ne.kr,
serviceMDS Registration, dckisti, dckr,
oGrid regtype mdsreg reghn sdd36.hpcnet.ne.kr r
egport 2167 regperiod 300 type ldap hn
sdd36.hpcnet.ne.kr port 2135 rootdn
hnsdd36.hpcnet.ne.kr, dckisti, dckr,
oGrid ttl 600 timeout 60 mode
cachedump cachettl 30
database shell suffix
"oGrid" suffix "dchpcnet, dcne,
dckr, oGrid" search /opt/globus/libexec
/grid-info-site-backend add
/opt/globus/libexec/grid-info-site-backend
28
Configuration files for intermediate-node GIS

• grid-info-site.conf

dn conf regdir localstatedir/grid-info-site-r
egdir cachedir localstatedir/grid-info-site-ca
che defaultttl 300 maxttl 2880 minttl
120 defaulttimelimit 10 maxtimelimit
300 mintimelimit 5 this entry is for
access-control only. it uses all the global
configuration data set above. dn serviceMDS
Resource, hn, serviceMDS Registration,
dckisti, dckr, oGrid
29
Configuration files for leaf-node GIS

• Hostname sdd113.hpcnet.ne.kr
• globus-daemons.conf
• grid-info-resource-slapd.conf

- sbindir/grid-info-soft-register -log
localstatedir/grid-info-system.log -f
sysconfdir/grid-info-resource-register.conf --
libexecdir/slapd -p 2135 -d 1 -f
sysconfdir/grid-info-resource-slapd.conf
database shell suffix "dcsrd,
dckisti, dckr, oGrid" suffix
"oGrid" search /opt/globus/libexec/grid-
info-resource-search
30
Configuration files for leaf-node GIS

• grid-info-resource-register.conf

dn serviceMDS Resource, hnsdd113.hpcnet.ne.kr,
serviceMDS Registration, dcs rd, dckisti,
dckr, oGrid regtype mdsreg reghn
sdd124.hpcnet.ne.kr regport 2167 regperiod
300 type ldap hn sdd113.hpcnet.ne.kr port
2135 rootdn hnsdd113.hpcnet.ne.kr, dcsrd,
dckisti, dckr, oGrid ttl 600 timeout
60 mode cachedump cachettl 30
31
Configuration files for leaf-node GIS

• grid-info-site.conf

dn conf regdir localstatedir/grid-info-site-r
egdir cachedir localstatedir/grid-info-site-ca
che defaultttl 300 maxttl 2880 minttl
120 defaulttimelimit 10 maxtimelimit
300 mintimelimit 5 this entry is for
access-control only. it uses all the global
configuration data set above. dn serviceMDS
Resource, hn, serviceMDS Registration, dcsrd,
dckisti, dckr, oGrid
32
MDS test

• Use MDS search tool grid-info-search
• Search on the GIS server based on search filters
• Conform to LDAP search
• grid-info-search options ltsearch filtergt
  attributes
• grid-info-search h mds.hpcnet.ne.kr p 2167
  (objectclass)
• Use LDAP browser
• Based on JAVA GUI
• Support to searchand browse LDAP data
• http//www-unix.mcs.anl.gov/gawor/ldap/index.html

33
MDS test

• Korea GIIS

34
MDS test

• KISTI GIIS

35
MDS test

• SRD GIIS

36
MDS test

• GRIS

37
MDS hierarchy for NGrid testbed