Information Services Grid Computers for Scheduler Vladimir Silva

1
Information Services Grid Computers for
Scheduler -Vladimir Silva

• June Seminar (Reading Session)-7/6/07
• Shamim Akhter
• PhD Student
• AIDA Lab
• Department of Information Processing
• Tokyo Institute of Technology, Japan

2
Information Services

• Known as Monitoring and Discovery Services (MDS).
• Provides information for resource discovery,
  selection, and optimization.
• Can be used by applications or virtual
  organizations (VO) to obtain the capabilities and
  services provided by grid.
• Globus devided the MDS as-
• Pre-Web Service (MDS2)
• WS Information Services (MDS3)

3
WS Information Services (MDS3)

• MDS3 is implemented as a set of grid services.
• Grid services classified as
• Persistent-if they outlive the process that
  created them.
• Transient-if otherwise.
• Services are identified by one or more instances
  and express their state via Service Data Elements
  (SDE)

4
MDS3-Information Model

• MDS has used the model based on OGSA(Open Grid
  Service Architecture).
• Factories-Object that create service instances.
  Return a GSH and maintain service data element.
• Grid Service Handle (GSH)-Unique Identifier for
  service. Convert to GSR before the service can be
  used.
• Grid Service References (GSR)-Included the GSH
  and binding information for transport protocol
  and data encoding format.
• Registry Services- A repository for GSHs. To
  register the GSHs for service discovery.
• Notification services-used by client subscription
  to send asynchronous messages between services.

5
MDS3-Data Collection

• Resource information is collected via service
  data provides.
• Providers are external program that generate
  service data dynamically.
• Some are from Globus Toolkit or some from by the
  developers.
• Data provides can be connected to service
  instances.

6
MSD3-Aggregation

• Services data are generated by providers and
  delivered from other grid services can be
  presented in different aggregate data views.
• Notification or subscription mechanisms can then
  be applied to service data by command or GUI
  clients.

7
MSD3-Queries

• MDS3 provides a standard, extensible query
  interface to service data elements.
• Queries can be executed by SDE name or XPath or
  XQuery.

8
User Interface

• GT3 provides two types of user interfaces to
  query Information Services
• Service browser GUI
• Set of command line query tools

Security

• The index service is compatible with Grid
  Security Infrastructure (GSI).
• GSI provides a single sign-on authentication
  service and it is built on open standards such
  as X.509 and proxy certificates.

9
Default Data Providers

• SimpleSystemInformation
• HostScriptProvider

Custom MDS3 Data ProviderHost information,
Resource availability, Network status

• GT3.2 Core Data Provide
• SimpleSystemInformationProvider Provides
  information about the host, CPU, memory and disk
  volume.
• HostScriptProvider-A set of Unix shell scripts to
  provide host resource information.
• ScriptExecutionProvider-A provider to execute
  shell scripts
• AsyncDocumentProvider-An asynchronous utility to
  read an XML periodically implements the
  AsyncDataProvider interface.

10
Real World An MDS3 Information Provider for
Remote Schedulers Using SSH
Create Information Service for MDS3

• Remote Cluster Information Provider
• Remote Server must have a Secure Shell (SSH)

Advantages over default Globus Provider

• Supports many simultaneous schedulers Condor
  6.6.6, OpenPBS and SGE 6.x
• Can be run from any GT3 container, such as
  windows Unix.
• Can be easily extended to support new schedulers.

11
Provider Architecture
Query
SSH Client
Scheduler
Contains Hostname, port, protocol. SSH supported
authentication via password or RSA private key
12
Pre-WS Information Services (MDS2)

• MDS2 is a information services component of
  Globus toolkit.
• MDS2 gathers information about grid resources by
  mean of the Grid Resource Information Service
  (GRIS) and Grid Index Information Service (GIIS).
• The MDS component of Globus Toolkit uses
  extensible framework for managing static and
  dynamic information about the status of
  computational grid and all its components
  networks, computer nodes, storage systems and
  instruments.

13
Benefits of MDS

• Access to static and dynamic information about
  system components.
• Uniform flexible access to information.
• Access to multiple information sources.
• A basis for configuration and adaptation in
  heterogeneous, dynamic environments.
• Decentralized maintenance.
• MDS can be used to answer the following
  questions-
• What resources are available?
• What is the state of the computational grid?
• How can applications be optimized based on the
  configuration of the underlying system?

14
Architecture GRIS and GIIS
Middleware information In a common interface
Query
Distributed Information Service
Current configuration Capabilities and status
15
Implementing a Grid Information Provider for MDS2

• LDAP(Light Weight Directory Access Service)
  Configuration
• GRIS to publish share file for each resource
  and machine.
• Java Commodity Grid (CoG) kit API to search for
  downloadable files in an application that
  resembles the music and file sharing application.
• GridFTP-download and upload files on the remote
  resources.
• The configuration file is located on
  GLOBUS_LOCATION/etc/grid-info-slaped.conf

16

• Modifying the grid-info-resource-ldif.conf file
• This file describes how the information provider
  will be called and what it should emit.
• Located in GLOBUS_LOCATION/etc
• Writing the Information Provider Program
• You can use any language and can pass any
  augments that you need.
• The only requirements are as follows
• The I/O interface of the program must be callable
  by the UNIX system calls fork and exec in the
  LDAP main process (slapd).
• The data returned must match the LDAP schema
  described in the LDAP Data Interchange Format
  (LDIF)

17

• Running the Information Provider Program
• You can run the program from the command line by
  issuing the following commend
• ./grid-info-fileshare devclassobj devobjs dn
  Mds-Host-hnHOSTNAME, Mds-Vo-namelocal, ogrid
  validto-secs 60 keepto-secs 900
• Information Provider Installation
• Restart the Globus service on OS
• Service globus stop
• Service globus start

18
Summary

• Information service (MDS) provide resource
  directory, selection and optimization.
• Applications can used then to obtain the
  capabilities and services provided by
  Computational Grid.