Grid%20Resource%20Allocation%20and%20Management%20(GRAM)

1
(No Transcript)
2
Grid Resource Allocation and Management (GRAM)

• Execution management
• Deployment, scheduling and monitoring
• Community Scheduler Framework (CSF) Provides a
  single interface to different resource
  schedulers.
• PBS, Condor(G).
• Workspace management
• Dynamically create and manage workspaces on
  remote hosts.
• Grid Telecontrol Protocol
• WSRF-enabled service interface for control of
  remote instruments.
• Remote goldfish surgical procedures.

3

• Jobs are computational tasks that may perform
  input/output operations while running.
• Affect the state of the computational resource
  and its associated file systems.
• May require coordinated staging of data into the
  resource prior to job execution and out of the
  resource following execution.
• Some users, particularly interactive ones,
  benefit from accessing output data files as the
  job is running. Monitoring consists of querying
  and subscribing for status information such as
  job state changes.

4

• Monitoring consists of querying and subscribing
  for status information such as job state changes.
  
• Operated under the control of a scheduler which
  implements allocation and prioritization policies
  (i.e., priorities).
• GRAM is not a resource scheduler but a protocol
  engine for communicating with different local
  resource schedulers.

5
Conceptual Details

• Targeted Job Types
• Not RPC
• reliable operation, stateful monitoring,
  credential management, and file staging are
  important (i.e., the performance is horrible so
  only use if necessary).

6
Component Architecture

• Based on Component architecture
• Job management services
• represent, monitor, and control the overall job
  life cycle. These services are the job-management
  specific software provided by the GRAM solution.
• File transfer services
• support staging of files into and out of compute
  resources.

7
Component Architecture

• Credential management services
• are used to control the delegation of rights
  among distributed elements of the GRAM
  architecture based on users' application
  requirements.

8
Security

• Secure Operation
• WS GRAM utilizes WSRF functionality to provide
  for authentication of job management requests as
  well as to protect job requests from malicious
  interference.
• Local System protection domains
• jobs are executed in appropriate local security
  contexts
• e.g. under specific Unix user IDs based on
  details of the job request and authorization
  policies.

9

• Credential delegation and management
• Client may delegate some of its rights to GRAM
  services
• e.g. rights for GRAM to access data on a remote
  storage element as part of the job execution.
• Audit
• To assist with normal accounting functions as
  well as to further mitigate risks from abuse or
  malfunction.

10
Job Management

• Reliable job submission.
• at most once semantics
• Job Cancellation
• a mechanism for clients to cancel (abort) their
  jobs at any point in the job life cycle.

11
Data Management

• Reliable Data Staging
• reliable, high-performance transfers of files
  between the compute resource and external
  (gridftp) data storage elements before and after
  the job execution.
• Output Monitoring
• mechanism for incrementally transferring output
  file contents from the computation resource while
  the job is running.

12
Task Coordination

• Parallel Jobs
• Task rendezvous
• mechanism for task rendezvous which job
  applications may use if they do not have another
  more appropriate solution
• Usually done in MPI

13

• WS-GRAM (Web Services version).
• Designed to support job execution with
  coordinated file staging.
• Uses a set of Web services in the GT4 WSRF core.
• ManagedJob Provides interface to monitor the
  status of the job, terminate. Each submitted job
  is a distinct resource.
• ManagedJobFactory Interface to create ManagedJob
  resources of appropriate type to perform a job
  in that local scheduler.
• ManagedJob resource creation ManagedJobFactoryc
  reateManagedJob invocation.

14
(No Transcript)
15

• Creation of Job
• ManagedJobFactorycreateManagedJob invocation.
• A meaningful WS GRAM client MUST create a job
  that will then go through a life cycle where it
  eventually completes execution and the resource
  is eventually destroyed
• Optional Staging Credentials
• Must be performed before call to createMnagedJob
• Optional Job Credential
• Store into user account for use by job process.

16

• Optional Credential Refresh
• Credentials delegated may be refreshed.
• Optional Hold of Cleanup
• User wants to directly access output files
  without waiting for stage-out.
• ManagedJob Destruction
• Can explicitly destroy job.

17
Globus Toolkit Components used by WS GRAM

• Reliable File Transfer (RFT)
• For file staging before and after job completes.
• GridFTP
• Supports retry
• Partial file transfer
• 3rd party file transfer

18
GridFTP
FOO2
FOO1
19
GridFTP
FOO2
FOO1
20
Delegation Services

• Can delegate credentials to any service that is
  deployed in the same container as the service.
• Tells delegation service it wants to delegate its
  credentials.
• The service that wants to use them must contact
  the delegation service to acquire them.

21
External Components Used by WS GRAM

• Local job scheduler
• PBS, LSF, Condor
• Sudo
• Access to user accounts without having root
  privilege.

22
(No Transcript)