Condor BirdBath SOAP Interface to Condor - PowerPoint PPT Presentation

About This Presentation
Title:

Condor BirdBath SOAP Interface to Condor

Description:

Condor BirdBath SOAP Interface to Condor Charaka Goonatilake Department of Computer Science University College London c.goonatilake_at_cs.ucl.ac.uk – PowerPoint PPT presentation

Number of Views:103
Avg rating:3.0/5.0
Slides: 12
Provided by: uca126
Category:

less

Transcript and Presenter's Notes

Title: Condor BirdBath SOAP Interface to Condor


1
Condor BirdBathSOAP Interface to Condor
  • Charaka Goonatilake
  • Department of Computer Science
  • University College London
  • c.goonatilake_at_cs.ucl.ac.uk

2
Background
3
Motivations
  • Adding Web Service interfaces to Condor daemons
  • Facilitate the development of third-party
    applications capable of interacting with Condor
    (remotely).
  • E.g. build higher-level application specific
    scheduler that submits jobs to multiple Condor
    pools based on application semantics
  • These can be built using a wide range of
    languages/SOAP packages
  • BirdBath has been tested on
  • Java (Apache Axis, XSUL)
  • Python (ZSI)
  • C (.Net)
  • C/C (gSOAP)
  • Condor accessible from platforms where its
    command-line tools are not supported/installed

4
The (Hypothetical) Big Picture
Site D
Condor-G
Globus/gridSAM
Client
SOAP
SOAP
flocking
Site A
Site B
Site C
5
Condor SOAP Interfaces
  • Now available in Condor 6.7.5
  • Obtain resource ClassAds
  • Obtain all daemon ClassAds
  • Begin/commit/abort transaction
  • Submit Job
  • Obtain Job ClassAds
  • Get/Send file
  • Removed/hold/release job

Execution machine
Execution machine
Submission machine(s)
Central manager
Schedd
Collector
Negotiator
Execution machine
Execution machine
6
Clients for BirdBath
Custom Client holdJob()
Submission machine(s)
SOAP library
Schedd
  • WSDLs available that describe each daemon
    interface
  • SOAP libraries (stub code) can be generated
    automatically from WSDLs

7
Example Query Collector
  • Obtain information (ClassAds) about all resources
    that have over 512MB RAM
  • collectorLocator new CondorCollectorLocator()
  • collector locator.getcondorCollector(new
    URL(http//kotturoti.cs.ucl.ac.uk9618)
  • classAds collector.queryStartdAds(Memorygt512)

8
Example Job Submission
  • Create New Transaction
  • Create New JobId
  • Send Files (input files, binaries)
  • Create Job Description
  • Submit Job
  • Commit Transaction
  • Transaction based process
  • Uses two phase commit to promote reliability and
    robustness
  • Files transferred in chunks Base64 encoded SOAP
    attachments
  • DAGMan jobs supported

9
OMII Collaboration gridSAM plugin
  • Job Submission Monitoring Web Service that
    submits to range of resource managers (e.g.
    Condor, SGE, Globus)
  • Uses JSDL (Job Submission Description Language)
    for standardised job description

10
gridSAM plugin
  • Existing Condor plugin uses shell-based
    submission via command-line tools and monitoring
    of log files

Client
JSDL
SOAP
Middle-tier
GridSAM
Condor
SGE
Fork

11
Future Work
  • Bundle plug-in into gridSAM
  • Also provide separate JSDL to ClassAd conversion
    library
  • Extensions to other daemons (what functionality
    to expose?)
  • For example, allow checkpoint data to be
    retrieved through the shadow in order to migrate
    jobs across pools
  • Roll out daemons on e-minerals mini-grid
Write a Comment
User Comments (0)
About PowerShow.com