Condor J2 Developer APIs to Condor A Tutorial on Condor

1
Condor J2Developer APIs to CondorA Tutorial
on Condors Web Service Interface
2
CondorJ2

• Quill/Quill Database reflects state of Condor
  pool
• Condor J2 Database is the state of Condor pool
• Overview of CondorJ2
• Use database to maintain operational data
  (workflow state, machine state, config policies,
  etc.)
• Implement workflow management, resource
  management and resource allocation in J2EE
  Application Server environment
• Modify master, startd and starter to be web
  service clients
• Provide web interface for all system services
  (workflow submission, machine reconfiguration
  etc.)

3
Motivation

• Flexibility
• Centralized Administratibility
• Attempt to leverage standard enterprise
  technology in this space
• Scalability
• As big as you want if you are willing to pay the
  big

4
Java Application Servers

• Industrial strength middleware for high
  performance scalable web applications
• Widely deployed systems
• Oracle AS 10g, IBM WebSphere, BEA WebLogic, JBoss
  (open source)
• Key features
• Database connection pooling
• Support for transactions
• Web service interfaces
• Support for clustering (for scalability)
• Pluggable security models / role based
  authorization
• Backend database independence

5
Condor Database
JDBC
Application Server
Machine Modules
Matchmaking Modules
Workflow Modules
Condor Web Services
Condor Pool Web Site
HTTP
SOAP over HTTP
Users Web Browser
master startd starter
Users Custom Tools
Web Service Clients
Execute Machines
6
(No Transcript)
7
What can do in CondorJ2 via browsers and web
services?

• Where do we stand now?
• Add and configure new machines
• Reconfigure machines on the fly
• Specify, submit, monitor and manage workflows
• Monitor global system state
• No matchmaking (yet)
• Is currently research work. When will it ship?
  Will it ever ship? Only time will tell.

8
Interfacing Applications w/ Condor

• Suppose you have an application which needs a lot
  of compute cycles
• You want this application to utilize a pool of
  machines
• How can this be done?

9
Some Condor APIs

• MW (previous talk)
• Command Line tools
• condor_submit, condor_q, etc
• DRMAA
• Condor GAHP
• Condor Perl Module
• SOAP

10
Command Line Tools

• Dont underestimate them!
• Your program can create a submit file on disk and
  simply invoke condor_submit
• system(echo universeVANILLA gt
  /tmp/condor.sub)
• system(echo executablemyprog gtgt
  /tmp/condor.sub)
• . . .
• system(echo queue gtgt /tmp/condor.sub)
• system(condor_submit /tmp/condor.sub)

11
Command Line Tools

• Your program can create a submit file and give it
  to condor_submit through stdin
• PERL fopen(SUBMIT, condor_submit)
• print SUBMIT universeVANILLA\n
• . . .
• C/C int s popen(condor_submit, r)
• write(s, universeVANILLA\n, 17/len/)
• . . .

12
Command Line Tools

• Using the Attribute with condor_submit
• universe VANILLA
• executable /bin/hostname
• output job.out
• log job.log
• webuser zmiller
• queue

13
Command Line Tools

• Use -constraint and format with condor_q
• condor_q -constraint webuserzmiller
• -- Submitter bio.cs.wisc.edu
  lt128.105.147.9637866gt bio.cs.wisc.edu
• ID OWNER SUBMITTED RUN_TIME
  ST PRI SIZE CMD
• 213503.0 zmiller 10/11 0600
  0000000 I 0 0.0 hostname
• condor_q -constraint 'webuser"zmiller"'
  -format "i\t" ClusterId -format "s\n" Cmd
• 213503 /bin/hostname

14
Command Line Tools

• condor_wait will watch a job log file and wait
  for a certain (or all) jobs to complete
• system(condor_wait job.log)
• can specify a timeout

15
Command Line Tools

• condor_q and condor_status xml option
• So it is relatively simple to build on top of
  Condors command line tools alone, and can be
  accessed from many different languages (C, PERL,
  python, PHP, etc).
• However

16
DRMAA

• DRMAA is a GGF standardized job-submission API
• Has C (and now Java) bindings
• Is not Condor-specific -- your app could submit
  to any job scheduler with minimal changes
  (probably just linking in a different library)
• SourceForge Project
• http//sourceforge.net/projects/condor-ext

17
DRMAA

• Easy to use, but
• Unfortunately, the DRMAA API does not support
  some very important features, such as
• Two-phase commit
• Fault tolerance
• Transactions

18
Condor GAHP

• The Condor GAHP is a relatively low-level
  protocol based on simple ASCII messages through
  stdin and stdout
• Supports a rich feature set including two-phase
  commits, transactions, and optional asynchronous
  notification of events
• Is available in Condor 6.7.X

19
GAHP, cont

• Example
• R GahpVersion 1.0.0 Nov 26 2001 NCSA\ CoG\
  Gahpd
• S GRAM_PING 100 vulture.cs.wisc.edu/fork
• R E
• S RESULTS
• R E
• S COMMANDS
• R S COMMANDS GRAM_JOB_CANCEL GRAM_JOB_REQUEST
  GRAM_JOB_SIGNAL GRAM_JOB_STATUS GRAM_PING
  INITIALIZE_FROM_FILE QUIT RESULTS VERSION
• S VERSION
• R S GahpVersion 1.0.0 Nov 26 2001 NCSA\ CoG\
  Gahpd
• S INITIALIZE_FROM_FILE /tmp/grid_proxy_554523.t
  xt
• R S
• S GRAM_PING 100 vulture.cs.wisc.edu/fork
• R S
• S RESULTS
• R S 0
• S RESULTS
• R S 1

20
Condor Perl Module

• Perl module to parse the job log file
• Recommended instead of polling w/ condor_q
• Call-back event model
• (Note job log can be written in XML)

21
SOAP

• Simple Object Access Protocol
• Mechanism for doing RPC using XML
• (typically over HTTP or HTTPS)
• A World Wide Web Consortium (W3C) standard
• SOAP Toolkit Transform a WSDL to a client
  library

22
Benefits of a Condor SOAP API

• Condor becomes a service
• Can be accessed with standard web service tools
• Condor accessible from platforms where its
  command-line tools are not supported
• Talk to Condor with your favorite language and
  SOAP toolkit

23
Condor SOAP API functionality

• Submit jobs
• Retrieve job output
• Remove/hold/release jobs
• Query machine status
• Query job status

24
Getting machine status via SOAP
Your program
condor_collector
queryStartdAds()
Machine List
SOAP library
25
Lets get some details
26
The API

• Core API, described with WSDL, is designed to be
  as flexible as possible
• File transfer is done in chunks
• Transactions are explicit
• Wrapper libraries aim to make common tasks as
  simple as possible
• Currently in Java and C
• Expose an object-oriented interface

27
Condor setup

• Start with a working condor_config
• The SOAP interface is off by default
• Turn it on by adding ENABLE_SOAPTRUE
• Access to the SOAP interface is denied by default
• Set ALLOW_SOAP and DENY_SOAP, they work like
  ALLOW_READ/WRITE/
• See section 3.7.4 of the v6.7 manual for a
  description
• Example ALLOW_SOAP/.cs.wisc.edu
• If using HTTP, must set QUEUE_ALL_USERS_TRUSTEDTR
  UE
• (not needed/wanted with HTTPS)

28
Necessary tools

• You need a SOAP toolkit
• Apache Axis (Java) - http//ws.apache.org/axis/
• Microsoft .Net - http//microsoft.com/net/
• gSOAP (C/C) - http//gsoap2.sf.net/
• ZSI (Python) - http//pywebsvcs.sf.net/
• SOAPLite (Perl) - http//soaplite.com/
• You need Condors WSDL files
• Find them in lib/webservice/ in your Condor
  release
• Put the two together to generate a client library
• java org.apache.axis.wsdl.WSDL2Java
  condorSchedd.wsdl
• Compile that client library
• javac condor/.java

All our examples are in Java using Apache Axis
29
Helpful tools

• The core API has some complex spots
• A wrapper library is available in Java and C
• Makes the API a bit easier to use (e.g. simpler
  file transfer job ad submission)
• Makes the API more OO, no need to remember and
  pass around transaction ids
• We are going to use the Java wrapper library for
  our examples
• You can download it from http//www.cs.wisc.edu/
  condor/birdbath/birdbath.jar
• Will be included in Condor release

30
Submitting a job

• The CLI way

cp.sub
universe vanilla executable /bin/cp arguments
cp.sub cp.worked should_transfer_files
yes transfer_input_files cp.sub when_to_transfer
_output on_exit queue 1
clusterid X procid Y owner
matt requirements Z
condor_submit cp.sub
31
Submitting a job

• The SOAP way
• Begin transaction
• Create cluster
• Create job
• Send files
• Describe job
• Commit transaction

Repeat to submit multiple clusters
Repeat to submit multiple jobs in a single cluster
32
Submission from Java

• Schedd schedd new Schedd(http//)
• Transaction xact schedd.createTransaction()
• xact.begin(30)
• int cluster xact.createCluster()
• int job xact.createJob(cluster)
• File files new File(cp.sub)
• xact.submit(cluster, job, owner,
  UniverseType.VANILLA, /bin/cp, cp.sub
  cp.worked, requirements, null, files)
• xact.commit()

33
Submission from Java

• Schedd schedd new Schedd(http//)
• Transaction xact schedd.createTransaction()
• xact.begin(30)
• int cluster xact.createCluster()
• int job xact.createJob(cluster)
• File files new File("cp.sub")
• xact.submit(cluster, job, owner,
  UniverseType.VANILLA, /bin/cp, cp.sub
  cp.worked, requirements, null, files)
• xact.commit()

34
Querying jobs

• The CLI way

condor_q -- Submitter localhost
lt127.0.0.11234gt localhost ID OWNER
SUBMITTED RUN_TIME ST PRI SIZE CMD
1.0 matt 10/27 1445 0024642 C
0 1.8 sleep 10000 42 jobs 1 idle, 1
running, 1 held, 1 unexpanded
35
Querying jobs

• The SOAP way from Java

String statusName , Idle, Running,
Removed, Completed, Held int cluster
1 int job 0 Schedd schedd new
Schedd(http//) ClassAd ad new
ClassAd(schedd.getJobAd(cluster, job)) int
status Integer.valueOf(ad.get(JobStatus)) Sys
tem.out.println(Job is statusNamestatus)
36
Retrieving a job

• The CLI way..
• Well, if you are submitting to a local Schedd,
  the Schedd will have all of a jobs output
  written back for you
• If you are doing remote submission you need
  condor_transfer_data, which takes a constraint
  and transfers all files in spool directories of
  matching jobs

37
Retrieving a job

• The SOAP way in Java

int cluster 1 int job 0 Schedd schedd new
Schedd(http//) Transaction xact
schedd.createTransaction() xact.begin(30) FileIn
fo files xact.listSpool(cluster, job) for
(FileInfo file files) xact.getFile(cluster,
job, file.getName(), file.getSize(), new
File(file.getName())) xact.commit()
38
Authentication for SOAP

• Authentication is done via mutual SSL
  authentication
• Both the client and server have certificates and
  identify themselves
• Possible in 6.7.20
• It is not always necessary, e.g. in some
  controlled environments (a portal) where the
  submitting component is trusted
• A necessity in an open environment -- remember
  that the submit call takes the jobs owner as a
  parameter

39
Questions?
40
Authentication setup

• Create and sign some certificates
• Use OpenSSL to create a CA
• CA.sh -newca
• Create a server cert and password-less key
• CA.sh -newreq CA.sh -sign
• mv newcert.pem server-cert.pem
• openssl rsa -in newreq.pem -out server-key.pem
• Create a client cert and key
• CA.sh -newreq CA.sh -sign mv newcert.pem
  client-cert.pem mv newreq.pem client-key.pem

41
Authentication config

• Config options
• ENABLE_SOAP_SSL is FALSE by default
• ltSUBSYSgt_SOAP_SSL_PORT
• Set this to a different port for each SUBSYS you
  want to talk to over ssl, the default is a random
  port
• Example SCHEDD_SOAP_SSL_PORT1980
• SOAP_SSL_SERVER_KEYFILE is required and has no
  default
• The file containing the servers certificate AND
  private key, i.e. keyfile after
• cat server-cert.pem server-key.pem gt keyfile

42
Authentication config

• Config options continue
• SOAP_SSL_CA_FILE is required
• The file containing public CA certificates used
  in signing client certificates, e.g.
  demoCA/cacert.pem
• All options except SOAP_SSL_PORT have an optional
  SUBSYS_ version
• For instance, turn on SSL for everyone except the
  Collector with
• ENABLE_SOAP_SSLTRUE
• COLLECTOR_ENABLE_SOAP_SSLFALSE

43
One last bit of config

• The certificates we generated have a principal
  name, which is not standard across many
  authentication mechanisms
• Condor maps authenticated names (here, principal
  names) to canonical names that are authentication
  method independent
• This is done through mapfiles, given by
  SEC_CANONICAL_MAPFILE and SEC_USER_MAPFILE
• Canonical map
• SSL .emailAddress(.) \1
• SSL is the authentication method,
  .emailAddress. is a pattern to match against
  authenticated names, and \1 is the canonical
  name, in this case the username on the email in
  the principal

44
HTTPS with Java

• Setup keys
• keytool -import -keystore truststore
  -trustcacerts -file demoCA/cacert.pem
• openssl pkcs12 -export -inkey client-key.pem -in
  client-cert.pem -out keystore
• All the previous code stays the same, just set
  some properties
• javax.net.ssl.trustStore, javax.net.ssl.keyStore,
  javax.net.ssl.keyStoreType, javax.net.ssl.keyStore
  Password
• Example java -Djavax.net.ssl.trustStoretruststor
  e -Djavax.net.ssl.keyStorekeystore
  -Djavax.net.ssl.keyStoreTypePKCS12
  -Djavax.net.ssl.keyStorePasswordpass