Development of the Perl CoG

1
Development of the Perl CoG

• Stephen Mock
• Computational Science Portals Group,
• San Diego Supercomputer Center
• Presented at
• EuroGlobus 2001
• June 20Marina di Ugento
• Lecce, Italy

2
Outline

• What is a CoG?
• Why PERL
• Evolution of the PERL CoG
• Modules Examples
• Future Work
• Conclusion
• Links and email
• Acknowledgements

3
CoGs

• CoG Commodity Grid Toolkit
• What is a CoG?
• Kits that provide users and developers in a
  virtual organization with seamless access to Grid
  Services using different programming languages
• Current CoG Projects
• Java (GPDK is based on the Java CoG)
• Python
• CORBA
• PERL (new!)
• Perhaps the GRB will release a C based CoG?

4
Why PERL

• PERL CoG driven by HotPage
• User portal that gives information about
  computational resources within NPACI
• Written in PERL since way before server-side java
  developed into a robust way to program web
  applications
• PERL Globus was the most simple solution to add
  interactive access to the portal
• Job submission and control
• File and directory manipulation/transfer
• This led to the creation of the GridPort toolkit
• Abstraction of the scripts used to add
  interactive access
• Used to develop other portals which use grid
  services
• Further abstraction of those scripts is leading
  to the PERL CoG modules

5
HotPage Snapshot
6
PERL Programming Language

• PERL is a useful and versatile programming
  language
• Adaptable for different uses and highly portable
• Ubiquitous
• Powerful for text manipulation
• Parsing output
• Generating html
• Easy to use in a webserver environment
• No extension to the webserver needed to run
  CGI/PERL
• No need to recompile or reboot the server if a
  script changes
• Familiar to many scientists
• Object oriented or not, depending on your needs

7
GridPort Toolkit

• Grid Portal Toolkit
• Toolkit for developing grid aware portals in Perl
• Written in PERL as libraries
• Several portals running using GridPort software
• https//hotpage.npaci.edu and https//hotpage.paci
  .org
• https//gridport.npaci.edu/LAPK
• https//gridport.npaci.edu/Telescience
• https//gridport.npaci.edu/GAMESS
• Allows authentication, batch job submission and
  control, file transfer, and interactive
  job/command submission

8
GridPort Architecture
9
PERL CoG vs GridPort

• GridPort
• GridPort is written as PERL libraries
• GridPort is written specifically for development
  of web applications
• However, GridPort is being rewritten with the
  PERL CoG
• PERL CoG
• The PERL CoG is written as PERL modules
• The PERL CoG can be used by any PERL application
  (command-line scripts, system utilities, user
  applications)
• The PERL CoG provides better abstraction of the
  grid services

10
Perl CoG - Globus

• Some of the modules are wrappers to Globus
  functions
• Use installed Globus utilites
• Grid-proxy-init
• Globusrun
• Globus-job-submit
• Grid-info-search
• Plans to rewrite some of the modules in Pure Perl
• No need to install globus utilities
• Direct connections to the services like the
  globus gatekeeper and MDS

11
CoG Use
12
The Modules

• Tentative first pass at the architecture
• Already have changes that will be applied
• Completed Modules
• GridGlobusRSL.pm
• GridGlobusGlobusrun.pm
• GridGlobusGlobusJob.pm
• GridInfoSearch.pm
• GridMyproxy.pm
• GridProxy.pm

13
GridGlobusRSL.pm

• Module for generating Globus RSL strings
• API
• toString()
• returns formatted rsl string with (s and s
• setRSL()
• sets the rsl string
• getRSL()
• prints non-formatted rsl string
• clearRSL()
• deletes the current rsl string
• addAttribute()
• adds an rsl attribute to the rsl string
• deleteAttribute()
• removes an attribute from the rsl
• getAttribute()
• returns the string associated with an attribute

14
GridGlobusRSL.pm Example

• use GridGlobusRSL
• my rsl GridGlobusRSL-gtnew()
• if(defined(rsl))
• rsl-gtaddAttribute(executable,/bin/ls)
• generated_rsl rsl-gttoString()

15
GridGlobusGlobusrun.pm

• Module wrapper to the globusrun command
• Runs an interactive job and returns the output
• first module written
• have many changes I would like to make to it.
• API
• globusrun()

16
GridGlobusGlobusrun.pm Example

• use GridGlobusGlobusrun
• use GridGlobusRSL
• my myRSL GridGlobusRSL-gtnew()
• myRSL-gtaddAttribute('count','1')
• myRSL-gtaddAttribute('executable','/bin/ls')
• myRSL-gtaddAttribute('arguments','-l')
• my _at_output globusrun(
• globusrun_arguments",
• "tfglobus.sdsc.edu",
• myRSL-gttoString(),
• timeout"
• )

17
GridGlobusGlobusJob.pm

• Module for submitting batch jobs via Globus
• Need a name change
• API
• new()
• getStatus()
• getError()
• getJobid()
• setContact()
• setExecutable()
• setCpus()
• setQueue()
• setMaxtime()
• submit()
• cancel()
• getStderr/Stdout()

18
GridGlobusGlobusJob.pm Example

• use GridGlobusGlobusJob.pm
• my job GridGlobusGlobusJob-gtnew()
• job-gtsetContact(host)
• job-gtsetExecutable(exe)
• job-gtsetMaxtime(maxTime)
• job-gtsetCpus(cpus)
• job-gtsetQueue(queue)
• job-gtsubmit()
• print Job status is . job-gtgetStatus()
• my _at_stdout job-gtgetStdout()
• if(impatient)
• job-gtcancel()

19
GridInfoSearch.pm

• Module for querying the MDS
• Currently uses grid-info-search utility, but will
  move to a pure perl implementation soon
• API
• new()
• execute()
• getResults()

20
GridInfoSearch.pm Example

• use GridInfoSearch
• my gis GridInfoSearch-gt
• new(
• brach_point,
• filter,
• timeout
• )
• if(defined(gis))
• foreach(gis-gtgetResults())
• print
• else die Could not perform search !

21
Myproxy.pm

• Module for using MyProxy
• API
• myproxyInit()
• myproxyDestroy()
• myproxyGetDelegation()

22
GridMyproxy.pm Example

• use GridMyproxy
• myp GridMyproxy-gtnew()
• myp-gtsetMyproxyInit(
• "username","pem_passphrase",
• "myproxy_passphrase","server_name
• )
• myp-gtmyproxyInit()
• myp-gtmyproxyGetDelegation()
• res myp-gtgetMyproxyResult()
• myp-gtmyproxyDestroy()

23
GridProxy.pm

• Module for dealing with user proxy files
• API
• proxyInit()
• Generate a user proxy file
• proxyDestroy()
• Delete a user proxy file
• getInfo()
• Get information about a proxy
• Lifetime
• Strength
• User information

24
GridProxy.pm Example

• use GridProxy
• my results proxyInit(pass,cert,key,out,hou
  rs,limited,bits,cert_dir)
• print "Proxy Init Results results\n"
• my destroy_results proxyDestroy(out)

25
Future Work

• Pure PERL implementations
• Eliminate the need to have globus client tools
  installed
• Grid-info-search via the NetLDAP module
• Gram module so that we dont have to use shell
  commands to run globus jobs
• Changes to the current architecture
• More modules
• Configuration module
• GASS modules
• GSI modules
• gsissh
• gsiftp
• Storage Resoure Broker (SRB) module
• Rewrite GridPort based on these modules
• Simplify the code and installation

26
Conclusion

• The PERL CoG provides users and developers of
  applications access to Grid Services using the
  PERL programming language

27
Links and Email

• Input
• What do YOU want to see?
• Find me here or email me
• mock_at_sdsc.edu
• Links
• GridPort - http//gridport.npaci.edu
• HotPage https//hotpage.npaci.edu
• CoG can be found at the GridPort web site soon

28
Acknowledgements

• Module design and development
• Stephen Mock, mock_at_sdsc.edu
• Maytal Dahan, mdahan_at_sdsc.edu
• Input
• Mary Thomas, mthomas_at_sdsc.edu
• Gregor von Laszewski, gregor_at_mcs.anl.gov
• Kurt Mueller, mueller_at_sdsc.edu
• Catherine Mills, cmills_at_sdsc.edu