How is the Globus Toolkit Being Used

1
How is the Globus Toolkit Being Used?

• Mary Thomas
• Texas Advanced Computing Center (TACC)
• (mthomas_at_tacc.utexas.edu)
• Presented at GlobusWORLD
• January 13-17, 2003, San Diego, CA

The University of Texas at Austin
Texas Advanced Computing Center
2
Projects Utilizing Globus

• Grid Portal Toolkit (GridPort)
• HotPage and other application portals
• NSF Grid programs at NPACI, including TeraGrid.
• University Grids for Texas
• UT campus grid
• Texas Internet Grid for Research Education
  (TIGRE)
• High Performance Computing Across Texas IHiPCAT)
• National Grids
• NSF/NPACI
• DoD, DoE, and NASA Information Power Grid (IPG)
• RD projects
• GridPort 3.0 (OGSA)
• Heterogeneous scheduling environments
• Information Archival Services
• Performance modelling and prediction for GCEs

3
NPACI HotPage
4
Simple Campus Grid
LSF

• Models
• Virtual Organization
• NWS Click

5
(No Transcript)
6
Grid Technologies Employed

• Globus GT 2.x (NMI R1, R2 (also earlier versions)
• GT 3.0 in next version
• Security
• GSI is key enabling infrastructure
• Globus Grid Security Infrastructure (GSI), SSH
• MyProxy for remote proxies
• Job Execution
• Globus/GRAM Gatekeeper (key)
• used to run batch, interactive jobs and tasks on
  remote resources
• Scheduler Platform Computing (LSF,
  Multicluster)
• Integration with SGE, AVAKI, others (Texas grid)
• Queing systems include PBS and others

7
Grid Technologies Employed (cont.)

• Information Services
• Globus MDS 2.2, GIIS, GRIS
• NWS, data from LSF, United Devices, etc.
• Custom information provider scripts
• Web service based GIS archival system IAWS
• File Management
• GridFTP --gt key technology
• SDSC Storage Resource (SRB)
• for file collection management
• Sun SAN between TACC/Campus

8
GridPort 2.0 Multi-Application Arch.
9
GridPort Application Portals

• UT/Texas Grids
• http//grids.tacc.utexas.edu, http//tigre.hipcat.
  net
• NPACI/PACI/TeraGrid HotPages (also _at_PACI/NCSA )
• https//hotpage.npaci.edu, hotpage.teragrid.org,
  hotpage.paci.org
• LAPK Portal Pharmacokinetic Modeling (live demo
  of Pharmacokinetic Modeling Portal)
• https//gridport.npaci.edu/LAPK
• NBCR Portals
• GAMESS (General Atomic and Molecular electronic
  Structure System)https//gridport.npaci.edu/GAMES
  S
• AMBER http//gridport.npaci.edu/Amber
• Telescience (Ellisman)
• https//gridport.npaci.edu/Telescience
• Protein Data Bank CE Portal (Phil Bourne)
• https//gridport.npaci.edu/CE

10
Programming Example Job Submit

• Client
• Example of Client HTML page
• HTML Code
• Server
• Perl/CGI parser script running on server
• GridPort Toolkit function code

11
Requirements/Issues

• Grid accounts, authorization, allocations
• Accounts and access managed at organizational
  level
• Single sign-on/Single logical file space
• Some level of centralized acct/proxy management
  system
• Current system (grid-mapfile) not scaleable
• Better performance of proxy delegation
• Need IP address for nodes (non-std cluster
  install)
• Grid schedulers
• Effective utilization of Grid resources requires
  efficient Grid scheduling beyond advanced
  reservations.
• Advanced Grid Tools
• Grid-aware compilers I/O libs, Grid-MPI libs
• Grid I/O libraries, and parallel libraries that
  interoperate with existing scheduling systems.
• Information services containing archived data
• QOS, prediction, performance analysis models/data

12
FUTURE WORK

• GridPort 2.0
• GPT 2.1 expanded features Integration into
  NPACkage/NMI
• GridPort 3.0 Grid Computing Environment
• Re-architect to include web services, OGSA, GT
  3.0, JSP/JetSpeed portlet infrastructure
• Application workflow key approach
• Define basic services for GPT 3.0 beta (Spring)
• Develop IAWS and other new tools
• TACC/UT
• Develop campus Grid - based on Sun SAN, new
  network
• Data grid in FY03
• Scheduling experiments/Platform Computing (eval,
  test, interop)

13
GridPort Architecture
14
TeraGrid HotPage
15
Project Collaborators

• GridPort Team
• TACC/Univ. of Texas Mary Thomas, Jay Boisseau,
  Maytal Dahan, Eric Roberts, Jeson Martajaya,
  Shyaml Mitra, Rich Toscano,Tom Urban
• SDSC/UCSD Cathie Mills, Steve Mock, Kurt Mueller
• Univ. of Michigan - Charles Severance, Joseph
  Hardin
• Others
• UT Jim Browne, Graham Carey, Bill Barth
• (UCSD/SDSC) Reagan Moore, Arcot Rajasekar. Kim
  Baldridge, Jerry Greenberg
• Alliance J. Alameda, S. Hampton, A. Rossi, G.
  von Laszewski
• Univ. of Indiana G. Fox, D. Gannon, M. Pierce

16
REFERENCES

• The Texas Advanced Computing Center
• http//tacc.utexas.edu
• GridPort Project (and downloads)
• http//gridport.npaci.edu
• Portals
• https//teragrid.hotpage.org
• http//grids.tacc.utexas.edu
• http//tigre.hipcat.net
• Contact Mary Thomas
• mthomas_at_tacc.utexas.edu

17
Additional Material

• HotPage Job Submission code example

18
HotPage View Job Submission
19
JobSubmit Web Page
20
JobSumbit HTML Code

• ltFORM action"https//hotpage.npaci.edu/tools/cgi-
  bin/job_submit.cgi"
• methodpost enctype"application/x-www-form-
  urlencoded" name"job_submit"gt
• Arguments ltINPUT TYPE"text" NAME"args"gt
• Select Queue ltSELECT NAME"queue"gt
• ltOPTION VALUE"low"gtlow
• ltOPTION VALUE"normal"gtnormal
• ltOPTION VALUE"high"gthigh
• ltOPTION VALUE"express"gtexpress
• lt/SELECTgt
• Number of Cpus ltINPUT TYPE"text" NAME"cpus"gt
• Max Time (min) ltINPUT TYPE"text"
  NAME"max_time"gt
• ltINPUT TYPE"hidden" NAME"mach" VALUE"SSPHN"gt
• ltINPUT TYPE"hidden" NAME"exe"
  VALUE"/rmount/paci/sdsc/mthomas/mpi_pi"gt
• ltINPUT TYPE"submit" METHOD"post"
  ACTION"https//hotpage.npaci.edu/tools/cgi-bin/jo
  b_submit.cgi" gt
• lt/FORMgt

21
JobSumbit Server Perl/CGI Parser

• GRABS HTTP/CGI data and sends it to GridPort
  subroutine, waits for results
• !/usr/local/bin/perl
• use CGI qw(all)
• my query new CGI
• 1
• BEGIN
• GET THE SCRIPTS LOCATION AND THE GLOBAL
  VARS
• MY_LOCATION "tools/cgi-bin"
• CURRENT_DIR pwd
• (PORTAL_ROOT, rest) split(/MY_LOCATION/,
  CURRENT_DIR)
• GLOBAL_VARS_CONFIG PORTAL_ROOT .
  "cgi-bin/global_vars.cgi"
• require "GLOBAL_VARS_CONFIG"
• require "PORTAL_HOME_DIR/cgi-bin/hotpage_authe
  n.cgi"

22
JobSubmit Server Perl/CGI code (cont.)

• load in code to do job submission through
  globus
• require "GRIDPORT_HOME_DIR/services/globus/cgi-bi
  n/gridport_globus_job.cgi"
• subroutines to get/set user directories
  (home,work, current) and do job handling
• require "PORTAL_HOME_DIR/tools/cgi-bin/user_dirs.
  cgi"
• require "PORTAL_HOME_DIR/tools/cgi-bin/user_jobs.
  cgi"
• my args query-gtparam(args)
• my queue query-gtparam(queue)
• my cpus query-gtparam(cpus)
• my max_time query-gtparam(max_time)
• mach query-gtparam(mach)
• my exe query-gtparam(exe)
• exe exe . " args"
• run the command through Globus, trap output,
  return to caller process
• _at_output gridport_globus_job_submit(mach,cpus,6
  0,exe,max_time,queue)