NPACI All Hands Meeting

1
GridPort API and Installation

• Maytal Dahan
• Texas advanced Computing Center

2
Tutorial Outline

• Part I Catherine Mills (SDSC)
• Portal and Grid Tools Overview
• Part II Kurt Mueller (SDSC)
• Grid Tools Installation Procedures
• Part III Maytal Dahan (TACC)
• Gridport Installation and API

3
Section goals

• Explaining GridPort functionality
• Step-by-step GridPort install instructions
• Set up gridport accounts
• Installing and using the demo portal

4
Outline

• What is gridport?
• GridPort components
• install instructions
• account management
• demo portal

5
What Is GridPort?

• collection of perl modules
• Create science portals on computational grids
• Access to remote resources
• web interface to applications
• simple perl and html programming

6
GridPort Toolkit

• Latest Release is GridPort 2.2.1
• https//GridPort.npaci.edu/downloads/
• What is new ?
• New module organization
• Improved interactive install and testing scripts
• Extensive Perl Pod documentation
• Please give us feedback and let us know how you
  are using GridPort - cmills_at_sdsc.edu

7
Module Organization

• CogUtil
• SRB
• Cog
• GridPort
• Each of these components have install and test
  scripts.
• SRB is optional and only needs to be installed if
  you are planning on using SRB.

8
What is CogUtil?

• These are external utilities that the other
  modules use. The portal can also use these
  directly.
• CogUtil Modules
• UnixRun - perl module that runs system calls with
  timeout capability.
• Log - Log file capability. The portal can use
  this to create log files. Specify location of log
  file then can write to the log file

9
What is SRB?

• SRB module
• standalone SRB module that wraps SRB executables.
• Run SRB from a perl API using GSI authentication
• Dependencies CogUtil, SRB 2.0 Executables
• SRB web pagehttp//www.npaci.edu/dice/SRB/
• interactive install and test script. It sets up
  the configuration for SRB on your system

10
What is SRB?

• Requirements
• SRB client executables
• An SRB account with GSI capabilities
• SRB server
• Set environment variables about your account
• Create a proxy
• How to use perl SRB?
• Make sure you have an account with GSI enabled
• Create a proxy (grid-proxy-init)
• Set SRB environment variables
• Use the SRB functions

11
What is SRB?

• SRB system environment variables
• srbHost
• srbPort
• mdasDomainHome
• mdasDomainName
• SERVER_DN
• defaultResource
• SRB account variables
• X509_USER_PROXY - the location of your proxy.
• srbUser - your SRB username
• GLOBUS_INSTALL_PATH - the location of globus,
  this is also specific to SRB install
• GLOBUS_LOCATION from where you installed globus
• Auth_SCHEME - Always GSI_AUTH
• Append PATH env to include SRB bin directory and
  globus bin directory
• Source /globus_user_env.csh

12
What is the CoG?

• CoG Commodity Grid Toolkit
• The cog is perl modules that are wrappers to
  globus functionality.
• The PERL CoG can be used by any PERL application
  (command-line scripts, system utilities, user
  applications)
• The PERL CoG provides better abstraction of the
  grid services

13
What is the CoG?

• Dependencies CogUtil, Globus executables, Perl
  Expect module, MyProxy executables (optional)
• Components
• GlobusJob
• GlobusRun
• GlobusRSL
• MDSSearch
• SecurityMyProxy
• SecurityProxy

14
What is the CoG?

• GlobusJob
• globus gram job submission
• Subroutines submit, get status, get stdout, get
  status, get jobid, cancel and other access
  functions
• GlobusRun
• run a command on a remote system
• GlobusRSL
• create and manipulate an rsl string.
• build an RSL and then use it to run a globus
  command.

15
What is the CoG?

• MDSSearch
• Perl extension for GIIS/GRIS searches
• SecurityMyProxy
• perl extension of MyProxy functionality
• Subroutines myproxy-init, myproxy-get-delegation,
  destroy, change passphrase, get info
• SecurityProxy
• perl interaction with proxy
• Subroutines create, destroy, get info

16
What is the GridPort ?

• portal api to globus and SRB
• handles authentication and session tracking
• A web portal uses gridport directly not the other
  modules
• GridPort modules
• Authentication, Proxy forward, File transfer,
  job, SRB
• Dependencies CogUtil, Cog, SRB

17
What is the GridPort ?

• Authentication -
• handles portal authentication, GSI login, myproxy
  login, logout, session information and proxy
  management
• Subroutines GSI login, myproxy login, logout
• What happens when you login?
• Create a proxy - either with create a proxy or
  get a delegation from myproxy
• Create a session file that has your username,
  timestamp, portal
• Set a cookie that is the name of the session
• Proxy forward
• Handles forwarding a proxy to a remote system.
• Used if you need to authenticate on the remote
  machine with a proxy

18
What is the GridPort ?

• File transfer -
• handles transferring files to and from the portal
  to a remote system and third party transfer
• Subroutines gass put and get, grid ftp put get
  and third party.
• Job
• Job submission and remote command execution
• Subroutines run, job submit, cancel
• SRB
• SRB functionality through the portal
• Subroutines login, run SRB command

19
Installing GridPort

• Make sure you have all the necessary components
  installed in preparation for GridPort (previous
  section by Kurt Mueller)
• Download the tar ball from the gridport
  downloads web page
• Read the README file for detailed install
  instructions

20
Before you install

• Perl modules to install
• Bundleexpect module
• netldap module
• Globus client binaries
• myproxy client binaries
• SRB client binaries and SRB account

21
Before you install

• Need to make sure you have gp_portals group and
  that you have permissions to set directories to
  be have group permissions
• The gridport directories must be group read/write
  by gp_portals group
• Make sure you have a certificate and key that you
  can use to authenticate with and that you are in
  the mapfile for at least one machine.

22
Installing GridPort cont.

• Install CogUtil
• cd gridport/CogUtil
• perl Makefile.pl
• make
• make test
• make install
• After you have installed you can also run perl
  test.pl to test the module that is in your
  installation

23
Installing GridPort cont.

• Install Cog
• cd ../Cog
• perl Makefile.pl
• make
• grid-proxy-init
• make test TEST_VERBOSE1
• make install

24
Installing Gridport cont.

• Installing SRB
• This will ask you where your SRB and globus
  binaries are located
• cd ../SRB
• perl Makefile.PL
• Make
• make test
• make install

25
Installing GridPort cont.

• cd ../GridPort/
• perl makefile.pl
• The make command is interactive. It will ask you
  questions about your install environment.
• GridPort repository
• Configuration information
• Session timeout in seconds (3600)

26
Installing GridPort cont

• Before you move on you must do these
• Gridport repository should not be in web space
• Make sure the gridport repository has the correct
  permissions and ownership
• set to owner read and write access
• Group read and write by GP_Portals
• add a directory under /repository/pr
  oxies/site_proxies that is the name of the web
  user for example gp_user
• Gp_user has read/write permissions to this
  directory.

27
Installing GridPort cont

• Set up a gridport account
• Copy your x509 certificate and key to
  /certificates/stored_certs/ and
  name them testuser_cert.pem and testuser_key.pem
• If you plan on using myproxy run myproxy-init so
  that you can test proxy delegation.

28
Installing GridPort cont.

• Now you can continue installing GridPort
• make
• make test
• make install

29
Installing GridPort SRB

• To enable gridport to use SRB it must know about
  the users SRB account.
• SRB account information goes in
  /user_account_info
• Each portal user that has an SRB account will
  have a file named .SRB
• This is where gridport looks for SRB information
  to authenticate the user

30
What is in your SRB file?

• Contents of your SRB file
• srbUser
• srbHost
• srbPort
• mdasDomainHome
• mdasDomainName
• AUTH_SCHEME
• SERVER_DN
• defaultResource

31
Account Management

• Gridport Repository
• This area handles users proxies, sessions,
  certificates and keys
• Gridport needs a user proxy to run interactive
  capabilities as a user.
• A GridPort account means that GridPort has some
  way of getting a proxy for the user
• User distinguished name (DN) has to be in the
  mapfile in order to access a remote resource.

32
Gridport/portal account

• If going to use myproxy then you dont have to do
  anything to set a portal account accept for
  making sure users have set up myproxy and dn is
  in mapfile
• Portal might want to have a registration page and
  ACL to monitor its users.
• The repository is located in the location that
  you specified when you run perl Makefile.pl

33
Portal account continue

• Another way to manage accounts besides myproxy is
  to have users give you a copy of their
  certificate and key.
• No passwords are stored just the certificate and
  key.
• The user would request an account and would need
  to give the portal developer their username,
  certificate and key.

34
Portal account continue

• What the developer does
• Rename the certificate and key _cert.pem
  and _key.pem respectively
• Put the certificate in /repository/c
  ertificates/stored_certs
• Set the permissions on the certificate to owner
  and group read.
• The distinguished name has to be in the mapfile
  for the remote machines the user has accounts on.

35
Account management cont

• Users can now log in 2 ways
• Get a delegation from MyProxy - this assumes that
  the user has already set themselves up with
  MyProxy
• Generate a proxy from the users certificate and
  key in the repository. This assumes that the user
  has given their cert and key to the portal admin.

36
Demo Portal

• Perl cgi and html scripts
• located in the examples directory under gridport
• allows you to test gridport functionality
• Can use this as a template for your own portal

37
Demo Portal

• certificate and key are in the repository with
  correct directory and file permissions
• MyProxy already set up
• SRB account information file in repository
• Copy the demo portal code to web space with
  correct permissions
• Make sure you can make an https connection
• Run the install script - perl install
• Modify Login.cgi and MyProxy.cgi in Login/cgi-bin
• modify the string YOURWEBADDRESS to be your web
  address of the demoportal.
• Example if your demo portal is
  https//test.domain.com/testingme/demoportal you
  YOURWEBADDRESS test.domain.com/testme.

38
demo portal functionality

• GSI Login
• MyProxy login
• Logout
• Check login state
• Run job
• File transfer - put, get, GridFTP put, get and
  third party transfer
• SRB functionality

39
demo portal
40
Future gridport work

• Gridport 2.X will evolve with NPACKage releases.
• System logging capabilities
• Updates to myproxy
• Add further SRB functionality
• More
• Check the gridport web site for updates.

41
References

• Gridport - https//gridport.npaci.edu/
• Cog - https//gridport.npaci.edu/cog/
• MyProxy http//www.ncsa.uiuc.edu/Divisions/ACES/My
  Proxy/
• Globus - http//www.globus.org/
• SRB - http//www.npaci.edu/DICE/SRB/

42
Acknowledgements

• Thank everyone on the gridport team
• Mary thomas, lead PI
• Catherine Mills, Stephen Mock, Maytal Dahan, Kurt
  Mueller, Tomislav Urban, Eric Roberts
• Thank other groups SRB, GAMESS, Cosmic