Installing and Building GTLAB

1
Installing and Building GTLAB

• Marlon Pierce
• Indiana Univeristy

2
GTLAB and OGCE

• OGCE contains multiple sub-projects
• Portlet-based Grid portal (with GridSphere and
  Tomcat).
• Workflow suite (services and add-ins to the
  portal)
• Information Web services
• Gadget container
• Cyberaide JavaScript libraries
• GTLAB
• These are packaged with Maven and include
  everything you need except Java and (for some
  services) MySQL.
• We try to make things installable with minimal
  fuss.
• Consistent directory structures across projects
• Edit one config file (pom.xml)
• Run one command (mvn clean install)
• You may need to futz a little with MySQL

3
Overview and Examples
4
Users Browser
Portlet, Gadget Containers
iGoogle, Facebook
GTLAB Gadgets
TeraGrid Services (GRAM, GridFTP, INCA)
Google GData Services, Twitter Feeds, etc
RPS, RDS, GPIR Services
GTLABs Relationship to Other Components
5
Various GTLAB applications deployed as portlets
Remote directory browsing, proxy management, and
LoadLeveler queues.
6
GTLAB Applications as Google Gadgets MOAB
dashboard, remote directory browser, and proxy
management.
7
MyProxy Example
lthform id"form"gt lthoutputText
value"Username"/gt lthinputText
value"resource.username" required"true"/gt
lthoutputText value"Password "/gt
lthinputSecret value"resource.password"
required"true"/gt lthoutputText
value"MyProxy Server "/gt lthinputText
value"resource.myproxyHostname"
required"true"/gt ltosubmit id"submit"
value"Submit" binding"builder.body"
action"builder.junkAction" gt
ltomultitask id"multi" persistent"true"
taskname"resource.taskname"gt
ltomyproxy id"mypr" hostname"resource.myproxyH
ostname" lifetime"2" password"resource.pa
ssword" port"7512" username"resource.user
name"/gt lt/omultitaskgt lt/osubmitgt
lt/hformgt
8
Executing Multiple Task
ltosubmit id"submit" value"Submit"
binding"builder.body" action"builder.j
unkAction" gt ltomultitask id"multi"
persistent"true" taskname"resource.tasknam
e"gt ltojobsubmit id"js" myfaces"true"
arguments"resource.arguments"
executable"resource.executable"
hostname"resource.hostname"
provider"resource.provider"
stdout"resource.stdout"/gt
ltofiletransfer id"ft" myfaces"true"
from"resource.from" to"resource.to"
/gt ltodependency id"dep" task"ft"
dependsOn"js"/gt lt/omultitaskgt lt/osubmitgt
9
Resource Prediction Service Client with GTLAB
ltosubmit id"submit" value"Submit"
binding"builder.body"
action"builder.junkAction"gt
ltomultitask id"multi" persistent"true"
taskname"resources.tasknamegt
ltorpsOptimal resource"task1-resources"
id"rpsoptimal-exec"
myfaces"true"/gt
lt/omultitaskgt lt/osubmitgt
10
Example with Dependency
ltosubmit id"submit gt ltomultitask
id"multi gt ltorpsAddModel
resource"task1-resources"
id"rpsAddModel-exec"
myfaces"true"/gt ltorpsManager
resource"task1-resources"
id"rpsManager-exec"
myfaces"true"/gt ltodependency
id"deps-rpsstuff task"rpsManager-exec"
dependsOn"rpsAddModel-exec"/gt
lt/omultitaskgt lt/osubmitgt
11
More Detail Available Tags and Features
12
GTLAB Features

• Extends Java Server Faces.
• Tag components wrap major COG Abstraction Layer
  features
• And Web Service clients, GridShib, etc.
• Supports tag inter-dependencies.
• Allows you to do standalone development.
• Use JSF portlet bridge to convert into portlets
• No new coding, just add/modify XML config files
  and jars.
• Process is automated
• Or forget about portlets
• Convert into Google Gadgets
• Develop as a Facebook application
• Etc.

13
Getting GTLAB

• See http//www.collab-ogce.org/ogce/index.php/GTLA
  B
• You can use your favorite SVN client to check
  out.
• svn co https//ogce.svn.sourceforge.net/svnroot/o
  gce/GTLAB (latest)
• svn co https//ogce.svn.sourceforge.net/svnroot/og
  ce/tags/GTLAB-tg09 (tagged)
• Latest will give you easy access to any updates
• svn update
• Best option if you want to actively develop and
  get fixes right away.

14
No SVN? Get the TAR

• SourceForges SVN/CVS viewer now provides a
  Download GNU Tar option.
• http//ogce.svn.sourceforge.net/viewvc/ogce/GTLAB/
  for latest.
• http//ogce.svn.sourceforge.net/viewvc/ogce/tags/G
  TLAB-tg09/ for tag

15
Levels of Abstraction in GTLAB code

• Note JSF is compatible with JSP, so you can mix
  and match.
• And you can embed JavaScript
• YUI, Scriptaculous user interface goodies.
• Google and Facebook APIs.

16
(No Transcript)
17
These will work as gadgets, portlets, or
standalone applications
18
Gadgets and Portlets
19
GTLAB gadgets can also run as portlets with no
code changes by using the JSF portlet bridge.
iGoogle Client Aggregator
Tomcat Web Server Portlet Container
Tomcat Web Server
Compile and deploy into server
GTLAB Gadgets Code Base
GTLAB Portlets Code Base
Decorate with bridge and container jars, xml
config files.
20
Making GTLAB Google Gadgets

• A Gadget is a standalone application that is
  integrated into your personal iGoogle display.
• They can run on your Web server.
• You can use SSL/HTTPS
• Google Gadgets have two versions
• http these use Googles JavaScript APIs
• url these wrap external applications with
  Iframes.
• We currently support url gadgets.

21
Example Gadget Config File
lt?xml version"1.0" encoding"UTF-8" ?gt
ltModulegt ltModulePrefs title"MyProxy Gadget
Example" scrolling"true"
height"500"/gt ltContent type"url"
href"http///8080/GTLAB/examples/MyProxyExamp
le.jsf"gt lt/Contentgt lt/Modulegt
Save this as MyProxy.xml and place in a Web
accessible place.
22
Gadget containers aggregate content from multiple
providers. Content is aggregated on the client
by the user. Nearly any web application can be a
simple gadget (as Iframes)
Other Gadgets Providers
Tomcat GTLAB Gadgets
Other Gadgets Providers
RSS Feed, Cloud, etc Services
Grid and Web Services (TeraGrid, OSG, etc)
Social Network Services (Orkut, LinkedIn,etc)
23
Click Add stuff and then Add feed or gadget.
Type the URL of your gadget.
24
GTLAB Applications as Google Gadgets MOAB
dashboard, remote directory browser, and proxy
management.
25
Common science gateway architecture.
Aggregation is in the portlet container.
Users have limited selections of components.
HTML/HTTP
Tomcat Portlets and Container
SOAP/HTTP
Grid and Web Services (TeraGrid, OSG, etc)
Grid and Web Services (TeraGrid, OSG, etc)
Grid and Web Services (TeraGrid, OSG, etc)
26
Making GTLAB Gadgets into Portlets

• Do all development in jsf_standalone directory
• This is where your template tags will be placed
• When you are ready to convert
• Download and build the OGCE portal
• cd GTLAB/transition
• mvn clean process-resources
• cd GTLAB/portlets
• mvn clean install
• You will need to make small edits to portlet.xml,
  group.xml, and layout.xml in GTLAB/portlet/src/mai
  n/webapp/WEB-INF/
• Thats it.

27
Various GTLAB applications deployed as portlets
Remote directory browsing, proxy management, and
LoadLeveler queues.
28
Getting Started with GTLAB
29
GTLAB Directory Structure
30
Build GTLAB
ltpropertiesgt ltportal.server.ipgt 156.56.104.143
lt/portal.server.ipgt lthost.base.urlgt http//
portal.server.ip8080/ lt/host.base.urlgt
ltproject.homegt env.HOME/GTLAB lt/project.hom
egt lttomcat.versiongt apache-tomcat-5.5.27 lt/tom
cat.versiongt ltcatalina.homegt project.home
/portal_deploy/tomcat.version/ lt/catalina.hom
egt ltdot.globus.homegt env.HOME/.globus/ lt/do
t.globus.homegt lt/propertiesgt

• Unpack or checkout code
• Cd GTLAB
• All commands are executed from here.
• Edit properties at the top of pom.xml.
• Change IP
• Change project.home if you unpack someplace
  besides HOME.
• Run mvn clean install

31
Run Examples

• From GTLAB, start tomcat with ./startup.sh.
• From GTLAB, stop Tomcat with ./shutdown.sh
• Point browser to http//localhost8080/GTLAB
• Start with MyProxy Example

32
Next Steps

• Play with examples.
• These are really bare bones. Make something
  interesting.
• Make a Google gadget.
• Mix and match tags in a pipeline to make a new
  application.
• Use the dependency tag.
• Note you can mix and match JSF and JSP if you are
  not familiar with JSF.
• Try making a new tag.
• Explained next.

33
Making New Tags and Examples
34
Making a New JSF Page from Tags

• I recommend starting from the examples.
• jsf_standalone/src/main/webapp/examples
• Build the examples with
• mvn o clean install f jsf_standalone/pom.xml
• The -o option is used to build offline. Will
  also avoid unnecessary Maven repository updates.
• The -f specifies only build this specific
  module.
• I recommend not futzing with the deployed
  versions under portal_deploy.
• A computer is a state machine. State must be
  reproducible.

35
Making a New Tag

• Run the following command from GTLAB
• mvn clean process-resources -Dtag.nametest
  -DprojectnameTest -f templateTag/pom.xml
• Add -Ddest.dir/tmp for a dry run.
• Replace test with the name of your tag.
• Replace Test with the name for your Bean.
• This will make 4 files
• TestBean.java, TestTag.java, TestBeanFactory.java,
  UITest.java
• Edits also 3 config files
• gtlab-factory.xml, managed-beans.xml,
  components.xml
• This will compile but to implement something
  useful, you will need to edit the highlighted
  files.

36
Implementing a Tag

• The place to start is TestBean.java (or whatever
  you used for Dprojectname).
• This includes several inherited methods that can
  be implemented.
• Most important is submit(). Use the try/catch
  block. This is where the action is.
• If you want to hook tags into chains, implement
  getOutput() and setInput().
• Also take a look at the other beans.

37
Suggested Tags A Wish List

• What can you do in your bean? Anything
  server-side Java can do.
• Some suggestions
• Implement a tag client to a remote Web service.
  Amazon has some interesting ones.
• Implement an RSS/Atom feed client to Twitter,
  your blog, Facebook, etc.
• Combine the feeds as a mash-up.
• Connect to a database with JDBC.
• Implement a JMS publisher or subscriber.
• Use Google Java APIs to interact with Blogger,
  Calendar, and YouTube.
• Try interacting with Facebook.

38
INCA Tags An Extended Example
39
Example Make an INCA Dashboard

• INCA is used as the testing framework for the
  TeraGrid.
• You can get the latest INCA test results from the
  URL
• http//inca.teragrid.org/inca/XML/kit-status-v1/po
  rtal_summary
• See Sangmis notes at http//sangpall.blogspot.com
  /2009/03/being-nice-client-of-inca-service.html
• Youll get back some fine looking XML.
• Lets make a little dashboard out of this.

40
First, Create a New, Empty Tag

• mvn clean process-resources \
• -Dtag.nameincastatus \
• -DprojectnameIncaStatus \
• -f templateTag/pom.xml

41
Second, Implement the Bean

• Main thing is to implement IncaStatusBeans
  submit() method.
• Code is hard to put in a PPT slide
• See http//ogce.svn.sourceforge.net/viewvc/ogce/GT
  LAB/jsf_standalone/src/main/java/ogce/gsf/gridbean
  s/IncaStatusBean.java

42
Third, Implement a JSF Page

• Actual tag is shown below.
• Full example is http//ogce.svn.sourceforge.net/vi
  ewvc/ogce/GTLAB/jsf_standalone/src/main/webapp/exa
  mples/IncaStatusExample.jsp?viewlog
• It includes some boilerplate (can be automated).

ltoincastatus idmytest"
resource"task1-resources
myfaces"true"/gt
43
Heres the resulting page. You can make into a
portlet in a couple of steps. You can also tie
to other tags in a pipeline.
44
Next Steps Make a Pipeline

• Implement the setInput() and getOutput() methods.
• Pipeline it with the Resource Description and
  Resource Prediction Service tags.
• Integrate QBETS tags
• Currently unwritten
• Integrate with Job Submission

Find Running Hosts (INCA or RDS)
Find Best Host for YourJob (RPS)
Find Host with Shortest Wait Time (QBETS)
Submit Job
45
(No Transcript)
46
More Stuff about GTLAB

• Supports linear tag dependencies.
• Removed dependence on COG graph processing (still
  use the COG)
• We can support graphs of any tag you care to
  write.
• Only support pipelines currently, not full DAGs
• Redesigned to simplify writing new tags.
• Extensive use of factories.
• You no longer have to change core code to add a
  tag
• Templates for 4 Java files and 3 XML config files
  now can be done in one (mvn) step
• Typically you will only need to edit one Java
  file.