Open Grid Computing EnvironmentsGateways Tutorial - PowerPoint PPT Presentation

1 / 41
About This Presentation
Title:

Open Grid Computing EnvironmentsGateways Tutorial

Description:

System Testing: We use Apache JMeter to verify portal builds function correctly. ... var='proxyEntry' id='proxyTable' h:column f:facet name='header' ... – PowerPoint PPT presentation

Number of Views:36
Avg rating:3.0/5.0
Slides: 42
Provided by: Marlon74
Category:

less

Transcript and Presenter's Notes

Title: Open Grid Computing EnvironmentsGateways Tutorial


1
Open Grid Computing Environments/Gateways Tutorial
  • Marlon Pierce, Suresh Marru, Gopi Kandaswamy,
    Gregor von Laszewski, and Tom Scavo

2
JM8TBevr myproxy.ncsa.uiuc.edu
3
Tutorial Overview
  • Part 1 OGCE Portal (Marlon), 830-930 am
  • Introduction and overview
  • Portal Demo
  • How to build Grid portlets and gadgets
  • Part 2 OGCE Workflow Tools (Suresh and Gopi)
  • Part A Demo and overview, 930-1000 am
  • Part B Hands on demo, deploying new
    applications, 1030-1115 am
  • Part 3 Web 2.0 for Grid Portals (Gregor)
  • Part 4 GridShib and Gateway security (Tom)

4
Last Part of Tutorial
  • Both Gregor and Tom had flight cancellations, so
    this opens up 45 minutes of the program.
  • We can do any or all of the following
  • More hands on with portal and workflow stuff.
  • Building stuff from scratch
  • Wrapping applications with GFAC
  • Discuss Gateway issues, requirements, etc
  • I also have Web 2.0 tutorial material in hand if
    there is any interest.

5
Goals of Tutorial
  • Goals
  • Review portal and gateway technologies old and
    new.
  • Demonstrate how to execute workflows and wrap
    TeraGrid applications.
  • Discuss security
  • Hear from TG Gateway community what do you want?
  • We have a lot of material to cover so we will
    sample many things.
  • You will have enough information to get going as
    a homework assignment.
  • Please also check with us at TG08
  • Please see Nancys Gateway Parade this
    afternoon.

6
Slides and Demo Site
  • Tutorial slides are available from
    http//www.collab-ogce.org/ogce/index.php/Tutorial
    s
  • We run a permanent demo portal at
    https//community.ucs.indiana.edu8443/gridsphere/
  • Also aliased as https//ogceportal.iu.teragrid.org
    8443/gridsphere
  • Accounts train30-train49 have been created for
    the workshop. Password is the same as the
    account name.
  • We also have training accounts with same names
    that can be used to retrieve TG proxy
    credentials.
  • These should be active all week.

7
Portal and Gadget Demos
  • Demonstration of basic capabilities

8
Getting the Code
9
What You Can Download
  • OGCE Portal
  • Various portlet samplers
  • Web and Grid Services
  • XRegistry Web Service Registry
  • GFAC Application Factory Service
  • Grid Portal Information Repository
  • Programming Libraries
  • Java and JavaScript COG
  • Grid Tag Libraries and Beans (GTLAB)
  • Information Service Client APIs

10
Project Release Schedule
  • We have two major releases per year with
    intermediate updates.
  • TeraGrid (June)
  • Supercomputing (November)
  • Focus of this release cycle the workflow suite
    tools shown in Part 2.
  • We do not currently package services and portlets
    together.
  • These are separate downloads.

11
Project Build and Test
  • Source Code Version Control and Building We use
    Maven 2 and SourceForges SVN
  • Anonymous checkouts
  • We are Open Source
  • Nightly Build Testing We use the NMI Build and
    Test facility at UW
  • Mac and Linux flavors
  • System Testing We use Apache JMeter to verify
    portal builds function correctly.
  • You can download this from us
  • You can also view the nightly dashboard

12
Project Information
  • Web site http//www.collab-ogce.org
  • Announcement Blog http//collab-ogce.blogspot.com
  • RSS/Atom Feeds http//collab-ogce.blogspot.com/at
    om.xml
  • This is also embedded in the wiki as News
  • Contact us discuss_at_ogce.org

13
Building the Portal
14
Choosing a Host Computer
  • Select your host computer.
  • You will need a real IP address for some
    COG-based portlets
  • VPN also OK (?)
  • Know general Globus et al firewall issues.
  • I recommend turning firewalls and SELinux off for
    development.
  • Operating Systems
  • We test with Mac OS X and many Linux flavors
  • Other Unix should be fine (assuming Java works)
  • We have not had any requests for Windows.
  • Software Environment
  • The portal download comes with everything you
    need EXCEPT Java. You need the JDK, not the JRE.
  • We test with Java 1.5. Dont use Java 1.4.
  • The more memory, the better.

15
Downloading and Building the Portal
  • Download the portal from http//www.collab-ogce.or
    g.
  • Unpack in HOME.
  • You should also download the Maven repository.
  • This is optional but recommended
  • Edit ogce-portal-home/pom.xml
  • Run the command mvn clean install.
  • Use mvn o clean install to build offline.
  • Build will typically take 5 minutes (new Mac) to
    30 minutes (old Linux PC).

You can also check out the latest code from SVN.
Check the Nightly Build dashboard to verify.
16
Practical Considerations
  • Increase your JVM Memory
  • export JAVA_OPTS"-server -Xms512m -Xmx1024m
    -XXMaxPermSize256m
  • Xmx2048m, 4096m, etc if you can.
  • Monitor catalina.out for problems
  • tail f catalina.out
  • Make sure that Tomcat shuts down
  • Kill if necessary
  • NFS can really slow down building
  • Use /usr/local/ if you have problems

17
Editing pom.xml
ltpropertiesgt ltportal.server.ipgt
156.56.104.143 lt/portal.server.ipgt
lthost.base.urlgt http//portal.server.ip80
80/ lt/host.base.urlgt ltgridftp.host.namesgt
. lt/gridftp.host.namesgt
ltgram.host.namesgt . lt/gram.host.namesgt
ltproject.homegt env.HOME/ogce-portal-only
lt/project.homegt lt/propertiesgt
  • You only need to edit the top level
    ogce-portal-only/pom.xml.
  • You only need to change ltpropertiesgt for local
    settings.
  • You MUST update the IP address.
  • You may want to add or remove GRAM and GridFTP
    hosts.
  • Update project.home if you are building someplace
    besides HOME.

18
Rebuilding the Portal
  • Running mvn clean install will completely wipe
    out the existing installation.
  • You may not want to do this.
  • If you want to rebuild individual components,
    just run the following command from the
    ogce-portal-only directory.
  • mvn clean install f portlets/myportlet/pom.xml
  • You can do this for all ltmodulesgt in
    ogce-portal-deploy/pom.xml

19
OGCE Portal Project Layout
  • We follow standard Maven 2 conventions for
    project layouts for each component.
  • moduleName/src/main/webapp
  • moduleName/src/main/java
  • Modules are listed on next slide
  • See ogce-portal-only/pom.xml.
  • Each module is a separate directory and has its
    own pom.xml.

20
(No Transcript)
21
Developing Grid User Interface Components
  • Portlets, gadgets, etc.

22
Under the Hood Building Components
  • OGCE portlets use the Java COG Abstraction Layer
    to access Grid services.
  • Provides a buffer over different Grid toolkits
    and versions.
  • Provides ability to compose graph-based
    workflows.
  • Tutorial in itself
  • Portlets can be built with our Apache Velocity
    bridge.
  • Template or steal this code approach.
  • We have also developed Grid Tag Libraries and
    Beans (GTLAB) to simplify this process.

23
GTLAB Features
  • Extends Java Server Faces.
  • Tag components wrap major COG Abstraction Layer
    features
  • Allows you to do standalone development.
  • Use JSF portlet bridge to convert into portlets
  • No new coding, just change, add config files and
    jars.
  • Or forget about portlets and
  • Convert into Google Gadgets
  • Develop as a Facebook application
  • Etc.
  • Status GTLAB is under active development, can be
    used to build simple things, but still has many
    loose ends.
  • You need to be technically adept to work with it
    now.
  • But you are welcome to give feedback or
    contribute.

24
Installing GTLAB
  • You need to install the OGCE portal
  • GTLAB depends on many shared jars.
  • Download the release code from www.collab-ogce.org
    or check out from SVN
  • svn checkout https//ogce.svn.sourceforge.net/svnr
    oot/ogce/GTLAB
  • Edit GTLAB/jsf_standalone/pom.xml
  • ltcatalina.homegt should point to your OGCE portal
    server.
  • Edit GTLAB/jsf_standalone/src/main/resources/resou
    rces.properties.
  • Run mvn clean install in GTLAB/jsf_standalone/

25
Levels of Abstraction in GTLAB code
  • Note JSF is compatible with JSP, so you can mix
    and match.
  • And you can embed JavaScript
  • YUI, Scriptaculous user interface goodies.
  • Google and Facebook APIs.

26
MyProxy Example
lthform id"form"gt lthoutputText
value"Username"/gt lthinputText
value"resource.username" required"true"/gt
lthoutputText value"Password "/gt
lthinputSecret value"resource.password"
required"true"/gt lthoutputText
value"MyProxy Server "/gt lthinputText
value"resource.myproxyHostname"
required"true"/gt ltosubmit id"submit"
value"Submit" binding"builder.body"
action"builder.junkAction" gt
ltomultitask id"multi" persistent"true"
taskname"resource.taskname"gt
ltomyproxy id"mypr" hostname"resource.myproxyH
ostname" lifetime"2" password"resource.pa
ssword" port"7512" username"resource.user
name"/gt lt/omultitaskgt lt/osubmitgt
lt/hformgt
27
Show the Proxy Credential
lthdataTable value"myproxy.storedProxyList"
border"1" binding"myproxy.stored
ProxyTable" var"proxyEntry"
id"proxyTable"gt lthcolumngt
ltffacet name"header"gt lthoutputFormat
id"subject" escape"false value"Subject"/gt
lt/ffacetgt lthoutputText
id"proxySubject" value"proxyEntry.subject"/gt
lt/hcolumngt lthcolumngt
ltffacet name"header"gt lthoutputFormat
id"Remove" escape"false value"Delete"/gt
lt/ffacetgt lthcommandButton
id"deleteProxy" value"Remove
actionListener"myproxy.deleteCredential"/
gt lt/hcolumngt lt/hdataTablegt
28
Executing Multiple Task
ltosubmit id"submit" value"Submit"
binding"builder.body" action"builder.j
unkAction" gt ltomultitask id"multi"
persistent"true" taskname"resource.tasknam
e"gt ltojobsubmit id"js" myfaces"true"
arguments"resource.arguments"
executable"resource.executable"
hostname"resource.hostname"
provider"resource.provider"
stdout"resource.stdout"/gt
ltofiletransfer id"ft" myfaces"true"
from"resource.from" to"resource.to"
/gt ltodependency id"dep" task"ft"
dependsOn"js"/gt lt/omultitaskgt lt/osubmitgt
29
These will work as gadgets, portlets, or
standalone applications
30
Making GTLAB Gadgets into Portlets
  • Do all development in jsf_standalone.
  • When you are ready to convert
  • cd GTLAB/transition
  • mvn clean process-resources
  • cd GTLAB/portlets
  • mvn clean install
  • You will need to manually edit portlet.xml,
    group.xml, and layout.xml in GTLAB/portlet/src/mai
    n/webapp/WEB-INF/
  • Thats it.

31
Making GTLAB Google Gadgets
  • A Gadget is a standalone application that is
    integrated into your personal iGoogle display.
  • They can run on your Web server.
  • You can use SSL/HTTPS
  • Google Gadgets have two versions
  • http these use Googles JavaScript APIs
  • url these wrap external applications with
    IFrames.
  • Limitations
  • Not much real estate (but location, location,
    location)
  • Cookies manage state on gadgets from the same
    server but we need a way to integrate multiple
    independent servers.

32
Some GTLAB Gadgets
33
Example Gadget Config File
lt?xml version"1.0" encoding"UTF-8" ?gt
ltModulegt ltModulePrefs title"MyProxy Gadget
Example" scrolling"true"
height"500"/gt ltContent type"url"
href"http///8080/GTLAB/examples/MyProxyExamp
le.jsf"gt lt/Contentgt lt/Modulegt
Save this as MyProxy.xml and place in a Web
accessible place.
34
Some Web 2.0 Resources
  • See recent tutorials by Geoffrey Fox and me
  • http//grids.ucs.indiana.edu/ptliupages/presentati
    ons/Web2.0Tutorial.ppt
  • http//grids.ucs.indiana.edu/ptliupages/presentati
    ons/CTSTechFutures_May19-08.pptx
  • Includes material on Google Gdata, Facebook API,
    Open Social, Microformats, etc.
  • See also full examples at http//communitygrids.bl
    ogspot.com

35
TeraGrid Information Services Portlets and
Gadget Libraries
  • Slides courtesy of Maytal Dahan

36
TG Information Services
  • What are TG Information Services?
  • Services that publish metadata about TeraGridd
  • TG Software, TG resources, TG Services
  • Software and applications can query TG Info
    services for latest information
  • Existing TG Info Services
  • Currently we are relying on the REST interface of
    TG Info Services
  • TG Also has WebMDS interfaces
  • Service includes CTSS4 information

37
Libraries for TG Info Services
  • Using the REST Interfaces we are
  • Developing JavaScript Libraries that
    automatically query and return service
    information
  • Handling all the querying and xml parsing
    necessary and return objects through libraries
  • API will include the ability to (and more)
  • getTGSites() Get a list of all TG sites
  • getResoures() Get list of resources at a site
  • getServices() Get list of TG resources
  • Late summer availability
  • Demo portlets available now from SVN
  • svn co https//ogce.svn.sourceforge.net/svnroot/og
    ce/ogce-portal-only/portlets/tg-infoservices-portl
    et/
  • Not yet integrated with full OGCE build, so you
    will need to use Maven 1.
  • Check with Maytal if you are interested.

38
Talks, Demos, and BOFs
39
Sample Portlet Interfaces
40
Future Work SC08
  • GTLAB enhancements generalized tasks, more
    gadgets, Facebook and Open Social examples,
    GridShib support.
  • Workflow Tools portal-based workflow client,
    WS-Messaging, GRAM Auditing, GridShib integration
  • JavaScript Libraries TG Info Services, support
    for Grid Services.

41
Finally
  • GCE08 workshop will be held at SC08 in Austin
  • More details to come.
  • E-Science 2008 conference is in Indianapolis in
    December.
  • Papers due July 20
Write a Comment
User Comments (0)
About PowerShow.com