Requirements for Grid Enabling an Application

1
Requirements for Grid Enabling an Application

• This presentation will probably involve audience
  discussion, which will create action items. Use
  PowerPoint to keep track of these action items
  during your presentation
• In Slide Show, click on the right mouse button
• Select Meeting Minder
• Select the Action Items tab
• Type in action items as they come up
• Click OK to dismiss this box
• This will automatically create an Action Item
  slide at the end of your presentation with your
  points entered.

Ruxandra Bondarescu, Gregory Daues National
Center for Supercomputing Applications University
of Illinois at Urbana-Champaign

2
Overview

• Description of applications we target
• LCA codes
• Enzo
• Zeus-MP
• BIMA pipeline
• Cactus
• When is an application ready to be
  Grid-enabled?
• How do we rework an application with the Grid in
  mind?
• Can this be standardized?
• Recommended list of features (e.g.,
  checkpointing, steering capabilities)
• Structure (grid functions, parameters ?)

3

• Developed by Mike Norman (UCSD), Greg Bryan (now,
  MIT), and others
• Adaptive Mesh Refinement (AMR) hydrodynamic
  cosmology code
• X-Ray clusters
• Molecular clouds
• Galaxy formation
• First star in the universe
• Driving scenarios
• Large-scale simulation (1000 processors)
• run to generate a catalog of simulated galaxies
• On the order of 100 simulations

4

• Benefits from the GAT
• Improved data management
• Moving, accessing, postprocessing complex AMR
  data
• Fault tolerance by checkpointing and restarting
• Organized production of catalogs by taskfarming
• Migration to new resources for AMR simulations
• Cosmological structures form late, suddently
  increasing the number of grids/work load
• Is Enzo ready to use the GAT?
• Has grid variables and parameters? (yes)
• Adequate checkpointing? (perhaps not)
• Checkpointing not robust and not optimal
• Conventional checkpointing is unsuitable for a
  multitimescale simulation work not uniform in
  time
• Not adapting to a change in the number of
  processors (impedes complex Grid operations such
  as migration)
• Monitoring and steering? (not yet implemented)

5
Is Cactus the model for being ready for the GAT?
6

• Berkeley Illinois Maryland Association (BIMA)
  Consortium
• runs the BIMA array (a 10-element mm-wave radio
  interferometer) in Hat Creek, California
• BIMA Radio Astronomy Data Processing Pipeline
• A driver application within the Alliance
  Portal Expedition
• Application Lead Ray Plante, NCSA
• Jay Alameda at NCSA
• Dennis Gannon, Beth Plale, Marlin Pierce at
  the University of Indiana
• BIMA Pipeline automated system for managing
  image processing simulations
• Automatically launch numerous instances of
  applications (e.g., Aips, Myriad) for
  processing raw telescope data
• Legacy software for astronomical image
  generation, analysis

7

• BIMA Pipeline (continued )
• Current telescope specific system
• Basic Web Service interface to job queuing system
• Each queue holds jobs of a given class
• e.g., create a mosaic from a set of images
• Central pipeline is a Java based program that
  starts Unix jobs on a dedicated server
• State of the pipeline/jobmanger and its queues
  stored in xml documents
• Web page display through XSLT
• Planned Upgrades
• Central pipeline/jobmager will use Java CoG to
  launch jobs on GT2.x Grid resources
• Individual jobs will be bundled into job requests
  
• Each job request is a set of jobs that must run
  in sequence (e.g., first remove noise from an
  image, then calculate a histogram)
• Portlet will display the pipeline status

8

• Migration to Grid and Portal technology offers
• application to other radio telescopes
• broader use throughout astronomy community
• Can the GAT bring further enhancements to the
  BIMA pipeline?
• Would it bring an improved taskfarming?
• Facilitate system recovery?
• Benefit to the application codes
• Are they fit to be Grid-enabled?

9
General comments

• Have a GAT webpage (like www.gridsphere.org)
• Describe the software that is already written
• E.g., Taskfarming was very successful can the
  code be made available in a documented way?
• GAT API questions
• How will the GAT interact with generic OGSA
  services?
• What objects will be used? (we saw only a
  ResourceBroker object)
• Will the GAT generate a global file space? Or
  rely on Storage Resource Broker (SRB), Globus
  Replica Catalog
• Can the GAT interact with a Data Grid composed of
  resources running the SDSC SRB? The Extended
  Terascale Facility (TeraGrid) will likely be SRB
  based.
• Is the GAT going to use/interact with a steering
  library? Or should applications do that? Can that
  be a way to standardize steering?
• Portal-GAT
• It would help to have a document describing the
  interaction Portal-GAT?
• Would a portal interact with Grid services
  through the GAT?