TeraGrid A National Production Cyberinfrastructure Facility

1
TeraGridA National ProductionCyberinfrastructure
Facility

• Charlie Catlett, TeraGrid Director
• University of Chicago and Argonne National
  Laboratory
• cec_at_uchicago.edu

2
TeraGrid Integrating NSF Cyberinfrastructure
PSC
PU
UC/ANL
NCAR (mid-2006)
IU
NCSA
ORNL
TACC
SDSC
3
NSF TeraGrid In Context
Operation Enhancement
Construction
Petascale
NSF Centers Program
PACI

1/00
1/05
1/10
1/95
1/90
1/85
4
The TeraGrid Facility

• Grid Infrastructure Group (GIG)
• University of Chicago
• TeraGrid integration, planning, management,
  coordination
• Resource Providers (RP)
• Currently NCSA, SDSC, PSC, Indiana, Purdue, ORNL,
  TACC, UC/ANL
• Additional RPs in discussion
• Systems (resources, services) support, user
  support
• Provide access to resources via policies,
  software, and mechanisms coordinated by and
  provided through the GIG.
• The Facility
• An integrated set of HPC resources providing NSF
  scientists with access to resources and
  collections of resources through unified user
  support, coordinated software and services, and
  extensive documentation and training.
• The Federation
• Interdependent partners working together under
  the direction of an overall project director, the
  GIG PI.

5
TeraGrid Objectives

• DEEP Science Enabling Terascale Science
• Make Science More Productive through an
  integrated set of very-high capability resources.
• WIDE Impact Empowering Communities
• Bring TeraGrid capabilities to the broad science
  community.
• OPEN Infrastructure, OPEN Partnership
• Provide a coordinated, general purpose, reliable
  set of services and resources.

6
Science GatewaysA new initiative for the TeraGrid

• Increasing investment by communities in their own
  cyberinfrastructure, but heterogeneous
• Resources
• Users from expert to K-12
• Software stacks, policies
• Science Gateways
• Provide TeraGrid Inside capabilities
• Leverage community investment
• Three common forms
• Web-based Portals
• Application programs running on users' machines
  but accessing services in TeraGrid
• Coordinated access points enabling users to move
  seamlessly between TeraGrid and other grids.

Source Dennis Gannon (gannon_at_cs.indiana.edu)
7
TeraGrid Resources
Updated by Kelly Gaither (gaither_at_tacc.utexas.edu)
8
TeraGrid Facility Today
Common TeraGrid Computing Environment
Local Value-Added User Environment
Heterogeneous Resources at Autonomous Resource
Provider Sites

• A single point of contact for help
• Integrated documentation and training
• A common allocation process
• A common baseline user environment
• Services to assist users in harnessing the right
  TeraGrid platforms for each part of their work,
• Enhancements driven by users.
• Science Gateways to engage broader communities

9
TeraGrid Usage
10
TeraGrid Allocation by Field of Science
Additional fields 2
Advanced Scientific Computing 3
Atmospheric Sciences
Molecular Biosciences
7
20
Chemical,
Thermal Systems
8
Physics
Materials Research
18
10
Astronomical Sciences 15
Chemistry
12
Data for 1000 projects as of April 2006 provided
by David Hart (dhart_at_sdsc.edu)
11
On Demand Predicting Severe Weather Droegemeier
(OU) and LEAD
Large Data Virtualized Resources Earthquake
Simulation Olsen (SDSU), Okaya (USC), Southern
California Earthquake Center
Sources Kelvin Droegemeier (OU), Dennis Gannon
(IU), Tom Jordan (USC). Images by PSC and SDSC.
12
Virtualized Resources, EnsemblesFOAM Climate
Model Liu (UWisc)
Coupled Simulation Full Body Arterial Tree
Simulation Karniadakis (Brown)
Sources Ian Foster (UC/ANL), Mike Papka
(UC/ANL), George Karniadakis (Brown). Images by
UC/ANL.
13
Social and Behavioral Science Gateway
For social and behavioral scientists, comparing
apples to oranges is essential. Understanding
neural, cognitive, and social behaviors can
depend on the ability to uncover coherent
patterns among disparate data collected at
different times and places. Researchers correlate
subjects videotaped reactions with eye
movements, heart rates, electroencephalogram
results, and answers to written surveys. Its
complicated, says Chicago psychologist Bennett
Bertenthal, and technological infrastructure has
not kept pace with research needs.So Bertenthal
and a multidisciplinary team of more than a dozen
scientists from Chicago, Argonne National
Laboratory, and the University of Illinois at
Chicago are working to develop a set of
cybertools to help collect and analyze behavioral
data, collaborating with the TeraGrid Science
Gateways Program.
Central to the projectfunded with 2 million
from the National Science Foundationis the
Social Informatics Data (SID) Grid, a vast and
sophisticated warehouse of readings and
measurements meant to foster collaboration and to
spur the development of standards for gathering
and coding datawhich, Bertenthal says, right
now is not part of the landscape. In two years,
when the SID Grid is complete, psychologists,
sociologists, anthropologists, economists, and
neuroscientists can share notes across the globe
and use software interfaces like the ones shown
above to synthesize numerous forms of streaming
data at once.
Adapted from University of Chicago Magazine
14
Enhancements Driven by User Input
Overall Score (depth of need)
Partners in Need (breadth of need)
Remote File Read/Write
High-Performance File Transfer
Coupled Applications, Co-scheduling
Grid Portal Toolkits
Grid Workflow Tools
Batch Metascheduling
Global File System
Client-Side Computing Tools
Batch Scheduled Parameter Sweep Tools
Advanced Reservations
Data
Results of in-depth interviews with over 30 users
from 16 TeraGrid user teams.
Capability Type
Grid Computing
Science Gateways
15
Source Jay Boisseau (TACC), Eric Roberts (TACC)
16
Science Gateways

• Independent infrastructure using TeraGrid
  resources in back-end fashion
• Local gateway user management, authentication,
  authorization
• Service Oriented Architecture using TeraGrid via
  web services such as job submission, file
  movement, authorization, job tracking and audit,
  etc.
• Current Status
• 15 prototypes at various stages of local
  production (some new gateways, most are
  operational with local services, early versions
  of TeraGrid-enabled services)
• Developing scalable model to allow for many
  independent gateways, using standard TeraGrid
  services interfaces, with gateway-managed
  TeraGrid computational and storage allocations.

17
Science Gateway Examples
As well as additional gateway projects that have
joined us or are planning to join, including
University of Buffalo, BIRN ,NEES, GEON, Several
NCAR projects, Cornell (large data collections),
LSU (coastal modeling), IU Hydra Portal
18
Integrating New Resources

• Primer outlining TeraGrid architecture with
  respect to integrating new computational systems
• www.teragrid.org/basics/
• Four-Stage process
• Definition, Evaluation and Negotiation
• Resource to be integrated, support commitments,
  etc.
• Planning
• Integration
• Production
• Working with several interested sites
• NCAR at stage 2
• LSU, PSU, Cornell at stage 1