US Grid Efforts

1
US Grid Efforts

• Lee Lueking
• D0 Remote Analysis Workshop
• February 12, 2002

2

• All of these projects are working towards the
  common goal of providing transparent access to
  the massively distributed computing
  infrastructure that is needed to meet the
  challenges of modern experiments (From the EU
  DataTAG proposal)

3
Grid Projects Timeline
Q3 00
GriPhyN 11.9M1.6M
Q4 00
EU DataGrid 9.3M
Q1 01
Q2 01
PPDG9.5M
Q3 01
EU DataTAG4M Euros
iVDGL13.65M
Q4 01
GridPP
Q1 02
4
PPDGParticle Physics Data Grid

• Vertical Integration of Grid middleware
  components into HENP experiments ongoing work
• An unparalleled laboratory for experimental
  computer science

iVDGLInternational Virtual Data Grid Laboratory

• A place to conduct Data Grid tests at scale
• A mechanism to create common Grid infrastructure
• A facility to perform production exercises for
  LHC experiments

5
PPDG

• Funded through DOE MICS and HENP programs
• PPDG will develop, acquire and deliver vitally
  needed Grid-enabled tools for data-intensive
  requirements of particle and nuclear physics.
• PPDG is a collaboration of computer scientists
  with a strong record in distributed computing and
  Grid technology, and physicists with leading
  roles in the software and network infrastructures
  for major high-energy and nuclear experiments.
• Our goals and plans are ultimately guided by the
  immediate, medium-term and longer-term needs and
  perspectives of the physics experiments.

6
GriPhyN Grid Physics Network

• Virtual data technologies. Advances are required
  in information models and in new methods of
  cataloging, characterizing, validating, and
  archiving software components to implement
  virtual data manipulations
• Policy-driven request planning and scheduling of
  networked data and computational resources. We
  require mechanisms for representing and enforcing
  both local and global policy constraints and new
  policy-aware resource discovery techniques.
• Management of transactions and task-execution
  across national-scale and worldwide virtual
  organizations. New mechanisms are needed to meet
  user requirements for performance, reliability,
  and cost. Agent computing will be important to
  permit the grid to balance user requirements and
  grid throughput, with fault tolerance.

7
iVDGLInternational Virtual Data Grid Laboratory

• The iVDGL will provide a global computing
  resource for several leading international
  experiments in physics and astronomy,
• Global services and centralized monitoring,
  management, and support functions functions will
  be coordinated by the Grid Operations Center
  (GOC) located at Indiana University, with
  technical effort provided by GOC staff, iVDGL
  site staff, and the CS support teams.
• The GOC will operate iVDGL as a NOC manages a
  network, providing a single, dedicated point of
  contact for iVDGL status, configuration, and
  management, and addressing overall robustness
  issues.

8
iVDGLInternational Virtual Data Grid Laboratory

• Management of the iVDGL will be integrated with
  that of the GriPhyN Project, funded by NSF in
  September 2000 for 11.9M.
• GriPhyN and Particle Physics Data Grid will
  provide the basic RD and software toolkits
  needed for the laboratory.
• The European Union DataGrid is also a major
  participant and will contribute basic
  technologies and tools.
• The iVDGL will be based on the open Grid
  infrastructure provided by the Globus Toolkit and
  will also build on other technologies such as
  Condor resource management tools.

9
So Whats the Difference?
PPDG iVDGL
Funding US DOE approved 1/1/3/3/3 M, 99 03 US NSF proposed 3/3/3/3/3 M, 02 06
Computer Science Globus (Foster), Condor (Livny), SDM (Shoshani), SRB (Moore) Globus (Foster, Kesselman), Condor (Livny)
Physics BaBar, Dzero, STAR, JLAB, ATLAS, CMS ATLAS, CMS, LIGO, SDSS, NVO
National Laboratories BNL, Fermilab, JLAB, SLAC, ANL, LBNL ANL,BNL, Fermilab (all unfunded collaborators)
Universities Caltech, SDSS, UCSD, Wisconsin Florida, Chicago, Caltech, UCSD, Indiana, Boston, Wisconsin at Milwaukee, Pennsylvania State, Johns Hopkins, Wisconsin at Madison, Northwestern, USC, UT Brownsville, Hampton, Salish Kootenai College
Hardware None 20 of funding(Tier-2 Centers)
Network No funding requested No funding requestedDataTAG complementary
10
PPDG Collaborators
11
PPDG Computer Science Groups
Condor develop, implement, deploy, and evaluate
mechanisms and policies that support High
Throughput Computing on large collections of
computing resources with distributed ownership.
http//www.cs.wisc.edu/condor/ Globus -
developing fundamental technologies needed to
build persistent environments that enable
software applications to integrate instruments,
displays, computational and information resources
that are managed by diverse organizations in
widespread locations http//www.globus.org/ SDM
- Scientific Data Management Research Group
optimized and standardized access to storage
systems http//gizmo.lbl.gov/DM.html Storage
Resource Broker - client-server middleware that
provides a uniform interface for connecting to
heterogeneous data resources over a network and
cataloging/accessing replicated data sets.
http//www.npaci.edu/DICE/SRB/index.html
12
PPDG Project Activities

• CMS GDMP Grid Data Mirroring Project
• D0 Job Management
• CMS-MOP Monte Carlo Distributed Production
• STAR-DDM Uses HRM, (Hierarchical Resource
  Manager)
• JLAB-SRBStorage Resource Broker, Replication
• ATLAS MAGDA distributed data manager
  (ATLAS-Globus)

13
PPDG Cross-cut Activities

• SC2001
• Certificate /Registration Authority
• Collaboration with IEPM, Network Performance
  Monitoring

14
Super Computing 2001 in Denver
 
15
(No Transcript)
16
Common Services

• Job Description Language
• Scheduling and Management of Processing and Data
  Placement Activities
• Monitoring and Status Reporting
• Storage Resource Management
• Reliable Replica Management Services
• File Transfer Services
• Collect and Cocument Current Experimental
  Practices
• R D, Evaluation
• Authentication, Authorization, and Security
• End-to-End Applications and Testbeds

17
Delivery of End-to-End Applications Integrated
Production Systems
to allow thousands of physicists to share data
computing resources for scientific processing and
analyses

• PPDG Focus
• Robust Data Replication
• - Intelligent Job Placement
• and Scheduling
• - Management of Storage
• Resources
• - Monitoring and Information
• of Global Services

• Relies on Grid infrastructure
• - Security Policy
• High Speed Data Transfer
• - Network management

Operators Users
Resources Computers, Storage, Networks
18
Project Activities,End-to-End Applicationsand
Cross-Cut Pilots

• Project Activities are focused Experiment
  Computer Science Collaborative developments.
• Replicated data sets for science analysis
  BaBar, CMS, STAR
• Distributed Monte Carlo production services
  ATLAS, D0, CMS
• Common storage management and interfaces STAR,
  JLAB

• End-to-End Applications used in Experiment data
  handling systems to give real-world requirements,
  testing and feedback.
• Error reporting and response
• Fault tolerant integration of complex components

• Cross-Cut Pilots for common services and policies
  
• Certificate Authority policy and authentication
• File transfer standards and protocols
• Resource Monitoring networks, computers,
  storage.

19
PPDG activities as part of the Global Grid
Community

• Coordination with other Grid Projects in our
  field
• GriPhyN Grid for Physics Network
• European DataGrid
• Storage Resource Management collaboratory
• HENP Data Grid Coordination Committee

• Participation in Experiment and Grid deployments
  in our field
• ATLAS, BaBar, CMS, D0, Star, JLAB experiment data
  handling systems
• iVDGL/DataTAG International Virtual Data Grid
  Laboratory
• Use DTF computational facilities?

• Active in Standards Committees
• Internet2 HENP Working Group
• Global Grid Forum

20
PPDG and GridPP Projects

• Use of Standard Middleware to Promote
  Interoperability
• Move to Globus infrastructure GSI, GridFTP
• Use of Condor as a supported system for job
  submission
• Publish availability of resources and file
  catalog
• Additional Grid Functionality for Job
  Specification, Submission, and Tracking
• Use Condor for migration and check pointing
• Enhanced job specification language and services
• Enhanced Monitoring and Diagnostic Capabilities
• Fabric Management

21
PPDG Management and Coordination
PIsLivny, Newman, Mount
Steering CommitteeRuth Pordes, ChairDoug Olson,
Physics Deputy ChairMiron Livny, Computer
Science Deputy ChairComputer Science Group
RepresentativesPhysics Experiment
RepresentativesPIs (ex officio)
Executive Team (gt1.0 FTE on PPDG)Steering
Committee ChairSteering Committee Physics and CS
Deputy Chairs
STAR
SDM
BaBar
SRB
JLAB
ATLAS
Globus
CMS
Condor
DZero
22
iVDGL
We propose to create, operate and evaluate, over
asustained period of time, an international
researchlaboratory for data-intensive
science. From NSF proposal, 2001

• International Virtual-Data Grid Laboratory
• A global Grid laboratory with participation from
  US, EU, Asia, etc.
• A place to conduct Data Grid tests at scale
• A mechanism to create common Grid infrastructure
• A facility to perform production exercises for
  LHC experiments
• A laboratory for other disciplines to perform
  Data Grid tests

23
iVDGL Summary Information

• Principal components (as seen by USA)
• Tier1 sites (laboratories)
• Tier2 sites (universities and other institutes)
• Selected Tier3 sites (universities)
• Fast networks US, Europe, transatlantic
• International Grid Operations Center (iGOC)
• Computer Science support teams
• Coordination, management
• Proposed international partners
• Initially US, EU, Japan, Australia
• Other world regions later
• Discussions w/ Russia, China, Pakistan, India,
  South America
• Complementary EU project DataTAG
• Transatlantic network from CERN to STAR-TAP (
  people)
• Initially 2.5 Gb/s

24
US Proposal to NSF

• US proposal approved by NSF Sept. 25, 2001
• Part 2 of GriPhyN project
• Much more application oriented than first GriPhyN
  proposal
• 15M, 5 years _at_ 3M per year (huge constraint)
• CMS ATLAS LIGO SDSS/NVO Computer Science
• Scope of US proposal
• Deploy Grid laboratory with international
  partners
• Acquire Tier2 hardware, Tier2 support personnel
• Integrate of Grid software into applications
• CS support teams ( 6 UK Fellows) to harden tools
• Establish International Grid Operations Center
  (iGOC)
• Deploy hardware at 3 minority institutions (Tier3)

25
US iVDGL Proposal Participants

• U Florida CMS
• Caltech CMS, LIGO
• UC San Diego CMS, CS
• Indiana U ATLAS, iGOC
• Boston U ATLAS
• U Wisconsin, Milwaukee LIGO
• Penn State LIGO
• Johns Hopkins SDSS, NVO
• U Chicago CS
• U Southern California CS
• U Wisconsin, Madison CS
• Salish Kootenai Outreach, LIGO
• Hampton U Outreach, ATLAS
• U Texas, Brownsville Outreach, LIGO
• Fermilab CMS, SDSS, NVO
• Brookhaven ATLAS
• Argonne Lab ATLAS, CS

T2/Software
CS support
T3/Outreach
T1/Labs
26
iVDGL Partners

• National partners
• PPDG (Particle Physics Data Grid )
• DTF Distributed Terascale Facility
• CAL-IT2 (new California Grid initiative)
• Current international partners
• EU-DataGrid
• UK PPARC funding agency
• UK Core e-Science Program ? 6 UK Fellowships
• INFN (Italy)
• 2 Japanese institutes
• 1 Australian institute (APAC)

27
iVDGL Map Circa 2002-2003
28
iVDGL Requirements

• Realistic scale
• In number, diversity, distribution, network
  connectivity
• Delegated management and local autonomy
• Management needed to operate as large, single
  facility
• Autonomy needed for sites and experiments
• Support large-scale experimentation
• To provide useful information for building real
  Data Grids
• Robust operation
• For long running applications in complex
  environment
• Instrumentation and monitoring
• Required for an experimental facility
• Integration with international cyberinfrastructur
  e
• Extensibility

29
Approach

• Define a laboratory architecture
• Define expected laboratory functions
• Build in scalability, extensibility,
  reproducibility
• Define instrumentation, monitoring
• Establish CS support teams (develop/harden tools,
  support users)
• Define working relationship, coordination with
  partners
• Create and operate global-scale laboratory
• Deploy hardware, software, personnel at Tier2,
  Tier3 sites
• Establish iGOC, single point of contact for
  monitoring, support,
• Help international partners establish sites
• Evaluate and improve iVDGL through
  experimentation
• CS support teams will work with experiments
• Extend results to partners
• Engage underrepresented groups
• Integrate minority institutions as Tier3 sites

30
iVDGL as a Laboratory

• Grid Exercises
• Easy, intra-experiment tests first (10-30,
  national, transatlantic)
• Harder wide-scale tests later (30-100 of all
  resources)
• CMS is already conducting transcontinental
  simulation productions
• Operation as a facility
• Common software, central installation to ensure
  compatibility
• CS teams to harden tools, support applications
• iGOC to monitor performance, handle problems

31
Emphasize Simple Operation

• Local control of resources vitally important
• (Site level or national level)
• Experiments, politics demand it
• Operate mostly as a partitioned testbed
• (Experiment, nation, etc.)
• Avoids excessive coordination
• Allows software tests in different partitions
• Hierarchy of operation must be defined
• E.g., (1) National experiment, (2) inter-expt.,
  (3) global tests

32
Other Disciplines

• Use by other disciplines
• Expected to be at the 10 level
• Other HENP experiments
• Virtual Observatory (VO) community in Europe/US
• Gravity wave community in Europe/US/Australia/Japa
  n
• Earthquake engineering
• Bioinformatics
• Our CS colleagues (wide scale tests)

33
US iVDGL Management and Coordination
Collaboration Board (Advisory)
External Advisory Board
Project DirectorsAvery, Foster
Project Coordination GroupProject
CoordinatorProject DirectorsCoordinators of
Systems Integration, Education/OutreachPhysics
Experiment RepresentativesUniversity Research
Center or Group RepresentativesPACI
Representatives
iVDGL Design and Deployment
Integration with Applications
University Research Centers / Groups
International Grid Operations Center
34
Conclusion

• Grid involvement offers many challenges and
  opportunities.
• PPDG, and iVDGL are complementary in their
  approach and deliverables.
• These efforts, along with our European partners
  will provide exciting new ways to share data and
  computing resources in the future.
• Acknowledgements Richard Mount (SLAC) and Paul
  Avery (University of Florida), Ruth Pordes
  (FNAL).