CMS%20Software%20and%20Computing%20FNAL%20Internal%20Review%20of%20USCMS%20Software%20and%20Computing

1
CMS Software and ComputingFNAL Internal Review
of USCMS Software and Computing

• David Stickland
• Princeton University
• CMS Software and Computing Deputy Project Manager

2
CMS Software and Computing - Overview

• Outline
• Status of Software / Computing Milestones
• CMS Software Architecture development plans
• CMS Computing plans
• CMS Reorganization of Software and Computing
  Project
• Key issues in 2001

3
Strategic Software Choices

• Modular Architecture (flexible and safe)
• Object-Oriented Framework
• Strongly-typed interface
• Uniform and coherent software solutions
• One main programming language
• One main persistent object manager
• One main operating system
• Adopt standards
• Unix, C, ODMG, OpenGL...
• Use widely spread, well supported products (with
  healthy future)
• Linux, C, Objectivity, Qt...
• Mitigate risks
• Proper planning with milestones
• Track technology evolution investigate and
  prototype alternatives
• Verify and validate migration paths have a
  fall-back solution ready

4
CARFCMS Analysis Reconstruction Framework
Physics modules
Specific Framework
Reconstruction Algorithms
Data Monitoring
Event Filter
Physics Analysis
Generic Application Framework
Calibration Objects
Event Objects
Configuration Objects
CMS adapters and extensions
Utility Toolkit
ODBMS
C standard library Extension toolkit
Geant3/4
CLHEP
Paw Replacement
5
Milestones Software

• CMS software development strategy
• First, transition to C, then functionality,
  then performance

6
Software Life Cycle
Design
Prototypes
Detector
Mass-production
Installation Commissioning
Maintenance Operation
Computing Hardware
Functional Prototype
Fully Functional
Production System
Software Integration
Design
Prototype
Deploy
Cyclic Releases
Test Integrate
2002
2001
2005
2000
2025
7
Software Development Phases

• 1 Proof of Concept End of 1998
• Basic functionality
• Very loosely integrated

• 5 Production System
• Online / Trigger Systems 75 ? 100Hz
• Offline Systems few 1015 Bytes / year
• 109 events / year to look for a handful of
  (correct!) Higgs
• Highly distributed collaboration and resources
• Long lifetime

• 2 Functional Prototype
• More complex functionality
• Integrated into projects
• Preparation for Trigger and DAQ TDRs
• Reality Check 1 Data Challenge

• 3 Fully Functional System
• Complete Functionality
• Integration across projects
• Reality Check 5 Data Challenge
• SW/computing TDR
• Preparation for Physics TDR

Logarithmic

• 4 Pre-Production System
• Reality Check 20 Data Challenge

2002
2001
2004
2003
2005
2000
2025
8
Significant Requirements from CMS TDRs or
its not just an evolution to 2005 software
TDR
Software Computing Dec 2002
Trigger Dec 2000
DAQ Dec 2001
Physics Dec 2003
Major Core Software Milestones
2002
2001
2004
2003
2005
2000
2025
9
Milestones Data

• Raw data 1 MB / event, 100 Hz ? 1 PB /
  year
• reconstructed data, physics objects,
  calibration data, simulated data, ...

Now moving towards distributed production and
analysis
10
Milestones Hardware
TDR and MoU
11
Why Worldwide Computing?Regional Center Concept
Advantages

• Common access for Physicists everywhere
• Maximize total funding resources while meeting
  the total computing need
• Proximity of datasets to appropriate resource
• Tier-n Model
• Efficient use of network bandwidth
• Local gt regional gt national gt international
• Utilizing all intellectual resources
• CERN, national labs, universities, remote sites
• Scientists, students
• Greater flexibility to pursue different physics
  interests, priorities, and resource allocation
  strategies by region
• Systems complexity
• ? Partitioning of facility tasks, to manage and
  focus resources

12
Milestone Review

• Regional Centres
• Identify initial candidates 06/2000
• Turn on functional centres 12/2002
• Fully operational centres 06/2005
• Central (CERN) systems
• Functional prototype 12/2002
• Turn on initial systems 12/2003
• Fully operational systems 06/2005
• Need to define intermediate working milestones

13
CMS Regional Centre Prototypes 2003

• Candidates
• Tier1 Tier2
• Finland - Helsinki
• France CCIN2P3/Lyon ?
• India - Mumbai
• Italy INFN INFN, at least 3 sites
• Pakistan - Islamabad
• Russia Moscow Dubna
• UK RAL ?
• US FNAL Caltech, U.C.-San Diego,Florida,
  Iowa, Maryland Minnesota, Wisconsin and
  others

14
The Grid Services Concept

• Standard services that
• Provide uniform, high-level access to a wide
  range of resources (including networks)
• Address interdomain issues security, policy
• Permit application-level management and
  monitoring of end-to-end performance
• Perform resource discovery
• Manage authorization and prioritization
• Broadly deployed (like Internet Protocols)

15
Why CMS (HEP) in the GRID?

• GRID Middleware provides a route towards
  effective use of distributed resources and
  complexity management
• GRID design matches the MONARC hierarchical Model
• We (HEP) have some Grid-like tools (hand-made).
  The scale of CMS Computing requires a more
  professional approach to live for decades.
• CMS already participates in relevant GRID
  initiatives, e.g.
• The Particle Physics Data Grid (PPDG) US
  Distributed Data Services and Data Grid System
  Prototypes
• Grid Physics Network (GriPhyN ) US
  Production-Scale Data Grids
• DATAGRID EU Middleware development and Real
  Applications Test

16
Grid Data Management Prototype (GDMP)

• Distributed Job Execution and Data
  HandlingGoals
• Transparency
• Performance
• Security
• Fault Tolerance
• Automation

Site A
Site B
Submit job
Replicate data
Job writes data locally
Replicate data

• Jobs are executed locally or remotely
• Data is always written locally
• Data is replicated to remote sites

Site C
17
Computing Progress

• CMS is progressing towards a coherent distributed
  system, to support production and analysis
• We need to study the problems and prototyping
  the solutions for distributed analysis by
  hundreds of users in many countries
• Production, via prototypes, will lead to
  decisions about the architecture on the basis of
  measured performances and possibilities

18
CMS Software and Computing

• Evolve the organization to build a complete and
  consistent Physics Software
• Recognize cross-project nature of key
  deliverables
• Core Software and Computing CSWC
• More or less what US calls SWC Project
• Physics Reconstruction Selection PRS
• Consolidate Physics Software work between the
  detector groups targeted at CMS deliverables (HLT
  design, test-beams, calibrations, Physics TDR..
• Trigger and Data Acquisition TRIDAS
• Online Event Filter Farm

19
Cross-Project Working Groups
Core Software and Computing
Physics Reconstruction Selection
TRIDAS
Reconstruction Project
Simulation Project
Calibration etc..
Joint Technical Board
20
Key issues for 2001

• Choice of baseline for database
• Ramp-up of Grid RD and use in production
  activities
• Perform HLT data challenge
• ( 500CPUs at CERN 500 offsite, 20 TB)
• Continue work on test beams, and validate
  simulation
• Validate OSCAR/Geant4 for detector performance
  studies
• Overall assessment, formalization and
  consolidation of SW systems and processes
• Work towards Computing MoU in the collaboration