U.S. ATLAS Software WBS 2.2

1
U.S. ATLAS SoftwareWBS 2.2

• S. Rajagopalan
• July 8, 2003
• DOE/NSF Review of LHC Computing

2
Outline

• Organizational Issues
• ATLAS U.S. ATLAS software
• Current Affairs
• Current resource allocation including LCG
  contributions
• Major milestones met
• FY04 Planning
• Planning, coordination with international ATLAS
• Near term milestones
• Priorities and request for FY04
• Conclusions

3
Organizational Issues

4
New Computing Organization
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
x
5
Computing Management Board

• Coordinate Manage computing activities
• Set priorities and take executive decisions
• Computing Coordinator (chair)
• Software Project Leader (D. Quarrie, LBNL)
• TDAQ Liaison
• Physics Coordinator
• International Computing Board Chair
• GRID, Data Challenge and Operations Coordinator
• Planning Resources Coordinator (T. Lecompte,
  ANL)
• Data Management Coordinator (D. Malon, ANL)
• Meets bi-weekly

6
Software Project Management Board

• Coordinate the coherent development of software
• (core, applications and software support)
• Software Project Leader (chair) D. Quarrie
• Simulation coordinator
• Event Selection, Reconstruction Analysis Tools
  coordinator
• Core Services Coordinator (D. Quarrie)
• Software Infrastructure Team Coordinator
• LCG Applications Liaison (T. Wenaus, BNL)
• Physics Liaison
• TDAQ Liaison
• Sub-System Inner Detector, Liquid Argon, Tile,
  Muon coordinators
• Liquid Argon S. Rajagopalan (BNL), Muon S.
  Goldfarb (U Mich)
• Meets bi-weekly

7
US ATLAS Software Organization
Software Project (WBS 2.2) S. Rajagopalan
Coordination (WBS 2.2.1)
Core Services (WBS 2.2.2) D. Quarrie
Data Management (WBS 2.2.3) D. Malon
Application Software (WBS 2.2.4) F. Luehring
Software Support (WBS 2.2.5) A. Undrus

• US ATLAS software WBS scrubbed, consistent with
  ATLAS
• Resource Loading and Reporting established at
  Level 4
• Major change compared to previous WBS
• Production and Grid Tools Services moved under
  Facilities

8
Current Affairs

9
WBS 2.2.1 Coordination

• David Quarrie (LBNL)
• ATLAS Software Project Manager
• ATLAS Chief Architect
• U.S. ATLAS Core Services Level 3 Manager
• David Malon (ANL)
• ATLAS Data Management Coordinator
• U.S. ATLAS Data Management Level 3 Manager
• Other U.S. Atlas personnel playing leading roles
  in ATLAS
• S. Goldfarb (Muon), T. LeCompte (Planning),
• S. Rajagopalan (LAr), T. Wenaus (LCG Liaison)

10
WBS 2.2.2 Core Services(D. Quarrie)

• P. Calafiura (LBNL)
• Framework support, Event Merging, EDM
  infrastructure
• M. Marino (LBNL)
• SEAL plug-in and component support
• W. Lavrijsen (LBNL)
• User interfaces, Python scripting, binding to
  dictionary, integration with GANGA.
• C. Leggett (LBNL)
• Conditions infrastructure, G4 Service integration
  in Athena, Histogramming support. Redirected to
  other tasks in FY04
• H. Ma, S. Rajagopalan (BNL) (Base Program) EDM
  infrastructure
• C. Tull (LBNL) (PPDG) Athena Grid Integration
  coordination

11
WBS 2.2.2 Key Accomplishments

• Python based user interfaces to CMT, Athena, and
  ROOT
• Interval of Validity Service to allow time-based
  retrieval of conditions data into transient
  memory
• Support for plug-in manager in LCG/SEAL
• gcc-3.2 support, multithreading support, pile-up
  support.
• Services to upload persistent addresses for
  on-demand retrieval of data objects
• Common Material Definition across sub-systems,
  creation of G4 geometries from this description
  demonstrated

12
WBS 2.2.3 Data Management(D. Malon)

• S. Vanyachine (ANL)
• Database Services Servers, NOVA database
• Kristo Karr (ANL)
• New Hire, replaces S. Eckmann
• Collections, Catalogs and Metadata
• Valeri Fine (BNL)
• Integration of Pool with Athena
• David Adams (BNL)
• Event datasets
• Victor Perevotchikov (BNL)
• POOL evaluation, foreign object persistent in
  ROOT.

13
WBS 2.2.3 Key Accomplishments

• ATLAS specific
• Athena-Pool conversion service prototype
• Will be available to end user in July (tied to
  POOL release)
• Support for NOVA database
• (primary source for detector description for
  simulation)
• Support for interval of validity
• NOVA automatic object generation
• Data additions, embedded MYSQL support for G4
• Authentications, access to databases behind
  firewalls
• LCG contributions
• Delivered POOL collections/metadata WP interface,
  doc unit tests
• Delivered relational implementation of POOL
  explicit collections
• Delivered MYSQL and related package support
• Foreign object persistence

14
WBS 2.2.4 Application Software(F. Luehring)

• Geant3 simulation support
• Calorimeter (LAr Tile) software incl.
  calibration
• Pixel, TRT detector simulation digitization
• Muon reconstruction and database
• Hadronic calibration, tau and jet reconstruction
• electron-gamma reconstruction
• High Level Trigger software
• Physics analysis with new software

BNL
ANL, BNL, Nevis Labs, U. Arizona, U. Chicago, U.
Pittsburgh, SMU
Indiana U., LBNL
BNL, Boston U., LBNL, U. Michigan
U. Arizona, U. Chicago, ANL, BNL, LBNL
BNL, Nevis Labs, SMU
U. Wisconsin
U. S. ATLAS
15
WBS 2.2.5 Software Support(A. Undrus)

• Release and maintenance of ATLAS and all
  associated external software (including LCG
  software, LHCb Gaudi builds) at the Tier 1
  Facility.
• Deployment of a nightly build system at BNL, CERN
  and now used by LCG as well.
• Testing releases with new compilers (gcc-3.2, SUN
  5.2).
• Software Infrastructure Team Forum for
  discussions of issues related to support of ATLAS
  software and associated tools. A. Undrus is a
  member of this body.

16
US FY03 contribution in international context

Excludes David Quarrie Torre Wenaus
coordination role contributions
17
LCG Application Component

• US effort in SEAL 1.0 FTE (FY03)
• Plug-in manager (M. Marino (0.75 FTE, LBNL)
• Internal use by POOL now, Full integration into
  Athena Q3 2003
• Scripting Services (W. Lavjrisen 0.25 FTE, LBNL)
• Python support and integration
• US effort in POOL 1.2 FTE (FY03)
• Principal responsibility in POOL collections and
  metadata WP
• D. Malon, K. Karr, S. Vanyachine (0.5 FTE) ANL
• POOL Datasets (D. Adams, 0.2 FTE, BNL)
• Common Data Management Software
• V. Perevoztchikov, ROOT I/O foreign object
  persistence (0.3 FTE, BNL
• POOL mysql package and server configurations
  (ANL, 0.2 FTE)

18
US ATLAS contribution in LCG

• Contribution to Application Area only
• Snapshot (June 2003) contribution

19
ATLAS interactions with LCG

• Lack of manpower has made ATLAS participation
  weaker than we would like
• Little or no effort available to
• Participate in design discussions of POOL SEAL
  omponents for which we are not directly
  responsible
• Evaluate and test new features
• Write ATLAS acceptance tests for POOL releases
  and for specifically requested features
• Ensure that ATLAS priorities are kept prominent
  in LCG plans (ATLAS does this, but our voice has
  at times seemed not as loud as that of our
  sisters)
• Less development contributed in the
  collections/metadata work package (for which we
  are responsible) than we would have liked, though
  this should improve soon with recent hire at ANL

20
FY04 Plans
21
International ATLAS Planning

• ATLAS has a planning officer T. LeCompte (ANL)
• The current focus is on defining the WBS and
  establishing coherent short term plans.
• US WBS used as a starting point!
• Responsibility in monitoring all deliverables
  including non-ATLAS components (such as LCG) and
  assessing the impact from any delays.
• Responsibility for establishing the software
  agreements and scope with international ATLAS
  institutions.

22
ATLAS Computing Timeline
23
Major near term milestones

• July to Dec 2003 SEAL/POOL/PI deployment by LCG
• Sept. 2003 Geant 4 based simulation release
• Dec. 2003 Validate Geant4 release for DC2 and
  test-beam
• Dec. 2003 First release of full ATLAS software
  chain using LCG components and Geant4 for use in
  DC2 and combined test-beam.
• Spring 2004 Combined Test-Beam runs.
• Spring 2004 Data Challenge 2
• Principal means by which ATLAS will test and
  validate its proposed Computing Model
• Dec. 2004 Computing Model Document released

24
U.S. scope issues

• 2003-2004 Develop sufficient core software
  infrastructure to deploy and exercise a
  reasonable prototype of the ATLAS Computing Model
• ATLAS is quite far from being able to do this
• Now is not the time to sacrifice core software
  development
• Doing so puts the TDR and hence the readiness for
  LHC turn-on at risk.
• U.S was asked to lead the effort in coordinating,
  developing and deploying the ATLAS architecture
  (from ground-zero in 1999).
• Leadership roles in Software Project,
  Architecture and Data Management. major
  responsibilities - but minimal resources to work
  with.
• We are responsible to ensure the success of ATLAS
  architecture.
• Efforts are continuing to be made in encouraging
  and recruiting non-US institutions US
  universities to contribute to core and leveraging
  from LCG.

25
Core Software Physicists

• The presence of a strong core team in the U.S.
  has helped U.S. physicists make significant
  contributions to reconstruction, simulation and
  physics analysis. in turn allowing them to play
  an influential role in the overall ATLAS software
  program.
• Examples from LAr, InDet simulation and Calo,
  Muon reconstruction, event generation
  infrastructure, egamma, tau, jet reconstruction,
  calibration,
• Conversely, this has also allowed U.S. physicists
  to provide valuable feedback to core software and
  in some cases contribute to the core development
• Examples are the Event Data Model and the
  Detector Description efforts.

This harmony is necessary to allow U.S. to
develop the necessary expertise and effectively
contribute to the physics at turn-on.
26
Incremental Effort Core Services

• Redirections
• C. Leggett (0.5 Calibration Infrastructure to
  EDM)
• M. Marino (0.25 Training to SEAL/Framework)
• Additions (prioritized)
• 1.0 FTE in Detector Description, WBS 2.2.2.3
  (U. Pittsburgh)
• New Hire to work with J. Boudreau
• 0.5 FTE in Analysis Tools support, WBS 2.2.2.5
• New Hire or redirection of effort
• 1.0 FTE in Graphics, WBS 2.2.2.4 (UC Santa
  Cruz)
• Existing person (G. Taylor) who is currently
  making significant contributions to ATLANTIS
  (Atlas Graphics Package).

27
Detector Description

• ATLAS lacked a Detector Description Model
• Numbers hardwired in reconstruction, no
  commonality with simulation.
• Along came Joe Boudreau (U. Pittsburgh) CDF
  experience
• Successfully designed, developed and deployed a
  prototype model for both material and readout
  geometry. We encouraged this!
• Automatically handles alignments, Optimized for
  memory (5 MB for describing ATLAS geometry), Not
  coupled to visualization software.
• Currently resident at Oxford, helping sub-systems
  migrate.
• No surprise, the work load on Joe has increased
• Critical items include Material Integration
  Service, Configuration Utility, Identifiers and
  Transient Model for readout geometry

Important to support such university based
initiatives to core software
28
Incremental Effort Data management

• Our plan has always been to sustain 6.5 FTE
  effort.
• Recent Cuts in 2002
• Ed Frank, U. Chicago
• BNL Hire job offered but retracted due to last
  minute budget cuts
• have impacted our ability to deliver the
  promised
• Unable to save and restore objects from
  persistent event store
• No ATLAS interfaces to Event collections,
  catalogs and metadata
• Approximate allocation of new effort
• 1.0 FTE Collections, Catalogs, and Metadata
  (WBS 2.2.3.5)
• 1.0 FTE Common Data Management Software (WBS
  2.2.3.2)
• 0.5 FTE Event Store (WBS 2.2.3.3)
• Redirect from WBS 2.2.3.1 2.2.3.5 (0.5 each) if
  no funds available.

29
Impact of Insufficient Funds

• -1.0 FTE in Graphics
• Impacts our ability to have any reasonable
  visualization software for test-beam or Data
  Challenge 2.
• - 0.5 FTE in Analysis Tools
• Impacts our ability to deliver a framework for
  analysis
• - 1.0 FTE in Data Management
• 0.5 FTE for supporting Non-Event Data Management.
• 0.5 FTE in supporting basic database services
• - 1.0 FTE in Detector Description
• Jeopardizes our ability to deliver key components
  including Material Service Integration, common
  geometry for simulation and reconstruction,
• - 1.0 FTE in Common Data Management Software
• Impacts contributions to POOL and integration
  aspects, schema management
• - 0.5 FTE in Event Store
• Support for a persistent EDM and Event Selection

30
FY04 Ramp-Up Cost
31
WBS-Personnel Summary
32
Conclusions

• Request for a 5 FTE in FY04
• 2.5 FTE to bring Data Management to its intended
  Level of Effort
• 1 FTE university based for Detector Description
• 0.5 FTE for contribution to Analysis Tools
• 1 FTE university based for support Graphics
• Guidance given for FY04 can handle only 1.5 FTE
• U.S. ATLAS LCG contribution will be 4.0 FTE in
  FY04
• 2.0 FTE each in Core Services and Data Management
  WP