SEAL Project Overview

1
SEAL Project Overview
ACAT03
1-5 December, 2003 KEK
IX International Workshop on Advanced Computing
and Analysis Techniques in Physics Research
Lorenzo Moneta/ CERN-EP on behalf of the SEAL
team
2
Outline

• Introduction
• Scope and motivation
• SEAL Work Packages
• Description
• Current status
• Future plans and summary

3
Project motivation

• What is SEAL ?
• project in LCG Application Area
• started last year after BluePrint RTAG
  recommendations
• common project on core tools and services
• SEAL Shared Environment for Applications at LHC
• SEAL aims to
• Provide the software infrastructure, basic
  frameworks, libraries and tools that are common
  among the LHC experiments
• Select, integrate, develop and support foundation
  and utility class libraries
• Develop a coherent set of basic framework
  services to facilitate the integration of LCG and
  non - LCG software
• Do not re-invent the wheel
• Try to use as much as possible of existing
  software
• Develop/adapt/generalize to achieve necessary
  level of coherency

4
Project scope

• Foundation Class Libraries
• Basic types (STL, Boost,), utility libraries,
  system isolation libraries, domain specific
  foundation libraries
• Mathematical Library
• Basic mathematical functions, linear algebra,
  fitting and minimization libraries
• Basic Framework Services
• Component model, reflection, plug-in management,
  incident (event) management, distributed
  computing, grid services, scripting

Software Structure
5
Project Work Packages
Foundation Foundation and Utility Libraries and Plug-in Manager
MathLibs Math Libraries Support and Coordination
Dictionary LCG Object Dictionary
Framework Component Model and Basic Framework services
Scripting Scripting Services
Grid Grid Services
Documentation Education and Documentation
Not yet active
6
Foundation and Utility Libraries

• Inventory of existing C utilities
  (http//seal.cern.ch/components.html)
• Recommended classes useful for LCG projects and
  experiments
• Large number from Boost (open source project)
• Support for Boost is provided (in contact with
  developers)
• Develop in SEAL what is missing in STL and Boost
• Have a set of low-level, fairly independent class
  libraries
• ?200 classes originated mainly from classlib
  (CMS)
• Basic abstraction and foundation utilities
  classes
• Timing utilities
• Stream-oriented I/O, compression and
  decompression
• High quality, efficient, well tested and
  documented library
• Recommend best practice to LHC developers

7
Plug-in Manager

• Plug-in Management
• Service in charge of managing, querying,
  unloading plug-ins
• Low level plug-in management (no framework
  constraints)
• Ability to create module libraries containing
  plug-ins
• Ability to instantiate concrete implementations
  knowing the plug-in category and the concrete
  type
• Caching information for fast loading
• what modules contain
• what kind of plug-ins exist and which categories
  
• Imported from CMS IGuana
• Basic concept advanced object factory
• Interfaces for object instantiation and plug-in
  provider
• Dynamic loading completely orthogonal and
  optional!

8
Mathematical Libraries

• GSL (Gnu Scientific Library) evaluation
• Extensive comparison with NagC
• Validation and test suite
• Re-implementation of MINUIT in C
• Prototype available with Migrad and Minos
• Undergoing numerical and performance tests
• Comparison tests with Fortran version
  (compatibility within errors)
• Linear Algebra
• Prototype wrapper around GSL matrices.
• Comparison of various linear algebra packages
  (CLHEP, GSL, uCBLAS)
• Participation in CLHEP project
• Provide to experiments math and statistics
  libraries to be used in analysis, reconstruction
  and simulation.
• support, consultancy, etc..

9
Object Dictionary

• Adding introspection/reflection capabilities to
  C
• Use for persistency and interactivity (e.g.
  Python)
• Modern languages provide reflection inherently
• In C reflection is very limited (RTTI)
• The basic C concepts shall be supported
• e.g. inheritance, methods, data members,
  accessibility, templates
• Seal provides
• Reflection Builder and Reflection (API interface)
  packages
• Dictionary generator to produce dict. source from
  C files
• Uses gcc-xml Python script
• Dictionary for some standard packages (STL,
  CLHEP)
• Used by POOL (persistency)

10
Dictionary Generator

• gcc-xml to parse header-files (extension to gcc
  )
• XML file generated with dictionary information
• Python script to generate dictionary C code
• use an additional XML file to customize class
  selection
• Compiled and loaded at run-time to create
  dictionary
• Provide meta information about the object to the
  user

User
.xml
selection file
lcgdict
.xml
.h
gccxml
filter exten
gendict
par ser
_dict.cpp
..Dict.so
make
(Python script)
.h
.h
.h
Reflection Libraries
Clients
include files
11
Framework Services

• Motivation
• Allow LCG developers and users to write portable
  and modular code
• Provide abstractions for modelling software
  systems
• Component model definition
• Basic Components
• Set of base classes and interfaces needed to
  build more complex components
• Provide standard functionality
• instantiation, configuration, lifetime
  management, etc
• Users (framework developers) should model their
  services using these proposed base classes
• SEAL provide in addition a set of basic services
  implemented following the component model

12
SEAL Component Model
Context
Component
Service

• Hierarchy of bases classes
• Components living in a Context
• Services are Components providing their local
  Context
• User classes inherit from Component or Services
• Plug-in functionality for free

13
Basic Common Framework Services

• Application Service
• Defines the top level Context
• Defines initial set of Components to be loaded in
  the application
• Message Service
• Message composition, filtering and reporting
• Configuration Service
• Management of Component properties and loading
  configurations
• Multiple backends foreseen
• Gaudi style options, .INI style, CMS style, XML,
  ...
• Other services will come later
• Object Whiteboard (Transient Object repository)
• Dictionary Service (Loading Dictionaries in the
  application)

14
Scripting Services

• Scripting needed for Interactivity,
  Configuration, Rapid prototyping, Integration
  (Component bus), etc.
• Main scripting language
• Python

• Define guidelines for developing Python bindings
  
• Various tools available (Boost.Python, SWIG)
• Recommend solutions
• Boost.Python
• LCG Dictionary

15
Scripting Packages

• PyLCGDict
• Python extension module to the LCG dictionary
• Provide access to C libraries from Python, for
  free
• Automatically generates Python proxies for C
  objects using dictionary information
• Everyone can create binding effortlessly
• PyROOT
• Python bindings for ROOT classes
• access to ROOT functionality from Python
• Binding using the ROOT dictionary (CINT)
• Missing gateway ROOT -gt Python

16
Education/Documentation

• Produce documentation
• Reference documentation (generated with doxygen)
• HowTo pages for the released elements
• Plugin Manager, Dictionary generation, PyROOT,
  PyLCGDict, etc.
• Release notes
• Training material (tutorials)
• Help developers incorporating SEAL into the LCG
  projects and the experiment frameworks
• Python courses
• Hands-on introduction course to Python
  programming
• Very positive feedback received

17
Software Process and QA

• Design
• Team design sessions
• Prototypes using Pythons
• Code Review
• No formal code reviews
• Coding done very often in pairs (XP style)
• More than one developer knowledgeable for each
  package
• Testing
• More than 200 SEAL unit tests based on CppUnit
• All are driven and run automatically by QmTest
• Bug reporting and tracking
• Savannah Portal (http//savannah.cern.ch/proj
  ects/seal)

18
Future plans (1)

• Foundation libraries (Plug-in Manager)
• Follow Boost developments
• Little development planned in SEAL (hash-maps)
• MathLibs
• Produce inventory of mathematical functions
• Evolve Minuit C into a final product
• Linear Algebra studies
• Generate dictionary library for Python and CINT
• Dictionary
• Implementation of new reflection model
• Overcome some existing limitations (typedef,)
• Converge to a common dictionary with CINT ?

19
Future plans (2)

• Framework
• Develop Whiteboard (object repository) and
  Dictionary service (loading of dictionary
  libraries on-demand)
• Scripting
• PYLCGDict
• Migrate much of functional core from C to
  Python
• Support more natural Python features (eg iterator
  protocol)
• PyROOT
• Unify core with PyLCGDict
• Develop gateway ROOT -gt Python
• Grid
• New LCG project (ARDA) is starting
• Expect need to interface to some of the ARDA
  services
• Documentation
• Produce tutorials for foundation classes, Boost,
  etc..

20
Summary

• SEAL has delivered basic foundation, utility
  libraries and object dictionary
• The main client have been POOL (persistency
  project) and PI
• Plug-in manager is also used in CMS
• Very positive feedback received for plug-in
  manager
• Currently being integrated into experiments
  frameworks of LHCB (Gaudi) and ATLAS (ATHENA)
• The first version of the Component Model and
  Framework services is available
• Start integrating in POOL and the LHC experiments
• Wait for feedback before developing more services
• Scripting based on Python
• Boost.Python and PyLCGDict recommended for Python
  bindings
• PyLCGDict is used to provide bindings for classes
  of Physics Interfaces (PI) project

21
Conclusions
http//seal.cern.ch/
Acknowledgments Seal team members R.
Chytracek, J. Generowicz, W. Lavrijsen, M.
Marino, P. Mato, S. Roiser, L. Tuura, M. Winkler
22
Additional Slides
23
LCG Application Domain Coverage
24
Plug-in Manager Design

• Originated from CMS Iguana

25
Reflection System

• Reflection Builder the loading interface
• Reflection the user API

26
LCG Object Dictionary Usage
.
adl
.h
.xml
Population
ROOTCINT
GCC
-
XML
ADL/GOD
CINT generated
Dict generating
code
code
in
(1)
Conversion
LCG
CINT
LCG to CINT
Dict gateway
Dictionary
Streamer
Dict
ROOT I/O
(2)
out
Reflection
27
Seal Component Model

• Hierarchy of bases classes
• Each Component lives in a Context
• Contexts form their own hierarchy
• Support for component look-up
• A Service provides its own local Context
• Components of a Service live in the scope defined
  by its own Context
• User classes inherit from Component or Service
• Plug-in functionality
• for free

1
1