Use of CMT in LHCb

1
Use of CMT in LHCb
CMT Workshop, LAL (Orsay) 28th February - 1st
March 2002 P. Mato / CERN
2
Contents

• History
• Package Categories
• Release Practices
• Useful Tools
• Current Problems

3
History

• We started using CMT in LHCb before CMT was
  called CMT (Methods)
• All the new C code (Gaudi) developed using CMT
• Also adopted afterwards for the legacy FORTRAN
  code
• Currently order of 100 packages organized
  hierarchically (hat/packagename)
• Used on Linux and Windows since the beginning
• Used directly by the physicists and librarians

4
Packages Categories

• Program is a package that contains a main
  routine and a list of dependent packages needed
  to link it.
• Library contains a list of classes and the list
  of dependent packages needed to compile it.
• Policy package defines macros, compilation
  link flags, patterns, fragments, etc.
• Package group contains a list of other packages
  with their version number (e.g. GaudiSys)
• Interface package interfacing to packages not
  managed with CMT (e.g. CERNLIB, CLHEP, ROOT,)

5
Policy Package (GaudiPolicy)

• Definition of tags
• Standard configurations
• General macros
• Compilation and link flags
• Patterns
• Simple and uniform requirements files
• Naming conventions, standardization,
• Fragments
• DevStudio, RuleChecker, etc.

6
Interface Packages
ExternalLibs
Version Platform Compiler
CLHEP
CMTSITE

• Set of environment variables to define where the
  external packages are installedXXX_DIR

• Compilation and link options
• Environment variablesPATH, LD_LIBRARY_PATH
• Path relative to XXX_DIR

7
Interface Package (contd)
package CLHEPversion v17r10branches doc
cmt include_path none use ExternalLibs v3 set
LHCXX_BASE "LHCXX_DIR" \ Linux-2.95.2
"LHCXX_DIR/gcc-2.95.2" include_dirs
(CLHEP_DIR)/include set CLHEP_DIR
"LHCXX_BASE/CLHEP/1.7.1.0" macro
CLHEP_linkopts "-LCLHEP_DIR/lib -lCLHEP" \
LinuxStatic "CLHEP_DIR/lib/libCLHEP.a"
\ WIN32 "CLHEP_DIR/lib/CLHEP.li
b" path_remove LD_LIBRARY_PATH "/CLHEP/" \
WIN32 "\CLHEP\" path_append
LD_LIBRARY_PATH "CLHEP_DIR/lib"
8
Package Group
package GaudiSys version v9r0 ---CMT and Gaudi
policy use GaudiPolicy v5r ---Gaudi libraries
- generic use GaudiKernel v11r use GaudiSvc
v7r use GaudiAud v5r use GaudiAlg
v5r use GaudiTools v5r use GaudiNagC
v6r use GaudiDb v5r use
GaudiIntrospection v2r --- External
libraries use CLHEP v17r10 use HTL
v13r10 use AIDA v1r0 use
ExternalLibs v3r

• Useful to fix the versions of a number of
  packages
• To be used directly by users
• For management purposes
• Release tools
• broadcast command

9
Use Practices

• CMTCONFIG
• Default configuration tag
• If tag not specified then use CMTCONFIG as tag
• .cmtrc
• The list of official release areas
• CMTPATH
• HOME/mycmt eventually DEV areas
• Using both optimized and debug versions

gt getpack MyPackage v1r1gt cd MyPackage/v1r1/cmtgt
make tagCMTDEBgt source setup.csh
tagCMTDEBgt
10
Release Practices

• Complete LHCb software release 3-4 times a year
• Major new functionality
• Incompatible changes allowed
• Changes in external packages versions
  dependencies
• Change in the major version number
• Partial/Incremental releases when convenient
• Bug fixes
• New functionality (changes in Algorithms/Services)
  
• Major emphasis on source code/binary
  compatibility
• Change in the minor version number

11
Integration and Testing new releases
Public DEV Area
Public RELEASE Area
Private Area
Private Area
Private Area

• We do not have a nightly build system
• Integration Area (DEV)
• Quasi-continuous builds of head revision of the
  packages involved in the integration
• Release by Layers
• One after the other

Applications
Algorithms
Event/Detector
Framework (Gaudi)
12
Extra Tools DevStudio add-in

• Integration of CMT in DevStudio
• Regeneration of DevStudio project
• Set environment
• Show uses
• Using exclusively the cmt.exe interface

13
Extra Tools GetPack

• Python script to checkout/update packages in the
  LHCb CVS repositories
• Easy to use (combines CVS and CMT commands)
• Platform independent
• We do not use cmt co command
• Unpredictable result
• Can not handle several repositories

usage getpack OPTIONS... package version
'tag''head' getpack OPTIONS... -i
'Gaudi' hat 'LHCb' hat
14
Extra Tools

• Scripts for the creation of tar files for
  distributions
• Prototype of a script for Grouping a number of
  packages into a single one
• Useful for using the package

• Improves scalability
• Reduces the length of PATHs and commands
• Not useful for package development

15
Current Problems

• Not able to use new CMT version
• Our current practices do not work ? big impact to
  the end-users
• Need to understand new private use feature
• Impact on shareable libraries and environment
  variables
• Problems with soft links. Introduced to reduce
  the length of PATH and LD_LIBRARY_PATH
• Not re-built when something has changed
• Not re-built on source setup
• No not work on Windows!

16
Current Problems (contd)

• Care with handling wildcards in version numbers
• Gained experience but still making errors time to
  time
• Missing standard Interface packages
• Difficult to integrate other software systems
  following different conventions

17
Summary

• Using CMT since the beginning
• Has taken some time to develop the current
  practices (package categories, use version
  wildcards, etc.)
• Additional tools developed around CMT are
  necessary
• Major concerns in future compatibility