Trands in Computing

1
F. P. Brooks, No Silver Bullet - Essence and
Accidents of Software Engineering, IEEE Computer
20(4)10-19, April, 1987
As we look to the horizon of a decade hence, we
see no silver bullet. There is no single
development, in either technology or in
management technique, that by itself promises
even one order-of-magnitude improvement in
productivity, in reliability, in simplicity.
... Not only are there no silver bullets now in
view, the very nature of software makes it
unlikely that there will be any - no inventions
that will do for software productivity,
reliability, and simplicity what electronics,
transistors, and large-scale integration did for
computer hardware. We cannot expect ever to see
twofold gains every two years. ... Although we
see no startling breakthroughs - and indeed, I
believe such to be inconsistent with the nature
of software - many encouraging innovations are
under way. A disciplined, consistent effort to
develop, propagate, and exploit these innovations
should indeed yield an order-of-magnitude
improvement. There is no royal road, but there is
a road.
2
Experience with Software Process in Physics
Experiments

• S. Guatelli, B. Mascialino, L. Moneta,
• I. Papadopoulos, A. Pfeiffer, M.G. Pia,
• M. Piergentili
• CERN - INFN Genova
• IEEE Nuclear Science Symposium 2004
• Roma, 21 October 2004

3
Challenges
Mission critical applications
Complexity of physics and experiments
Geographically distributed teams and computing
resources
Quality and reliability for sensitive applications
4
Trends in software engineering
Shift attention from computing technology to
software process as the way to achieve quality,
lower costs
Develop a discipline for software engineering
Define quantitative standards to evaluate
software capability maturity SEIs Software
Capability Maturity Model ISO 15504

• The culture of software process is not widely
  spread in the physics research environment yet
• Software is still perceived as writing code
• The success of software projects is mainly left
  to individual efforts (CMM heroic, level 1)

5
Process framework
Based on the Unified Process (Booch, Jacobson
and Rumbaugh) Rational Unified Process UP
practical guidance and tools
Customized to the specific development environment
Equivalent to CMM / ISO 15504 Level 3 at least
A bi-dimensional process
An incremental and iterative software life-cycle
Widely used in software industry
The first (only?) application of the RUP in
HEP/nuclear/medical/ astrophysics research
6
Projects adopting the RUP

• Pilot project Anaphe (Data Analysis Tools
  project, CERN/IT Division)
• a playground to learn the RUP and to adapt it to
  the physics research environment
• RUP in projects
• Geant4 Low Energy Electromagnetic Physics
• Geant4 Electromagnetic Physics Validation Project
• Statistical Toolkit
• Brachytherapy Simulation Dosimetry
• IMRT Geant4-based Dosimetry System
• REMSIM Simulation (radioprotection, Aurora
  Programme)
• Bepi Colombo Simulation (planetary astrophysics)
• Solar System Radiation Environment Generator
• RUP in an organization (encompassing several
  projects)
• Geant4 Advanced Examples (in progress)
• Experience of application
• Collection of metrics and analysis for software
  process improvement

7
Adapt the process to the project

• Do what you need, dont do what you dont need
• not an excuse to neglect some good practices
• but an optimisation of the best practices in your
  own specific environment
• distinguish critical process needs from wish list
• Learn with experience
• and with quantitative process metrics
• Software process improvement
• a continuous process to improve the process
• based on metrics collected
• Specific features of the research environment
  w.r.t. commercial software
• users often coincide with developers
• volunteer work, no control on the software
  organization
• project management complemented by guidelines
  from E.I. studies

8
Guidelines for process tailoring

• Retain all the best practices suggested by the
  RUP
• Iterative development
• Management of the requirements
• Component-based architecture
• Visual modeling with UML
• Continuously verify quality
• Change management
• Identify essential activities, artifacts and key
  roles
• for every RUP activity is it justified in our
  environment? What are the benefits? How much does
  it cost?
• Justify all artifacts to be produced in relation
  to the project objectives

9
Essential artifacts
A minimal subset identified within the large set
of RUP artifacts
Vision Risk List Development Plan Development
Case CVS repository
User Requirements Architecture Model Design
Model Test Plan Test Suite Prototype Tools Guideli
nes
The system Documentation
The product Release Notes Training Material
refinement of artifacts from previous phases
refinement of artifacts from previous phases
Other optional artifacts may be produced in some
projects
refinement of artifacts from previous phase
much more than just the code!
10
Metrics

• Quantitative process control
• Objective evaluations are the key to software
  process improvement

Test Number of unit/system tests
Test Fraction of code covered by unit testing
Test of defects found by unit testing
Test of defects found by integration testing
Test of defects found by system testing
Test Fraction of deliverables reviewed 
Test Number of defects found by reviews
Test Defect status
Test Defect recurrence
Test Defect source count
Project management Number of tasks planned and completed
Project management Number of reviews, walkthroughs
Project management Staff size and profile
Project management Staff turnover
Support Number of support requests
Support Support request aging
Support Average resolution time
Support Re-opened support requests
Requirements Number or user requirements / use cases
Requirements Requirement status 
Requirements Traceability
Analysis Design Number of subsystems, packages, classes
Analysis Design Number of methods in a class
Analysis Design Inheritance tree depth
Implementation Code size
Implementation Method size
Implementation Fraction of design implemented
Deployment New features implemented per release
Deployment Released defect levels
Deployment Documentation/examples coverage
Schedule Estimated vs. actual task duration
Schedule Estimated vs. actual duration between milestones
Schedule Schedule estimating accuracy
Process Process capability level
Process Process deviations
Process Number of metrics collected
11
Some metrics

• 95 projects delivered on time, on budget and
  with all the features originally planned
• CHAOS Report 2000 (Standish Group)
• 28 software projects failed
• 49 challenged
• 23 successful
• 5 failure due to neglect of applying the process
• no way to enforce the application of the process
  onto independent researchers
• 5 of the papers accepted at Monte Carlo 2005
  comes from one small team (adopting the RUP),
  representing lt 0.5 of the active community
• productivity gt 10 higher than average

12
A case study

• Development of simulation analysis of a
  dosimetric system for IMRT
• Software requirements functionality high
  quality for medical application
• Master thesis work, duration 1 year
• Manpower 1 developer 1 process specialist
• Developer no prior software knowledge

• Results
• on-time delivery of code, documentation, physics
  results
• all high and medium priority requirements
  satisfied
• 30 billion simulation events produced and
  analysed
• high quality physics results from the software
  (paper submitted to IEEE-NSS 2004)
• 355 page MSc. thesis
• fraction of code discarded during the development
  period 0

D 0.005 p-value 1
13
Conclusion

• A rigorous software process is the key technology
  enabling development teams to achieve success
• The RUP can be effectively adapted to the
  peculiar physics research environment
• a powerful, but flexible process framework
• A RUP-based software process has been adopted by
  several physics projects in may research fields
  (fundamental physics, space science,
  astrophysics, medical physics, statistics)
• Success rate is 95 (to be compared to CHAOS 23)