Producing software fitforpurpose

1
Producing software fit-for-purpose

• Graeme Parkin
• graeme.parkin_at_npl.co.uk
• Mathematics and Scientific Computing Group
• Division of Enabling Metrology

2
Aims

• Need software that is fit-for-purpose
• How to validate software to show it is
  fit-for-purpose
• How to certify software has having been validated

3
Contents

• Introduction to the process based on Best
  Practice Guide No 1
• Failure of software
• Scientific software
• Software problem
• Software fit for purpose
• Further information
• Example
• Relationship to IEC 61508
• Issues on certifying software to the Guide
• Summary

4
Failure of software

• Ariane 5 - The internal SRI software exception
  was caused during execution of a data conversion
  from 64-bit floating point to 16-bit signed
  integer value.

PhotoCNN (http//www.cnn.com/WORLD/9606/04/rocket
.explode/)
5
Failure of software continued

• Saturns moon Titan - But a single line of
  software code omitted from the craft's onboard
  communications system resulted in the failure of
  one of its two channels of communication, meaning
  data on only one channel was radioed back to
  Cassini.

Photo ESA/NASA/JPL/University of Arizona Image
First images from Titan
6
Scientific software
Validate software with respect to the selected
model and algorithms
7
Software problem

• Producing software fit for purpose
• Problem software cannot in general be shown to
  be error free
• Solution use development techniques that produce
  software of suitable quality
• Software validation is the process of ensuring
  that the appropriate techniques have been used
  and used correctly

8
Software fit for purpose

• Three steps in our approach
• A risk assessment, the purpose of which is to
  make an objective assessment of the likely risks
  associated with a software error.
• Assigning a Measurement Software Level (MSL)
  indicated by the results of the above risk
  assessment.
• Applying software validation techniques indicated
  by the assigned MSL.

9
Software fit for purpose continued

• The assessment considers the risk factors of
• Legal requirements
• Criticality of usage
• Impact of complexity of control
• Complexity of data processing.

10
Further information

• The whole process is contained in (freely
  available)
• Software Support for Metrology, Best Practice
  Guide No. 1, Validation of Software in
  Measurement Systems,Brian Wichmann, Graeme
  Parkin and Robin Barker, March 2004, Version
  2.1http//www.npl.co.uk/ssfm/download/documents/s
  sfmbpg1.pdf

11
Example

• Software to evaluate surface roughness parameters
• Software environment
• Reference software
• Platform independent
• Easily accessible

12
Example - functionality

• Functionality of software to be developed
  consisted of
• Apply a Gaussian filter with cut-off wavelength
  ?c to data representing a primary (P) profile to
  obtain data representing waviness (W) and
  roughness (R) profiles.
• Evaluate surface texture roughness parameters Ra,
  Rq, Rsk, Rku, Rp, Rv, Rz, RSm, Rc, Rt and their
  waviness (W) and primary (P) profile equivalents.
• Read in SMD format (surface measurement data, ISO
  5436-22001(E))

13
Example the RSm parameter
Spacing parameter RSm for a roughness profile
14
Example risk analysis

• Risk analysis
• No legal requirements
• Business critical
• Simple complexity of control
• Moderate complexity of processing

Taking into account issues like ease of testing
15
Example selection of techniques and tools
16
Example other tools
17
Relationship to IEC 61508

• IEC 61508 standard for safety systems
• Part 3 for safety critical software
• The 61508 Association, members include
• Sira, Baseefa
• Moore Industries Inc, MTL
• British Energy Generation Ltd
• Observers HSE
• NPL is working with The 61508 association on the
  application of IEC 61508-3

18
Relationship to IEC 61508 continued

• The Guide
• Provides support to meet Part 3 of IEC 61508
• Includes a mapping from the guide to the elements
  of IEC 61508
• Includes checklists used to generate the
  questionnaire for auditing
• Is being considered by The 61508 Association in
  the development of certification methodology

19
Issues on certifying software to the Guide

• HSE (Health and Safety Executive in UK) with
  respect to IEC 61508 do not like the use of
  complexity to select the validation techniques
  would rather the techniques are used to determine
  the complexity of the software
• How can we gain acceptance of the Guide as the
  basis of an certification service?
• What is sufficient evidence for auditing
  purposes?
• Who will accredit a certification service based
  on the Guide?

20
Summary

• Shown
• Need to validate software as fit-for-purpose
• How software can be validated using the Guide
• Progress towards setting up a certification
  service based on the Guide