Software Quality Assurance

1
Software Quality Assurance

• Chip Martin
• Defense Nuclear Facilities Safety Board

2
Topics

• Background
• Board Actions to Date
• Observations
• Standards Guidance
• Key Processes
• Future Evolutions

3
Background

• Software is used by DOE and its contractors to
  analyze hazards, to design effective controls,
  and for automatic control of safety systems
• As a result, the safety posture of many
  facilities is strongly dependent on the quality
  of this underlying body of analysis, design, and
  control software

4
What is Different about Software

• Other engineering projects are governed by laws
  of physics and material properties, but software
  engineers work with abstract materials with no
  natural limits
• Software errors can be hard to find implying a
  need for a rigorous, well documented process,
  particularly for development of safety-related
  software

5
TECH-25

• In January 2000, DNFSB/TECH-25 identified
  numerous deficiencies in safety-related software
  at DOE the Board asked for a corrective action
  plan (CAP)
• On October 3, 2000, the Board received the CAP
  but found it was not responsive to the Boards
  concerns
• Board has never received a revised plan

6
Board/Staff Actions to Date

• The Board held three public meetings on QA with
  SQA as a special interest item
• The staff has reviewed directives and guidance
  from other agencies as well as industry consensus
  standards and visited the NASA Independent
  Verification and Validation Center
• The staff completed several on-site reviews of
  SQA implementation at Y-12, SNL, Hanford, Pantex,
  and SRS

7
Key Observations

• Expectations for SQA in DOE directives and
  guidance are not clearly set
• Responsibility and authority for SQA functions
  within DOE are not adequately defined, nor is
  there an effective champion

8
Key Observations (cont.)

• There is no formal DOE training program for SQA
• Contractor implementing procedures do not have
  sufficient detail to define a process which will
  ensure quality software products

9
Non-Nuclear Software Safety Standards

• NASA NASA-STD-8719.13A, Software Safety and NASA
  GB-1740.13-96, Guidebook for safety-critical
  systems
• DoD MIL-STD-882B, Mishap Risk Management
• U.K. Ministry of Defence DEF STAN 00-55,
  Requirements for safety-related software
• Joint Services Software Safety Committee, G-48
  Committee, Software System Safety Handbook
• NATO Standardization Agreement, STANAG 4404
  Safety design requirements and guidelines for
  computing systems

10
Nuclear Industry Software Safety Standards

• International Electrotechnical Commission, IEC
  608801986-09, Software in Computers in Safety
  Systems of Nuclear Power Stations
• CE-1001-STD Rev.1, Standard for Software
  Engineering of Safety Critical Software.
• IEEE STD 7-4.3.2-1993, Computers in Safety
  systems

11
General Software Engineering Standards
12
Software Safety Process

• Acceptance Test
• System Test
• Integration Test
• Unit Test

• Concept of Ops
• Software Requirements Specification
• Hi-Level Design
• Low-Level Design

Safety Assessment Should Occur in All Phases
13
Tailoring the Software Safety Effort

• Identify safety critical software
• Determine how critical the software is
  (classification)
• Determine the extent of the development effort
  and oversight required 

14
Software Requirements Phase
15
Architectural or Preliminary Design
16
Detailed Design
17
Software Implementation
18
Software Integration and Test
19
Software Maintenance
Same Activities as in Development
20
Future Evolutions

• DOE is preparing a QA Improvement Plan (Ray
  Hardwick)
• Boards staff is drafting a report on SQA
  standards
• Potential need for ANSI/ANS 10.4 revisions and
  ASME NQA-1 Subpart 2.7 revisions?

21
Questions