DOE O 414'1C, Quality Assurance Improving Safety Software Quality

1
DOE O 414.1C, Quality Assurance Improving
Safety Software Quality

• Frank Russo
• Deputy Assistant Secretary
• Corporate Performance Assessment (EH-3)

2
Welcome

3
Video Conference Ground Rules

• Mute remote locations
• Silence cell phones
• Hold questions until the end
• Allow us to defer your question if it will be
  covered during the FAQs

4
DOE O 414.1C General Information Video Conference
Agenda

• Welcome (F. Russo)
• DOE Expectations for Safety and Quality (R.
  Shearer)
• Background - Order and Guide Development (F.
  Russo)
• Safety Software Quality Responsibilities (F.
  Russo)
• Safety Software Requirements (B. Danielson)
• Field Experiences (C. Lagdon)
• PSO Perspective (L. Dever, E. Schmidt, R. Singh,
  J. Poppiti, M. Janaskie)
• Path Forward (R. Loesch)
• QA (D. Sparkman, C. Lagdon, C. Thayer)

5
DOE Expectations for Safety and Quality

• Russell Shearer
• Principle Deputy Assistant Secretary
• Environment, Safety and Health

6
Background

7
Background

• Importance of Software Quality Assurance as part
  of a Quality Assurance Program
• New and revised Quality Assurance requirements in
  DOE directives
• Majority of new requirements are for software
  quality assurance
• First phase of a Department-wide rollout

8
Order and Guide Development

• DOE EH-31 responsible for QA Order requirements
• Surveyed applicable industry standards and
  guidance
• Consulted DOE organizations, field staff, and
  SMEs
• Working groups established for development of
  Guides
• Wide spectrum of individuals
• Selected based upon DOE organization, site
  location, and SME knowledge
• Members authored sections in Guides
• Members were SME reviewers for all sections

9
Informal and Formal Review Processes

• Informal Reviews
• SME Reviewers
• Defense Nuclear Facility Safety Board
• Formal Process
• RevCom
• Hundreds of valuable comments

10
Thank You

• DOE G 414.1-2A Quality Assurance Writing Team

11
Thank You

• DOE G 414.1-4 Safety Software Quality Assurance
  Writing Team

12
Roles and Responsibilities

13
Order Implementation Roles and Responsibilities

• EH roles and responsibilities
• DOEs independent element responsible for safety
  of the public, worker and environment
• Develops and maintains QA policy requirements,
  guides, and standards for all DOE work
• Provides advise and assistance to DOE elements
  and contractors implementing DOE QA policy
• Identifies and proposes resolutions for
  crosscutting QA issues
• Manages DOE Safety Software Central Registry

14
Order Implementation Roles and Responsibilities
continued

• PSO roles and responsibilities
• Ensure HQ, field elements, and contractors
  implement the requirements in the QA Order
• Set priorities and schedule for compliance
• Provide direction and resources, including
  prioritization, for implementing the QA
  requirements
• Review and approve QAPs or other documents that
  include implementation approaches for the QA
  requirements

15
Safety Software Requirements

• Gustave E. (Bud) Danielson, Jr.
• Quality Policy Manager
• Office of Quality Assurance Programs

16
Significant Changes to DOE O 414.1C

• Addition of SQA for safety software
• Cancels DOE N 411.1, SQA Functions,
  Responsibilities and Authorities
• Reflect existing requirements in 10 CFR 830
• Order references 1 updated and 2 new Guides
• DOE G 414.1-2A, QA Management System
• DOE G 414.1-3, Suspect/Counterfeit Items
• DOE G 414.1-4, Safety Software
• Inclusion of aviation safety into Corrective
  Action Management Program (CAMP)

17
Significant Changes to DOE G 414.1-2A Quality
Management

• Strengthened the use of a single management
  system or work process for similar requirements
• Incorporated review guidance for use by DOE in
  evaluating contractor quality management systems
• Discussed the use of third party management
  system validation
• New generic QAP assessment plan

18
Significant Changes to DOE G 414.1-2A Quality
Management continued

• Updated information pertaining to the
  identification and control of suspect/counterfeit
  items (S/CIs) and links to expanded guidance for
  S/CIs
• Expanded information on the grading process to
  include programmatic and mission-critical
  considerations and a description of the steps in
  implementing the grading process
• Expanded the description of identification,
  tracking, and resolution of quality problems
• Clarified the guidance on procurement processes
  for nuclear safety applications

19
New Guidance with DOE G 414.1-3 Suspect
Counterfeit Items

• Issued November 3, 2004
• Incorporates the new complex-wide S/CI Process
• Updates S/CI information and resources
• Supersedes DOE G 440.1-6, Suspect Counterfeit
  Items Guide for use with DOE O 440.1, Worker
  Protection Management 10 CFR 830.120 and DOE O
  5700.6C, Quality Assurance

20
Safety Software Changes to DOE O 414.1C

• Scope of 10 CFR 830 and DOE O 414.1B Included
  software related to nuclear (including
  radiological) facilities
• Establishes specific category of software, SAFETY
  SOFTWARE
• Identification of Safety Software definitions,
  responsibilities and requirements

21
Safety Software Definitions

• Safety System Software Software for a nuclear
  facility that performs a safety function as part
  of a structure, system, or component and is cited
  in either (a) a DOE approved documented safety
  analysis or (b) an approved hazard analysis per
  DOE P 450.4, Safety Management System Policy,
  dated 10-15-96, and the DEAR clause.

22
Safety Software Definitions continued

• Safety and Hazard Analysis Software and Design
  Software Software that is used to classify,
  design, or analyze nuclear facilities.  This
  software is not part of a structure, system, or
  component (SSC) but helps to ensure the proper
  accident or hazards analysis of nuclear
  facilities or an SSC that performs a safety
  function.

23
Safety Software Definitions continued

• Safety Management and Administrative Controls
  Software Software that performs a hazard control
  function in support of nuclear facility or
  radiological safety management programs or
  technical safety requirements or other software
  that performs a control function necessary to
  provide adequate protection from nuclear facility
  or radiological hazards. This software supports
  eliminating, limiting, or mitigating nuclear
  hazards to workers, the public, or the
  environment as addressed in 10 CFR 830, 10 CFR
  835, and the DEAR ISMS clause.

24
Significant Changes to DOE O 414.1C continued

• Invokes ASME NQA-1-2000,supplemented by other
  consensus standards or other consensus standards
  that provide an equivalent level of SQA
  requirements
• Sites define grading levels and consensus
  standards in DOE approved QAP or other
  appropriate DOE approved document
• Using the grading levels, select and implement
  the applicable software quality assurance work
  activities defined in the Order
• Identification, documentation, and maintenance of
  safety software inventory

25
Significant Changes to DOE O 414.1C continued

• Identifies 10 SQA work activities
• Software project management
• Software risk management
• Software configuration management
• Procurement and supplier management
• Software requirements identification and
  management
• Software design and implementation
• Software safety
• Verification and validation
• Problem reporting and corrective action
• Training of personnel in the design, development,
  use and evaluation of safety software

26
New Guidance in DOE G 414.1-4 Safety Software

• Facilitates implementation of SQA
  responsibilities and requirements
• Detail guidance for each of the10 SQA work
  activities for safety software
• Procedures for updating, adding and removing tool
  box codes from Central Registry
• Cross references from ASME NQA-1-2000 to
• 10 CFR 830 and DOE O 414.1C
• Sample Criteria Review and Approach Document for
  DOE O 414.1C

27
Impact

• Update and maintain an inventory of safety
  software
• NA and EM have initial lists
• Grade safety software
• Many sites institutional SQA programs include
  graded approach
• Implement 10 SQA work activities
• Basic SQA practices
• Consistent with consensus standards
• Map very closely to most sites institutional SQA
  program practices

28
Field Experiences

• Chip Lagdon
• Chief of Nuclear Safety (Acting)Energy, Science
  and Environment

29
Lessons Learned

• Software Requirement Specification (SRS) and
  Software Design Document (SDD) are essential for
  developing quality software and life cycle
  maintenance.
• Majority of software projects did not have SRSs
  and SDDs
• Sites using the SRSs and SDDs have clear
  understanding of what was needed to develop and
  maintain software quality.
• The sites without SRSs and SDDs appeared to be
  relying heavily on the available experts to
  ensure software is developed or procured to meet
  the project needs.
• Related SQA Work Activities
• Software requirements identification and
  management (5)
• Software design and implementation (6)
• Verification and validation (8)

30
Lessons Learned continued

• Software procurement specifications should
  specify details of software requirements, not
  just catalog data.
• Sites procuring PLCs for process systems only
  specified the vendors catalog model information
  as procurement specifications
• Supporting documentation for the suitability and
  applicability of the technical requirements not
  included
• Related SQA Work Activities
• Software requirements identification and
  management (5)

31
Lessons Learned continued

• Formal procedures for software problem reporting
  and corrective actions for software errors and
  failures need to be maintained and rigorously
  implemented.
• Many sites resolve software errors and corrective
  actions at the project level and maintain
  informal coordination with vendors or other
  effected entities.
• Related SQA Work Activities
• Procurement and supplier management (4)
• Problem reporting and corrective action (9)

32
Lessons Learned continued

• Appropriate qualifications and training on
  software use is essential for proper use of
  safety software.
• Very sophisticated and complex software are being
  used without appropriate training in their use.
• Related SQA Work Activities
• Training of personnel in design, development, use
  and evaluation of safety software (10)

33
Lessons Learned continued

• Appropriate software control and configuration
  management are essential for safe use of the
  software.
• Lack of proper control has resulted in multiple
  versions being available at the same time and
  even some with known errors.
• Deficiencies have been noted with configuration
  control in terms of software version and
  documentation.
• Related SQA Work Activities
• Software configuration management (3)

34
SC Perspective

• Leah Dever
• Associate Director for Laboratory Operations
• Office of Science

35
NNSA Weapons Perspective

• Ed Schmidt
• Director
• Office of Nuclear Weapon Surety and Quality

36
NNSA Perspective

• Rabi Singh
• NNSA QA Roadmap Leader
• National Nuclear Security Administration

37
EM Perspective

• James Poppiti
• Environmental Management

38
NE Perspective

• Mark Janaskie
• Office of Integrated Safety and Project
  Management
• Nuclear Energy, Science and Technology

39
Path Forward

• Robert Loesch
• Director (Acting)
• Office of Quality Assurance (EH-31)

40
Commitment and Accountability

• The Department is committed to implementing SQA
  requirements as part of its overall Quality
  Assurance Program
• We have worked extensively with NNSA (NA-121
  NA-124) and EM
• Directives have been revised to reflect the SQA
  requirements along with roles and
  responsibilities
• Central Registry established for toolbox codes
• http//www.eh.doe.gov/sqa/central_registry.htm
• QA and SQA web sites along with SQA List Server
  established to share information and solicit
  feedback

41
Upcoming Activities

• Toolbox code upgrades
• Other directive revisions
• Minor changes to include reference to revised QA
  Order and new SQA Guide
• QA and SQA Qualification Standards Update
• Order Roll Out
• Regional training and support
• Site visits upon request
• Newsletter series on 10 work activities
• Safety software assessment support

42
Resources

• EH QA Web Site
• http//www.eh.doe.gov/qa/
• EH SQA Knowledge Portal
• http//www.eh.doe.gov/sqa/
• SQA List Server
• CENTREG_at_VM1.HQADMIN.DOE.GOV
• EH S/CI Web Site
• http//www.eh.doe.gov/sci/

43
Resources continued

• EH Staff
• QA - Bud Danielson 301-903-2954
• bud.danielson_at_eh.doe.gov
• SQA - Debra Sparkman 301-903-6888
• debra.sparkman_at_eh.doe.gov
• S/CI - Rick Green 301-903-7709
• rick.green_at_eh.doe.gov

44
FAQs

• Debra Sparkman,
• Software Quality Engineer
• Gustave (Bud) Danielson, Jr.
• Quality Policy Manager
• Office of Quality Assurance Programs

45
FAQs

• 1. Why does the safety software definition
  include references to 10 CFR 835, DOE P 450.4,
  and the DEAR ISMS clause?
• 2. How were the 10 work activities determined?
• 3. Our site currently does not use NQA-1-2000.
  Will this be a big change for our programs?

46
FAQs continued

• 4. Our sites SQA program is based on 10-CFR-830
  , ASME NQA-1 2000, QC1, RW0333P, and DOE Orders.
  Our SQA / QA program and implementing procedures
  cover all software. Can we continue to use our
   grading levels if they are different from those
  suggested in the Guide?
• 5. Facility design software used by a DOE
  contractor may be graded differently than the
  same software used by a supplier of design
  services to the DOE contractor. Why does DOE G
  414.1-4 recommend different grading of the work
  activities?

47
FAQs continued

• 6. The Order is silent on software quality
  requirements for "non-safety software".  What
  software quality standards are required for
  "non-safety software"?
• 7. How do the safety software requirements in DOE
  O 414,1C apply to DOE weapons related work?
• 8. How do the safety software requirements in DOE
  O 414.1C differ from those in QC-1?

48
FAQs continued

• 9. Are there any changes in the way software
  users will be contacted on software bugs or
  major issues, especially with respect to
  software used by many contractors?
• 10. Is there a centralize list of safety analysis
  software used by DOE contractors?
• 11. How does the graded approach apply to safety
  software? Can you provide examples?

49
FAQs continued

• 12. Can a developer or contractor submit
  software to DOE to be considered a toolbox code
  and included in the Central Registry?
• 13. When will the contractor be required to
  comply with DOE O 414.1C?