How Much Testing Documentation Is Enough

1
How Much Testing Documentation Is Enough?

• By Armand Mintanciyan

of 22
2
Introduction

• Software testing is the process used to measure
  the quality of developed computer software.
• If it is not documented it did not happen.

3
Introduction (continued)

• Computer validation is a requirement for
  companies regulated by 21 Code of Federal
  Regulations (CFR) Parts 11, 58, 211, 312, 812 and
  820.
• A validated computerized system is aligned with a
  typical Systems Development Life Cycle (SDLC)
  which includes concept, requirements, design,
  development, test, go-live, maintenance, and
  retirement.

4
Tailoring

• Based on the complexity, classification, and
  criticality of an application (system) a
  methodology can be established to combine and
  streamline testing documents (integrated with
  risk management).
• Complexity
• How many man years a project will take.
• Classification
• Category 1 OS, 2 firmware, 3 COTS, 4
  configurable, and 5 customized
• Criticality
• Business non-critical, critical, GxP

5
Typical Testing

• Although testing varies between industries and
  organizations, a typical cycle to testing
• 1. Requirements Analysis Testing should begin in
  the requirements phase of the SDLC.
• During the design phase, testers work with
  developers in determining what aspects of a
  design are testable and under what parameter
  those tests work.

6
Typical Testing (continued)

• 2. Test Planning Test Strategy, Test Plan(s),
  Test Bed creation.
• A lot of activities will be carried out during
  testing, so that a plan is needed.
• 3. Test Development Test Procedures, Test
  Scenarios, Test Cases, Test Scripts to use in
  testing software.

7
Typical Testing (continued)

• 4. Test Execution Testers execute the software
  based on the plans and tests and report any
  errors found to the development team.
• 5. Test Reporting Once testing is completed,
  testers generate metrics and make final reports
  on their test effort and whether or not the
  software tested is ready for release.
• 6. Retesting the defects.

8
Typical Testing (continued)

• A test case (script) is a software testing
  document, which consists of event, action, input,
  output, expected result, and actual result.

9
Requirements Analysis

• Documentation
• User Requirements Specification (URS)
• Business needs
• Functional Requirements Specification (FRS)
• Intended use
• Vendor Audit Report

10
Test Planning

• Documentation
• Validation Plan (QA Plan)
• Test Types
• Unit
• (individual units of source code are working
  properly)
• Integration
• (individual software modules are combined and
  tested as a group)
• User Acceptance
• (confirmation by a business owner that the
  software being tested meets agreed upon
  requirements)
• Systems
• (testing conducted on a complete, integrated
  system to evaluate the system's compliance with
  its specified requirements)
• Validation and Verification
• (Determination of the correctness of the products
  of software development with respect to the user
  needs and requirements)
• Regression
• (Retesting of a previously tested program
  following modification to ensure that faults have
  not been introduced or uncovered as a result of
  the changes made)

11
Test Development

• Design / Build or Configure
• Documentation
• Design Specification
• Test Procedures
• Test Scenarios
• Test Cases
• Test Scripts
• Code Review Checklist

12
Test Development

• Typical Elements in Coding Standards
• Header information
• File naming
• Program description
• Revision record
• Coding style
• Modularity
• Annotation
• Variable definition and equates
• Parameters for interaction

13
Test Execution

• Documentation
• Executed Test Scripts/Test Case from IQ, OQ, PQ
  protocols
• Instructions
• Expected Results
• Actual Results

14
Test Reporting

• Documentation
• Incident Reports
• Test Summary Reports
• Validation Summary Reports
• Go-Live
• Change Controls
• Periodic Review Report
• Retirement
• Decommissioning Report

15
Objective Evidence

• Documentation
• Screen Shots
• Reports

16
Testing OOS

• eCRF would use reusable components. Instead of
  testing every CRF over one would test each
  component and perform integration testing on the
  components.
• Requirement Information
• Information on reusable software products or
  components, which describes the functions and
  capabilities of the reusable entity.
• Information on the reusable entity when
  incorporated into new and evolving products.

17
Testing OOS (continued)

• Design Information
• Design information on reusable software products
  or components.
• Design information on the reusable entity when
  the it is incorporated into new and evolving
  products.

18
Testing OOS (continued)

• Coding
• Code information on the reusable software conform
  to established code management practices.
• Testing Information
• Written test information on the reusable software
  to show that it was fully tested.
• Written test information on the reusable entity
  when incorporated into new and evolving products.
  
• Written test documents for reusable components
  contain test cases with defined inputs, defined
  outputs, observed results, tester ID, and
  recorded errors.

19
Testing SOA

• Service-oriented architecture (SOA) is a strategy
  to build software assets in a company using a
  service-oriented programming methodology.
• Services are software components (e.g. payroll,
  asset management), constructed so that they can
  be easily linked with other software components.

20
Testing SOA (continued)

• Web services are a linking and communications
  methodology.
• SOA is an overall IT strategy.
• The testing strategy mentioned previously all
  still apply.

21
Automated Testing

• HP Quality Center (formerly Mercury Interactive
  Test Director/WinRunner) - manages and governs
  quality processes and automates software testing
  across an application environment.
• Paperless execution screenshots can be attached
  to the test case and the test case can be signed
  electronically with audit trail.

22
Questions

• To contact Armand Mintanciyan, his e-mail address
  is
• armand_at_mintanciyan.com