Software Test Documentation IEEE Std 8291998

1
Software Test DocumentationIEEE Std 829-1998

• H. David Le
• EEL 6883
• Software Engineer II
• Feb 16 2004

2
1. Scope

• Standard for basic documentation for dynamic
  aspect of software testing as prescribed by IEEE.
  It defines purpose, outline, and content for
  each basic documentation.

3
2. References and 3. Definitions

• Can be found in Centralized IEEE Software
  Engineering Standards Glossary.

4
4. Test Plan

• Should provide
• Scope
• Approach
• Resources
• Scheduling

5
4. Test Plan (cont.)

• It should identify
• Item being tested
• Features being tested
• Testing task to be performed
• Personnel responsible for each task
• Risk associated with the plan

6
4.2 Outline

• Test plan identifier
• Introduction
• Test items
• Features to be tested
• Features not to be tested
• Approach
• Item pass/fail criteria
• Suspension criteria and resumption requirements
• Test Deliverables

7
4.2 Outline (Cont.)

• Testing tasks
• Environmental needs
• Responsibilities
• Staffing and training
• Schedules
• Risk and contingencies
• Approvals

8
4.2.1 Test plan identifier

• Specify the unique identifier assigned to the
  test plan

9
4.2.2 Introduction

• Summarize the software items and features to be
  tested
• Project authorization
• Project plan
• Quality assurance plan
• Relevant policies
• Relevant standards

10
4.2.3 Test items

• Identify the test items including the
  version/revision level
• Requirements specification
• Design Specification
• User guide
• Operation guide
• Installation guide

11
4.2.4 Features to be tested

• Identify all software features, and combinations
  of software features to be tested.

12
4.2.5 Features not to be tested

• Identify all software features, and combinations
  of software features not to be tested and the
  reason

13
4.2.6 Approach

• Describe the overall approach to be tested
• Groups of features or combination
• Activities
• Techniques
• Tools

14
4.2.7 Item pass/fail criteria

• Specify the criteria to be used to determine
  pass/fail testing

15
4.2.8 Suspension criteria and resumption
requirements

• Specify the criteria used to suspend all or
  portion of the testing activities

16
4.2.9 Test Deliverables

• Documentation
• Test plan
• Test design specification
• Test case specification
• Test procedure specification
• Test item transmittal reports
• Test logs
• Test incident reports
• Test summary reports

17
4.2.10 Testing tasks

• Identify the set of tasks necessary to prepare
  for and performing testing.

18
4.2.11 Environmental needs

• Specify the necessary and desired properties of
  the test environment.
• Hardware
• Communication and system software
• Mode of usage (e.g. stand alone)
• Other software or supplies need to perform
  testing
• Level of security
• Special tools needed

19
4.2.12 Responsibilities

• Identify following groups responsible
• Managing
• Designing
• Preparing
• Executing
• Witnessing
• Checking
• Resolving

20
4.2.13 Staffing and training

• Specific testing stall and skill level needed.

21
4.2.14 Schedules

• Milestone
• Time for each task
• Schedule for task and milestone
• Each resource use specific period of time

22
4.2.15 Risk and contingencies

• Identify the high risk assumption and
  contingency plan for each.

23
4.2.16 Approvals

• Specific the name and titles foe all person who
  must approve the plan.

24
5. Test design specification

• To specific the refinements of this approach and
  to ID the features to be tested by this design
  and its associated tests.

25
5.2 Outline

• Test design specification identifier
• Features to be tested
• Approach refinements
• Test identification
• Feature pass/fail criteria

26
5.2.1 Test design specification identifier

• Specify the unique identifier assigned to the
  test design specification.

27
5.2.2 Features to be tested

• Identify all software features, and combinations
  of software features to be tested.

28
5.2.3 Approach refinements

• Specify the refinements to the approach
  described in the test plan.
• Test techniques
• Analyzing test results
• Common attributes for any test case

29
5.2.4 Test identification

• List the identifier and a brief description
  associated with the design.

30
5.2.5 Feature pass/fail criteria

• Specify the criteria to be used to determine
  pass/fail testing

31
6. Test case specification

• Identify a test case identified by the test
  design specification.

32
6.2 Outline

• Test case specification identifier
• Test items
• Input specifications
• Input specifications
• Environmental needs
• Special procedural requirements
• Intercase dependencies

33
6.2.1 Test case specification identifier

• Specify the unique identifier assigned to the
  test case specification

34
6.2.2 Test items

• Requirements specifications
• Design specification
• Users guide
• Operations guide
• Installation guide

35
6.2.3 Input specifications

• Specify each input required to execute the test
  case.

36
6.2.4 Output specifications

• Specify output and features required to test
  item.

37
6.2.5 Environmental needs

• Hardware
• Software
• other

38
6.2.6 Special procedural requirements

• Describe any special constraints on the test
  procedures that executes this test case.

39
6.2.7 Intercase dependencies

• List test cases that must be executed prior ro
  this test case.

40
7. Test procedure specification

• Specific the step for executing a set of test
  cases.

41
7.2 Outline

• Test procedure specification identifier
• Purpose
• Special requirements
• Procedure steps

42
7.2.1 Test procedure specification identifier

• Specify the unique identifier assigned to the
  test procedure specification

43
7.2.2 Purpose

• Specify the step for executing a set of test
  cases.

44
7.2.3 Special requirements

• Identify any special requirements that are
  needed for the execution of the procedure.

45
7.2.4 Procedure steps

• Log
• Set up
• Start
• Proceed
• Measure
• Shut down
• Restart
• Stop
• Wrap up
• Contingencies

46
8. Test item transmittal report

• Identify any test items being transmitted for
  testing.

47
8.2 Outline

• Transmittal report identifier
• Transmitted items
• Location
• Status
• Approval

48
8.2.1 Transmittal report identifier

• Specify the unique identifier assigned to the
  test item transmittal report.

49
8.2.2 Transmitted items

• Identify the test item being transmitted.

50
8.2.3 Location

• Identify the location of the transmitted item.

51
8.2.4 Status

• Describe the status of the test item.

52
8.2.5 Approval

• Specific the name and titles of all person who
  must approve this transmittal.

53
9. Test log

• The purpose of the test log is to provide a
  chronological record of details about execution
  of test.

54
9.2 Outline

• Test log identifier
• Description
• Activities and event entries

55
9.2.1 Test log identifier

• Specify the unique identifier assigned to the
  test log.

56
9.2.2 Description

• Identify the item being tested.
• Identify the attributes of the enironments in
  which the test is conducted.

57
9.2.3 Activities and event entries

• For each event, including the beginning and end
  of activities, record the occurrence date and
  time along with the ID of the author.
• Execution description
• Procedure results
• Environmental information
• Anomalous event
• Incident report identifiers

58
10. Test incident report

• To document any event that occurs during the
  testing process that requires investigation.

59
10.2 Outline

• Test incident report identifier
• Summary
• Incident description
• Impact

60
10.2.1 Test incident report identifier

• Specify the unique identifier assigned to the
  test incident report.

61
10.2.2 Summary

• Summarize the incident

62
10.2.3 Incident description

• Inputs
• Expected results
• Actual result
• Anomalies
• Date and time
• Procedure step
• Environment
• Attempts to repeat
• Testers
• Observers

63
10.2.4 Impact

• Indicate what impact this incident will have on
  the test plans, test design specification, test
  procedure specification, or test case
  specification.

64
11. Test summary report

• To summarize the results of the designated
  testing activities and to provide evaluation base
  on the results.

65
11.2 Outline

• Test summary report identifier
• Summary
• Variances
• Comprehensive assessment
• Summary of results
• Evaluation
• Summary of activities
• Approvals

66
11.2.1 Test summary report identifier

• Specify the unique identifier assigned to the
  test summary report.

67
11.2.2 Summary

• Summarize the evaluation of the test items.

68
11.2.3 Variances

• Report and variances of the test items from
  their design specifications.

69
11.2.4 Comprehensive assessment

• Evaluate the process vs. the criteria.

70
11.2.5 Summary of results

• Summarize the results.

71
11.2.6 Evaluation

• Provide the overall evaluation of each test item
  including the limitation.

72
11.2.7 Summary of activities

• Summarize the major test and event activities.

73
11.2.8 Approvals

• Specific the name and titles of all person who
  must approve this transmittal