Test Specification

1
Test Specification

• Sabesan

2
Why test?

• Improve quality
• Find bugs (no crashes, erroneous calc)
• Verify the requirements?
• To put it another way
• Testing is a verification process to improve the
  confidence that the system conforms to a
  specification
• Your job is to prove that the specification is
  implemented correct

3
Fulfill requirements

• Think of how to test each requirement
• Some requirements should be considered in all
  test cases (e.g. write to output file)
• Others have specific tests
• Nothing unexpected (if spec. is not open)

4
Writing a test case

• Numbered
• Refers to requirement(s)
• Prove valid test case (against spec)
• Shows that all requirements are addressed (Suite)
• Tip A requirement that would have generated an
  error earlier does not need to be mentioned over
  and over again
• Explains HOW to execute the test
• For dummies, include all required info
• Explains the in-parameters and the expected
  output.
• User (or script) able to determine the verdict
  PASS or FAIL

5
Reference

• The purpose of the Test Specification is to
  verify a system according to a referenced
  Requirement Specification, i.e., add VERSION
  number of the specification
• Most of the formalism as RS

6
Test Specification A. Part 2 Test case A.1
Tested requirements - Req. A.3.10 ) (boundary
testing value inside the range) - Req. A.3.14
) (boundary testing value inside the range) -
Req. A.3.15 ) (boundary testing value inside
the range) - Req. A.1.27 ) mainly targeted
requirements for this test case Test data -
Change to a directory where you have write
permissions. For example cd / - Create a
simple XML-Input file containing a camera
(location0,0,0 sky0,1,0 look_at0,0,1)
and a red sphere (radius 1.2 location0,0,12
surfacepigmentcolor1,0,0) and name the file
simple_scene1.xml. - Write the following
command to the shell tracer -z 500 -x 500 -y 500
-w 1 -h 1 test_a1.ppm ltsimple_scene1.xml - Write
the following command to the shell to view the
output picture xview test_a1.ppm Expected
result - The program should terminate without
error message.
7
Tests for Non-Functional Req

• How to test a req. Code should be written in a
  nice and structured way?
• Refer to code review
• Verdict PASS if code review PASS
• If code not available write N/A not applicable
• This should be in the Test Plan (not in TS)
• How to test a req. Use getopt lib
• Query supplier e.g. ask developer/manager
• This is a simple YES/NO question

8
Testing limitations

• A crashed program is not an option.
• Parameter range e.g. if the specified range is
  1..1000 this gives the tests
• Inside range close to bound, i.e., 1 and1000
• Common or other value picked random inside the
  range, e.g., 100
• Outside range close to bound, i.e., 0 and 1001
  WHAT will happen here

9
Combinations

• You cannot test all cases, e.g., 100010001000
  values
• Vary upper and lower bound
• Vary object hidden each other (partly or fully)
• Vary object not in view of the camera
• You can discuss functionality with you MP2 group,
  e.g., how the camera works or how the XML file
  works, but NOT the test cases

10
Discussion

• Is it good to slim the test suite?

11
Equivalent tests

• For function f(x)
• Test f(0 if x0),(x if 0ltxlt10), and (0 if x10)
• If range for x is 0..10
• f(0) f(some in 1..9) f(10) must be tested
• Classes 0 1,...,910
• Is f() affected by something else?
• Can we suspect problems if something else changes?

12
Report
Test Verdict Comment
1 PASS
2 FAIL No error mess
13
Report

• Which system is tested (code version)
• Who has tested
• Which test specification used (version)
• RS