A Survey of Unit Testing Practices

1
A Survey of Unit Testing Practices

• Prof. Per Runeson Lund University
• Per Beremark SPIN-syd

2
What is Unit Testing?

• testing of individual software or hardware units
  or groups of related units IEEE610.12
• Unit testing tests individual software
  components or a collection of components. Testers
  define the input domain for the units in question
  and ignore the rest of the system. Unit testing
  sometimes requires the construction of throwaway
  driver code and stubs and is often performed in a
  debugger Whittaker 00
• a test, executed by the developer in a
  laboratory environment, that should demonstrate
  that the program meets the requirements set in
  the design specification Koomen and Pol 00

3
What Does It Mean In Practice?

1. What do people refer to as unit testing?
2. What are the strengths regarding unit testing in
   industry?
3. Which are the problems regarding unit testing?

4
Key Findings on Unit Testing

• What?
• Technical!
• Specified or not?
• How?
• Structure-based
• Not formally
• Who?
• Developer (team)
• Not test or QA

• When?
• Each build/day/week
• Takes seconds to hours to run
• Why?
• Assuring functionality

5
ParticipantsSPIN-syd of Southern Sweden

• 19 companies
• Telecom
• Automation
• Case tools
• Information systems
• Banking
• Health care
• Consulting
• Transportation

• Company size
• 1-1000 developers
• 17 participants in focus group
• 15 in questionnaire

6
Survey Procedures
1. Individual reflection
2. Focus group discussion
3. Document and structure
4. Analysis
Analysis -What? -How? -Where? -Who?
Unit testing is
Note
Note
Note
5. Survey preparation
6. Survey conduct
7. Survey analysis
7
Focus Group Meeting
Unit test smallest specified unit
3rd party verification
Test first
Training
8
Analysis Framework
Unit test
Unit test

• What?
• How?
• Where?
• Who?
• When?
• Why?

Unit test
Unit test
Unit test
Unit test
Unit test
Unit test
9
Questionnaire
10
Analysis
11
What?

• Definition
• Test of smallest unit or units
• Strength
• Unit identification
• Test of surrounding modules
• Weaknesses
• GUI test
• Unit identification
• Test scripts and harness maintenance
• Data structures

12
How?

• Definition
• Structure-based
• Preferably automated
• Strength
• Test framework
• Weaknesses
• Documentation
• Framework tailoring
• Test selection
• Test metrics

13
Who?

• Definition
• By developer
• Strength
• Independent test
• Competence network
• Weaknesses
• Competency
• Independence
• Introduction strategy

14
When?

• Definition
• Quick feedback
• Strength
• Continuous regression test
• Weaknesses
• Stopping criteria

15
Why?

• Definition
• Ensure functionality
• Strength
• External requirement (safety)
• Agile methods
• Weaknesses
• Cost vs. value

16
Further Use of the Survey

• Clarify unit test definition in your company
• Lightweight assessment of unit testing practices
• Benchmark
• Improve

17
Read more in IEEE Software July/August 2006
18

• http//serg.telecom.lth.se

19
Definitions
20
Strengths and weaknesses