The effect of experience on the use of TDD

1
The effect of experience on the use of TDD

• Tor Stålhane
• According to
• M.M. Muller and A. Høfner

2
Goal

• Is TDD a techniques that just anybody can use and
  immediately achieve good results?
• Or
• Is TDD a technique that will help experienced
  developers to perform better that they otherwise
  would?
• The reported experiment tries to find answers to
  these questions.

3
Participants

• Novice group 11 students from an XP course.
• Experts 7 experienced developers.
• Statistically, the two groups are too small to
  identify small to medium effects. For large
  effects, however, 28 persons will be enough.

4
The development task 1

• Each participant should develop a control system
  for an elevator. The system has
• A request component activated when we press the
  up or down button
• A job component activated when we pushes the
  selected floor button
• A state machine with the states
• Driving up
• Driving down
• Waiting
• Open

5
The development task 2

• For each cycle of the state machine, the system
  fetches a list of requests and jobs.
• The participants received
• A state transition diagram
• A program skeleton that contained the
  implementation of the three states driving up,
  driving down and waiting as mock objects
• The experiment participants should implement code
  for the state Open.

6
Research hypothesis 1

• H1 Experts have a higher degree of conformance
  to the TDD rules than novices have.
• H2 The experts have a shorter development cycle
  than the novices have.
• H3 The experts have a smaller variation in cycle
  length than the novices have
• H4 The experts have less code changes than the
  novices have

7
Research hypothesis 2

• H5 The experts are faster than the novices when
  it comes to code changing lines of code changed
  per hour
• H6 The tests developed by the experts has a
  higher coverage than the tests developed by the
  novices.

8
Conformance to TDD rules

• This was measured as follows

• TDD changes
• change in methods which were previously called by
  a test that failed
• new methods that were later called by a test that
  failed
• Refactoring changes in the structure of the code
  but not its observable behavior.

9
What is a box plot

• In order to read some of the diagrams that
  follows, you need to understand a box plot.

10
Small box plot example

• A total of 24 observations
• M median
• K1 first quartile
• K2 second quartile the same as the median
• K3 third quartile

x x x x x x x x
x x x xx x x xx
x xx
x x x x
11
TDD conformance results 1
12
TDD conformance results 2

• H1 is confirmed experts are more conform to the
  method than the novices are

13
Development cycle

• H2 and H3 are confirmed experts work faster and
  more consistent

14
Code changes

• H4 is rejected experts and novices change the
  same amount of test code and application code

15
Speed of change

• H5 is accepted for application code but not for
  tests.

16
Coverage

• H6 is accepted both for statement coverage and
  for block coverage

17
Added LOC at first acceptance test

• Experts write smaller code chunks both for
  application and for tests

18
Failed tests at first acceptance test

• Novices wrote many tests that did not fail.

19
Do TDD like a pro

• Improve refactoring. Change structure, not
  behavior.
• Strive for high test coverage.
• Write small chunks of code and tests
• Focus on missing functionality - write tests that
  will fail