Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise

1
Evaluating Pair Programming with Respect to
System Complexity and Programmer Expertise
IEEE Transactions on Software Engineering, Vol
33, No 27, pp 65 86, February 2007
By Erik Alsholm, Hans Gallis, Tore Dyba, Dag
I.K. Sjoberg Simula Research Laboratory,
Norway Presented by Mukundan Venkataraman
Mohammad Zubair Ahmad School of EECS, UCF
2
Pair Programming

• TWO programmers work on the same task using ONE
  computer and keyboard
• Requires two specific roles
• Driver types and handles code details
• Navigator observes driver, looks for defects,
  referencing, think up alternatives
• In effect, the driver continues to perform the
  stated task, while the navigator provides
  alternative ways or suggestions to speed up the
  process.

3
Motivation

• - Programming in pairs is advantageous rather
  than programming individually
• - Previous experiments concluded that Pair
  Programming (PP) have various benefits
• - State significant improvements in
• Functional correctness
• Quality of programs
• Reduced duration
• Minor additional overhead

4
Problem with existing studies

• - Cannot be compared directly due to differences
  in sample populations
• - Have not accounted for moderating effect of the
  complexity of programming tasks
• - System complexity and Programmer expertise
  would have a significant impact on when and
  how PP is beneficial

5
Main Focus The Question

• What is the effect regarding duration, effort,
  and correctness of pair programming for various
  levels of system complexity and programmer
  expertise when performing change tasks?

6
Dependent Variables
Duration Elapsed time taken to perform a set of
change tasks Effort Total number of programmer
hours taken to develop a correct
program Correctness Whether or not final program
possessed required functionality
7
Conceptual Model and Hypotheses

• - Substantial differences in how novices,
  intermediates and experts perceive the
  difficulties
• - For simple tasks, individuals might perform
  better than groups
• - For complex tasks, groups benefit from
  competence of their peers
• - Results in increased performance compared with
  individuals

8
Conceptual Model and Hypotheses
- Systematically study the interactions of
programmer expertise and system complexity on
three outputs duration, effort and correctness
9
Experiment Design

• Only professional software developers considered
• A total of 295 Java consultants considered
• - 98 pairs
• - 99 individuals
• First phase conducted on individual developers in
  2001
• Second phase conducted on pairs in late 2004 and
  early 2005

10
Experiment Design

• Pairs made up of individuals with similar levels
  of programmer expertise
• Subjects were given six major tasks
• - One training task
• - One pre-test task
• - Four main experimental tasks

11
The nine null hypotheses myths about PP
12
Validity of sample population

• - One major reason that most previous studies
  failed was lack of a representative population,
  or sample
• - Samples in the past have consisted of
  university students, or professionals with
  varying (unknown) degree of expertise
• -Authors use professional hiring channels
  (consultants) to hire paid groups of developers
  with varying degrees of expertise
• - People with similar expertise have known to
  work effectively in pairs
• - The student samples were altogether removed.
  This left the authors with professional Java
  developers

13
Overall effect of PP, compared to individuals
- Marginal decrease in duration, significant
increase in effort
14
Overall PP Effect of system complexity

• Variability in duration for simple and complex
  tasks
• Complex tasks require more effort and duration

15
Effect of PP for juniors in isolation
- Marginal increase in duration, but a big leap
in both effort and correctness
16
Juniors Effect of system complexity
- Indistinguishable results on duration and
effot, but a big leap in correctness of solutions
17
Overall effect of PP on intermediates
- Decrease in duration and increase in effort
18
Intermediates Effect of system complexity
- Simple tasks achieved faster, and complex tasks
produce more correctness in results
19
Overall effect of PP on seniors
- Effort is significantly high, and correctness
actually drops
20
Seniors Effect of system complexity
- Effort doubles with more complex systems.
Simple systems tend to perform better.
21
Understanding overall results

• H01 Time taken to perform change tasks is equal
  for SQ and PP practices.
• Pairs worked 8 faster
• H02 Difference in time taken for projects does
  not depend upon system complexity
• For simple tasks, pairs worked 20 faster
• For complex tasks, pairs worked 6 slower
• -H03 Difference in time taken for projects does
  not depend upon programmer expertise
• Junior pairs worked 5 slower than individuals,
  intermediate pairs 28 faster, and senior 9
  faster.

22
Understanding overall results (cont.)

• - H04 Effort expended for change tasks is same
  for PP and SQ
• Pairs took 84 more effort overall
• -H05 Difference in effort remains the same with
  increasing system complexity
• For simple tasks, pairs required 60 more effort
• For complex tasks, pairs required 112 more
  effort
• -H06 Difference in effort does not depend on
  programmer expertise
• Pairs required more effort in general. Juniors
  111, Intermediates 43 and seniors 83.

23
Understanding overall results (cont.)

• - H07 Correctness for maintained programs is
  equal for both PP and SQ.
• Pairs had a 7 increase in proportion of correct
  solutions
• -H08 Difference in correctness between PP and SQ
  does not depend on system complexity
• For simple tasks, pairs had a 16 decrease, while
  for complex tasks, pairs had a 48 increase in
  correct solutions.
• -H09 Difference in correctness between PP and SQ
  does not depend on programmer expertise
• Junior pairs had a 73 increase in correct
  solutions, while intermediates had a 4 increase,
  and seniors an 8 decrease

24
Implications for Research and Practice

• - When is PP really effective?
• - This is a very pertinent question for
  organizations, especially with extreme
  programming practices
• Junior programmers performing maintenance tasks
  would find PP highly beneficial
• Senior programmers hard to come by, and pairs of
  juniors are easy to hire
• Authors cannot seem to explain differences in
  results
• Social psychology might offer more insights into
  the results
• Improvement in performance due to presence of
  others seems to increase with simple repetitive
  tasks. This quickly fades with complex tasks
  requiring deep insights.

25
Conclusions

• - PP has enjoyed considerable popularity
• - Its effectiveness is unknown when should
  organizations adopt this practice, and how much
  of it
• - With increasing competition, and the need to
  reduce costs, improve quality and decrease effort
  and turn around time, an in-depth knowledge on
  PPs applicability is an important study
• -This study provides a first such insight, and
  the authors take on PP showcases surprising
  results.
• - The study, as the authors admit, is not
  complete. Their results can best be interpreted
  as conservative
• - Much needs to be investigated the effect of
  jelling, the usage of realistic workloads, and
  psychological factors.