Beyond the Personal Software Process: Metrics collection and analysis for the differently discipline

1
Beyond the Personal Software Process Metrics
collection and analysis for the differently
disciplined

• Philip M. Johnson Hongbing Kou
• Joy Agustin Christopher Chan
• Carleton Moore Jitender Miglani
• Shenyan Zhen William E.J. Doane

25th International Conference on Software
Engineering 2003 6 pages
Poster by Greg Borota
2
Summary

• Personal Software Process (PSP) improves
  estimation and quality assurance significantly.
• PSP adoption problem
• high overhead of PSP-style metrics collection and
  analysis
• context switch between product development and
  process recording
• Solution Hackystat system fully automate both
  data collection and analysis, eliminating
  overhead and context switching.
• Some new issues introduced by Hackystat

3
Background

• 1995 Watt Humphrey of Software Engineering
  Institute (SEI) adapts organizational-level
  software measurement and analysis techniques to
  the individual Personal Software Process.
• PSP makes two conjectures
• PSP-style collection and analysis of metric data
  can provide significant benefits
• These benefits are large enough to motivate the
  developer to continue using PSP even if not
  mandated to

4
Background

• Research supports the first conjecture PSP
  improves estimation and quality assurance.
• Anecdotal evidence does not support the second
  conjecture (empirical research missing) high
  overhead pushes developers to drop PSP when not
  mandatory no (voluntary) adoption at all.
• Tools like Leap, PSP Studio or PSP Dashboard
  lower the overhead of metrics collection and
  analysis context switch problem still not solved
  though some adoption but still very low.
• Hackystat shots at widespread adoption reduce
  overhead to zero and eliminate context switch.

5
Hackystat overview

• Leap system started in 1998 continual context
  switch back and forth between working and telling
  the tool what work was being done major problem
  surprisingly low adoption
• Hackystat project started in 2001 addresses Leap
  shortcomings by
• metrics collected automatically by attaching
  sensors to development tools
• metric data automatically sent by sensors to a
  sever
• sever analyzes gathered data
• analysis results via email alerts or in a web
  interface

6
(No Transcript)
7
Results

• Currently no known way of automatically
  collecting all metrics as defined by PSP.
• Hackystat changes the definition of PSP metrics
  to make them susceptible to automatic collection
  and analysis
• Effort active modification of a file with a
  fixed grain size of five minute increments
  differs from PSP
• Size option to choose from a large variety of
  ways of measuring size PSP LOC, number of new
  methods by analyzing Java .class file, etc.
• Defects sensors are attached to the unit
  testing framework and to the bug reporting system
  quite different from PSP definition and
  oversimplified

8
Conclusions

• Strengths
• In Hackystat metrics collection and analysis is
  completely automated and transparent to the user
  once the sensors are in place no user
  intervention
• Weaknesses
• Hackystat does not provide a solution to
  gathering PSP data automatically. Metrics
  definition is substantially changed and sometimes
  oversimplified
• A new adoption barrier is created, the specter of
  Big Brother. Hackystat server provides a
  detailed log of developer activity. Developer has
  no control over the recorded data management
  can easily access it.

9
Questions

• What are the two components that define the PSP
  adoption problem?
• What is the solution to PSP adoption problem
  provided by Hackystat?
• Should Hackystat provide an option to hide the
  recorded data from others (management, etc.) like
  encryption with a personal key/password?