Metrics Applicable to Software Maintenance Projects A Case Study

1
Metrics Applicable to Software Maintenance
ProjectsA Case Study

• Paul Knize
• Software Engineering 406
• 4/23/2006

2
IntroductionMotivation

• Estimates of the magnitude of software
  maintenance costs range from slightly over 50 to
  75 of overall software life-cycle costs. Boehm
  81
• The maintenance of existing software can account
  for over 60 percent of all effort expended by a
  development organization Pressman 05
• Personal Experience IDRS Project

3
IntroductionMotivation

• IDRS Project
• Integrated Diagnostic and Repair System
• Legacy Software (20 years)
• Started as an Experiment
• Development followed ad-hoc process
• Grew into viable solution
• Now we have to maintain this thing!
• Maintenance Project

4
IntroductionDefinitions

• Maintenance Phase
• The period of time from a successful acceptance
  test to the end of the softwares service life

Development Phase
Maintenance Phase
5
IntroductionDefinitions

• Software Maintenance
• The set of software engineering activities
  performed in the maintenance phase
• The process of modifying existing operational
  software while leaving its primary functions
  intact Boehm 81
• 4 Types of Software Maintenance
• Corrective Bug Fixing
• Perfective Optimizations
• Adaptive Change due to new operating
  environments
• Preventative Change to make future mods easier

6
IntroductionDefinitions

• Maintenance Process
• Based on IDRS Projects Maintenance Process
• Metrics Based on Data From Change Management Tool

7
Which Metrics Are Important?Goal Question -
Metric

• Goal 1 - Maximize Customer Satisfaction
• How Many Faults are the Customers Experiencing?
• Total of Change Requests of Type Defect
• What is the Magnitude of the Faults?
• Mean Priority of Change Requests of Type Defect
• How Frequently are the Customers Experiencing
  Faults?
• of Change Requests of Type Defect per unit
  Time
• How Long Does the Customer Have to Wait for a
  Fix?
• Mean Time to Release
• How Well does the System Satisfy the Customer
  Requirements?
• of Change Requests of Type Enhancement for
  current release
• Software Maturity Index

8
Which Metrics Are Important?Goal Question -
Metric

• Goal 2 Deliver On Time
• What is the Complexity of the Release?
• Average Complexity of Change Requests for Current
  Release
• What is our Current Progress?
• SPI Schedule Performance Index
• How Productive Are We?
• Complexity Weighted Change Requests Closed per
  unit Time

9
Which Metrics Are Important?Goal Question -
Metric

• Goal 3 Deliver Within Budget
• What is the Cost of the Release?
• COCOMO Maintenance Model
• BCWP Budgeted Cost of Work Performed
• How Much Have We Spent?
• ACWP Actual Cost of Work Performed
• Are we Above or Below Cost?
• CPI - Cost Performance Index
• How are the Costs Distributed?
• Person-Hours per Module
• Person-Hours per Change Request Type

10
How does a Maintenance Project Generate These
Metrics?

• Goal 1 - Maximize Customer Satisfaction
• What is the magnitude of the defects the Customer
  experiences?
• SCCB classifies and prioritizes change requests

• How frequently do Customers experience faults?
• Change requests dated upon entry into system

• How long do customers have to wait for a fix?
• Mean time to Release

11
How does a Maintenance Project Generate These
Metrics?

• Goal 1 - Maximize Customer Satisfaction
• How well does the system satisfy the customer
  requirements?
• Software Maturity Index
• provides an indication of the stability of a
  software product based on changes that occur for
  each release of the product Pressman 05

MT of modules in current release Fc
changed modules in the current release Fa
added modules in the current release Fd
modules from preceding release deleted in current
release SMI varies between 0 and 1 where an SMI
1 indicates a completely unchanged or stable
release SMI 1 should never happen
12
How does a Maintenance Project Generate These
Metrics?

• Goal 2 Deliver On Time
• What is the current progress?
• SPI Schedule Performance Index

BCWP Budgeted Cost of Work Performed Effort
Estimation of Closed Change Requests BCWS
Budgeted Cost of Work Scheduled Effort
Estimation of All Change Requests for Current
Release
SPI 1 On Schedule SPI lt 1 Behind
Schedule SPI gt 1 Ahead of Schedule
13
How does a Maintenance Project Generate These
Metrics?

• Goal 3 Deliver Within Budget
• Are we Above, On or Below Cost?
• CPI Cost Performance Index

BCWP Budgeted Cost of Work Performed Effort
Estimation of Closed Change Requests ACWP
Actual Cost of Work Performed Actual Effort of
Closed Change Requests for Current Release
CPI 1 On Cost CPI lt 1 Above Cost CPI gt 1
Below Cost
14
What Do These Metrics Tell Us?

• Example An Increase in the frequency of change
  requests of type Defect indicates a degradation
  in S/W quality
• May indicate an overly complex module(s),
  inadequate testing or a change in the operating
  environment
• Should trigger an investigation look at other
  metrics e.g. Defects per Module

15
What Do These Metrics Tell Us?

• Example The number of Change Requests weve
  closed is trailing behind the number we have
  scheduled for this release
• We are behind schedule and its getting worse

16
What Do These Metrics Tell Us?

• Example At first its taking us more effort than
  estimated to close change requests, we soon get
  on track but then go over cost again
• We are over cost but acceptably
• Good way for management to keep an eye on cost

17
Conclusions

• It is possible and beneficial for a maintenance
  project like the IDRS project, that may not have
  generated or used software metrics at any time
  during its development phase, to generate and use
  software metrics to achieve high customer
  satisfaction, stay on schedule and within budget
• Metrics from the Maintenance Process give us
  insight into the quality/state of the product
• They can supplement but not take the place of
  good product metrics

18
Questions?