Requirements Metrics: Scaling Up

1
Requirements Metrics Scaling Up

• Kimberly S. Wasson
• University of Virginia

2
Goal of Comparison

• Establish the relative value of results across a
  field
• Advances the academic maturity of the field
• Advances the practical utility of the results
• Thus, a benchmarking initiative
• One step along the way is after-the-fact data
  aggregation

3
Position

• It will take a focused community effort
• to achieve
• benchmark-quality requirements metrics.

4
Premise

• Requirements metrics as they are currently used
  dont scale
• Useful local definitions and uses of metrics are
  inconsistent when compared together
• Linguistically
• Methodologically
• Cant aggregate data
• Cant visualize larger patterns or trends
• Cant found purpose-designed benchmarks

5
Implication

• To enable data aggregation, we must use
  standardized metrics
• Standardized along several dimensions
• Meaning
• Use
• Value
• Since they dont exist, we have to create them

6
Creating Standard Metrics

• Some dimensions of concern
• Meaning
• Linguistic issues in bounding the meaning of a
  metric and encouraging its consistent
  interpretation
• Use
• Methodological issues in encouraging consistent
  and tractable operationalization
• Value
• Is a metric uniformly interesting, and does it
  have implications for how we do requirements
• This is why it will take a community effort

7
Proposal

• I propose a structured community effort to
  undertake this task
• Pre-meeting this afternoon
• Refine and plan this agenda
• Long-term working group
• Formalize goals
• Decide set of metrics of interest
• Construct definitions and operationalizations
• Assemble and disseminate use guide
• Engage in follow-up

8
Intention

• Demonstrate the possibility of valid data
  aggregation
• Show that results independently obtained in
  distributed environments can be compared with
  validity if they conform to guidelines
• Use this result as foundation for
  purpose-designed benchmarks
• Shepherd larger, rigorous comparative studies
  with goal of comparison at outset

9
Summary

• Metrics as they are currently used dont scale
• If we want to aggregate data, we must standardize
  metrics for this purpose
• I propose a working group to undertake this task
• The product has potential to shepherd larger,
  rigorous comparative studies