Software Quality : The Elusive Target

1
Software Quality The Elusive Target

• Kitchenham B. Pfleeger S. L.

January, 1996 IEEE Software
Zhenyu Wu
2
Introduction
Questions

• What do you really mean by software quality (SQ)?
• Is your definition adequate?
• How to evaluate your products SQ ?
• How to measure SQ?
• How to achieve SQ?

3
Meaning of SQ

• A good definition should let us measure quality
  in a meaningful way
• Measurements let us know
• if techniques improve quality
• how process quality affects product quality
• how quality methods affect product quality during
  use
• Is the investment in SQ methods profitable?

4
Meaning of SQ

• Depends on how you approach software quality
• Different groups involved have different views
• Product-based
• Process-based
• People believe quality is important and can be
  improved

5
Views of SQ

• SQ can be perceived in various domains
  (philosophy, economics, marketing and operations
  management)
• SQ is complex and multifaceted concept and can be
  described from five different perspectives
• Transcendental view
• User view
• Manufacturing view
• Product view
• Value-based view

6
Transcendental View

• See quality as something that can be recognized
  but not defined
• Quality as something toward which we strive as an
  ideal, but may never implement completely
• Abstract sense

7
User View

• More concrete, grounded in the product
  characteristics that meet users needs
• Evaluate product in a task context can be
  personalized view
• Asses product behavior with respect to
  operational profiles
• Related to product usability and reliability

8
Manufacturing View

• Focus on product quality during production and
  after delivery
• Concerns whether or not the product was
  constructed right the first time
• Can lead to quality assessment independent of the
  product
• Adapted by ISO 9001 and CMM, advocates
  conformance to process rather than to
  specification

9
Product View

• Consider the products inherent characteristics,
  looks inside
• Assumption measuring and controlling internal
  product properties will result in improved
  external product behavior
• More research needed to conform this assumption

10
Value-Based View

• Different views can be held by different groups
  involved in software development
• users requirement for a useful product conflict
  with manufacturers goal of minimizing rework
• consider the trade-offs between cost and quality
• manage conflict ? design to cost
• compare with potential benefits
• This view tries to bring in compromise between
  different groups and views

11
Measuring Quality

• We need to measure quality so we can
• Establish baselines
• Predict likely quality
• Monitor improvement
• Product attributes contribute to user
  satisfaction are a mixture of
• products functions
• products nonfunctional qualities
• constraints
• ISO definition of quality the totality of
  characteristics of an entity that bear on its
  ability to satifsfy stated and implied needs.

12
Measuring user's view

• Reliability how long the product functions
  properly between failures
• Usability including ease of installation,
  learning and use
• Tom Gilbs technique characteristics can be
  measured directly
• Example
• Learning time average elapsed time (in
  hours) for a typical user to achieve stated level
  of competence

13
Measuring user's view

• This technique can be generialized to any quality
  feature
• The quality concept is broken down into component
  parts until each can be stated in terms of
  directly measurable attributes

14
Measuring the Manufacturer's View

• Defects count
• Count the number of known defects recorded during
  development and use
• Can compare modules, products or projects (with
  same way and same time)
• Relationship between defects count and
  operational failures is unclear
• Can indicate test efficiency and identify process
  improvement areas

15
Measuring the Manufacturer's View

• Rework costs
• Any additional effort required to find and fix
  problems after documents and code are formally
  signed-off as part of configuration management
• Count debugging effort during integration and
  testing
• Not count end-phase verification and validation
• Pre-release rework measure manufacturing
  efficient and process improvement
• Post-release rework measure of deliverd quality

16
Modeling Quality

• Several models have been built to understand and
  measure quality
• Old Models
• McCalls model
• ISO 9126
• New model
• Dromeys model

17
McCall's Model

• Defines product qualities as hierarchy of
  factors, criteria and metrics
• Quality factor represents behavior characteristic
  of the system
• Quality criterion is an attribute of a quality
  factor that is related to software production and
  design
• Quality metric is a measure that captures some
  aspect of a quality criterion.

18
McCall's quality Model

• The metrics have to be answered with yes or
  no answer
• The final quality is assessed by dividing the
  yes answers by the total number of questions
  chosen to describe the quality

19
McCalls Model
20
ISO 9126

• Six characteristics, completely hierarchical
• Quality charactertistics are divided into
  sub-characteristics which can be measured using
  metrics
• Recommends measuring the char-acteristics
  directly, but does not indicate clearly how to do
  it

21
Quality Characteristic Definition
Functionality A set of attributes that bear on the existence of a set of functions and their specified propertise. The functions are those that satisfy stated or implied needs.
Relibity A set of attributes that bear on the capability of software to maintain its performance level under stated conditions for a stated period of time.
Usablity A set of attributes that bear on the effort needed for use and on the individual assessment of such use by a stated or implied set of users.
22
Efficiency A set of attributes that bear on the relationship between the softwares performance and the amount of resources used under stated conditions.
Maintainability A set of attributes that bear on the effect needed to make specified modifications (which may include correction, improvements, or adaptations of software to environmental changes and changes in the requirements and functonal specifications).
Portability A set of attributes that bear on the ability of software to be transferred from one environment to another (this includes the organizational, hardware or software environment).
23
ISO 9126
Suitability Accurecy Interoperability Secuity
Functionality
Materity Fault Telerance Recoverability
Reliebility
Understandability Learnability Operability
Usability
Characteristics
Sub-characteristics
24
ISO 9126
Time behavior Resource behavior
Effiency
Analyzability Changeability Stability Testability
Maintainability
Adaptability Installability Conformance Replaceabi
lity
Portability
Characteristics
Sub-characteristics
25
Main Differences between Mccalls and ISO 9126
model

• Mccalls
• with different quality frame
• quality factor
• not hierarchical

• ISO 9126
• work and terminology
• quality characteristic
• and subcharacteristics
• completely hierarchical

26
Model Problems

• Lack rationale for determining which factor
  should be included in the quality definition
• Lack rationale for deciding which criteria relate
  to a particular factor
• The measure of metrics can be too subjective
• Does not describe how lower level metrics are
  composed into an overall assessment of higher
  level quality characteristics

27
Dromey's models(new model)

• Higher level quality attributes cannot be
  measured or built directly
• Higher level quality can be achieved by building
  components which exhibit a consistent, harmonious
  and complete set of product properties that
  result in manifestations of quality attributes
• Can allow us to verify models

28
Business Value of Quality

• Businesses invest valuable money for obtaining
  good software, we have to pay them back in good
• We should be careful in assessing and providing
  software with good quality
• How much "less than perfect" software are the
  businesses willing to accept should be determined

29
Conclusion

• Quality is a complex concept It means different
  things to different people
• You must define aspects of quality in which you
  are interested
• Quality should be defined in a measurable way, so
  that it can be understood and verified
• Quality must be related to business goals