Evaluation of Software Quality using Quality Models

1
Evaluation of Software Quality usingQuality
Models

• S G Ganesh

2
Code Quality and Quality Models Why Bother?
3
Bugs Resulting Disasters or Huge Losses

• Historically software defects have caused major
  disasters/huge losses!
• Therac-25 accidents
• Ariane-5 disaster
• Pentium FDIV bug
• ATT long distance line failure (9 hrs)
• many many more

4
Simplified Code ATT Failure
5
Bugs cost huge money!

• Quote from National Institute of Standards and
  Technology (in 2002, now the cost will be more)
• Software bugs, or errors, are so prevalent and
  so detrimental that they cost the U.S. economy an
  estimated 59.5 billion annually, or about 0.6
  percent of the gross domestic product.
• Software is becoming more pervasive, so software
  quality is becoming more important than ever
  before

6
Why Writing Quality Code Is Difficult Case study

• Joshua Bloch is the author of popular Effective
  Java and Java Traps and Pitfalls books he
  wrote many classes in Java library, including
  java.util
• In 2006, he wrote an online blog/article Nearly
  all Binary Searches and Mergesorts are broken
  googleresearch.blogspot.com/2006/06/extra-extra-re
  ad-all-about-it-nearly.html
• It is about a customer reported overflow bug in
  binarysearch algorithm in java.util.Arrays! The
  bug was reported after the library was in wide
  use (millions of users) for 9 years!!
• The algorithm was based on Jon Bentleys widely
  read Programming Pearls book!

7
Here is that code
8
Discussion on that overflow bug

• Code tries to find the mid-point from low to
  high (both 32 bit signed integers) to determine
  which way to continue the search.
• int mid (low high) / 2
• For large values, the expression (low high) can
  overflow. One solution
• int mid low (high - low) / 2
• The problem is not with binary search algorithm
  per se
• for integers as in mathematics, it is correct.
  The limitation is in representing integers in
  computers
• well known limitation gt bug could have been
  avoided

9
Why Quality Models?

10
Software Quality An Elusive Goal

• What is not measurable make measurable
  Galileo
• Driving force behind quality of products in
  other engineering disciplines
• You cant control what you cant measure.
  DeMarco
• Metrics are widely and successfully used in
  other engineering disciplines
• However, measuring software quality is a problem
  
• In general, it is difficult to evaluate quality
  of a thing thats intangible

11
Software Metrics Mixed Success

• Software metrics are measures related to
  software
• They seem to help solve the problem of
  evaluating software quality
• For example, static measures like McCabes
  Cyclomatic Complexity or dynamic measures like
  Meantime Between Failures (MTBF)
• However, they have severe limitations (unlike
  metrics in other engg. disciplines)
• Software metrics alone are often inadequate or
  ineffective in evaluating software quality
• There is a gap between the quality
  characteristics of the software that we care
  about and what metrics provide

12
Quality Models Motivation

• Consider this case As developer, one of the
  characteristics of quality software is that
  its well designed
• How do we measure intangible aspect such as
  design?
• There are lots of metrics available on design
• For example, metrics on coupling, cohesion,
  complexity etc.
• But what metric values will indicate good
  design? Can you see the problem here??
• Quality models comes to rescue
• They can bridge between metrics and
  characteristics we expect in the software

13
Quality Model Definition and Example

• Here is a definition of Quality Model (from ISO
  15498-1)
• The set of characteristics and the relationships
  between them which provides the basis for
  specifying requirements and evaluating quality
• Quality models have relationship with the

14
4 Levels in QMOOD Mapping Approach

• The model has 4 levels and mappings between them
  is done
• Quality attributes are evaluated using design
  properties design properties from metrics
  metrics from design components (4 levels)

15
QMOOD How Mapping Works

• Step 0 Identify high-level design quality
  attributes that should be evaluated

16
QMOOD How Mapping Works

• Step I Design components such as methods,
  classes, objects and relationship between them
  are identified

17
QMOOD How Mapping Works

• Step II Mapping is done from design components
  to metrics

18
QMOOD How Mapping Works

• Step III Mapping from design metrics to design
  properties

19
QMOOD How Mapping Works

• Step IV Mapping from design properties to
  design quality attributes

20
Analysis Does Quality Model Help?

• First of all, it is clear that measuring design
  quality - to an extent - is feasible
• It is repeatable process, based on metrics,
  language independent
• Basically its a divide-and-conquer approach
• We need to admit that quality models help in
  evaluating quality
• But all are not well
• Obscure mappings from metrics to evaluating
  quality attributes
• How did we arrive at these magic numbers? Are
  they correct?
• When quality ratings are low, how to improve
  design is not clear

21
McCalls Model

• McCalls model is one of the earliest model
• Very influential ISO 9126/SQuaRE are based on
  that, for example
• Here is McCalls Triangle of Quality

22
McCalls Model

• An example of Factor-Criteria-Metrics Model

23
ISO 9126 Model

• Most widely used model
• Also because its an ISO standard
• Strictly hierarchical model (unlike McCalls)
• Quality characteristics are divided into
  sub-characteristics
• They should in turn be measured using metrics
• Provides a guidance on selecting metrics
• Leaves the process and how of evaluating
  quality characteristics
• The process of product evaluation is provided
  in ISO 14598 standard

24
ISO 9126 Product Life Cycle Software Quality

• Process quality and internal quality (code
  quality) affects external quality (product
  quality) effect of the software product is what
  is perceived by the uses when they actually use
  it
• Internal and external quality are closely related
  and are provided together in this model quality
  in use is covered separately

25
ISO 9126 Quality Model
26
ISO 9126 Model Problems

• Just a couple of problems listed here for
  discussion
• The selection and measure of metrics can be too
  subjective
• Which metric to choose is open
• What values to use for measure of metrics for
  evaluation is also open
• How evaluation of metrics maps to evaluation of
  software sub-quality and later quality attributes
  is not clear

27
SQuaRE Quality Model
28
Thank You
It is time for QA