Software Quality Views and Approaches

1
Software QualityViews and Approaches

• CFICSE October 2001
• Diane Kelly
• (adapted from lectures by
• Dr.Terry Shepard)

2
References

• The Elusive Target, Kitchenham, Pfleeger, IEEE
  Software Jan 96
• Software Architecture in Practice, Bass,
  Clements, Kazman

3
Views of Quality (not software specific)

• Transcendental eureka!, the ideal
• User fitness for purpose
• Manufacturing conformance to spec
• Product characteristics of the product
• Value-based willingness to pay

4
User View

• can be highly personalized
• typical user views
• reliability
• performance
• usability
• ISO 9126

5
Manufacturing View

• avoid rework and associated costs
• often leads to process focus (CMM, ISO 9001-3)
• process does not guarantee good products only
  guarantees uniformity of output (does insist on
  improving process)

6
Product View

• measure internal characteristics?
• Measurements chosen depend on type of product
  (e.g. code, design, )
• may not ensure quality of external
  characteristics
• measure external characteristics?
• Corresponds to user/manufacturer views
• measurements chosen depend on product (e.g. OS,
  spreadsheet, )

7
Value-based view

• helps to decide trade-offs during development
• cost vs. reliability
• cost vs. functionality
• cost to consumer is affected by scale
• high volume products have a higher perceived
  value
• value is defined by
• potential savings/productivity enhancements
• profit opportunities

8
Problems with Qualities

• qualities are at odds with one another
• always a compromise
• portability versus performance
• security versus fault tolerance
• modularity versus efficiency
• difficulty in trying to measure qualities e.g.,
  accuracy, usability
• what does quality really mean?

9
McCall (1977) product view

• 11 factors, 23 criteria
• multiple dependencies
• subjective metrics

10
McCall's Quality Model
Quality Factors
Quality Criteria
11
ISO 9126 (1992)

• 6 independent (?) characteristics,
• 21 sample subcharacteristics (definitions of
  subcharacteristics firmly in users view)
• states that developers must use same quality
  characteristics as the user, but can use
  different metrics to measure them
• The developers view must also incorporate the
  view of the quality characteristics required by
  those maintaining the software. (???)
• no specific metrics proposed here
• does give an evaluation process model

12
Qualities from ISO 9126
13
Problems with McCall, ISO 9126

• why those particular factors/characteristics?
• why those particular criteria/subcharacteristics?
• vague in definitions of lower levels what do we
  actually do to achieve the qualities?

14
Proposed solution 1 Focus on Product (Dromey,
1995, 1996)

• most dimensions of quality are measured after the
  fact
• need to identify actions that can embed needed
  qualities in the products, up front
• implementation quality model identify what the
  programmer can do to ensure quality
• link these actions to high level qualities

15
Proposed solution 1 (contd)

• requirements quality model all requirement
  statements must be measurable
• design quality model define qualities for
  design, identify characteristics of design that
  would embed those qualities

16
Proposed solution 2Focus on Process (Evans
Marciniak, 1987)

• integrate quality decisions into the design
  process, e.g.
• by using quality checklists during inspection,
  design
• by determining testing criteria up front
• by following a documented design process

17
Quality Approach with Design (Bass, Clements,
Kazman)

• product approach
• design (architecture and low-level)
• code
• quality attributes discernable at runtime
• quality attributes not discernable at runtime
• business qualities

18
Qualities Discernable at Runtime (1)

• performance
• responsiveness of system
• addressed at both architecture level and code
  level
• security
• ability to resist unauthorized attempts at usage
  at denial of service
• addressed through architectural solutions

19
Qualities Discernable at Runtime (2)

• availablity
• proportion of time system is up and running
• (mean time to failure)/(mean time to failure
  mean time to repair)
• fault-tolerant architectures, separation of
  concerns, modifiability, testability
• functionality
• ability to do intended work
• largely non-architectural in nature

20
Qualities Discernable at Runtime (3)

• usability
• learnability, efficiency, memorability, error
  avoidance, error handling, satisfaction
• mostly use interface concerns
• flow of information and performance are
  architectural concerns

21
Qualities Not Discernable at Runtime (1)

• modifiability
• ability to make changes quickly and cost
  effectively
• quality attribute most closely aligned with
  architecture
• portability
• ability to run under different computing
  environments
• typically addressed by layered architecture

22
Qualities Not Discernable at Runtime (2)

• reusability
• ability for structures and components to be
  reused in future applications
• architectural components are units of reuse
• integrability
• ability to make separately developed components
  work correctly together
• depends on the external complexity of components
  and interactions with other components

23
Qualities Not Discernable at Runtime (3)

• testability
• ease with which software can be made to
  demonstrate its faults through execution based
  testing
• tied to observability and controlability
• related to architectural documentation,
  separation of concern, information hiding,
  incremental development

24
Business Qualities

• time to market
• cost
• projected lifetime of system
• targeted market
• rollout schedule
• integration with existing legacy systems

25
Goal/Question/Metric(Basili 1984)

• addresses more than software quality
• goals defined by organisational units
• questions formulated to assess progress to goal
• metrics quantify answers to questions
• e.g. Hewlett Packard (Grady 1992)

26
GQM Example

• Goal
• Determine when the project is completed.
• Question
• How many new defects are found by the testing
  activities?
• How many of the required features are
  implemented?
• Metric
• defect counts
• defect types
• feature count

27
Can Quality be reduced to a single attribute?

• Need to do so in order to make a go/no go
  decision
• Is the quality good enough in all dimensions?
• Hard problem
• Cadorette thesis (1994) maybe Analytical
  Hierarchical Process theory (from other
  disciplines)
• Joannou et al (Ontario Hydro 1990) yes, by
  requiring a specific set of quality attributes to
  be satisfied in a specific context