Software Quality Assurance: A Management Perspective 1997 Robert H' Dunn

1
Software Quality Assurance A Management
Perspective1997Robert H. Dunn

• EEL6887 Software Engineering Life Cycle Control
• Prashanth Aedunuthula
• School of EECS
• University of Central Florida
• February 01, 2006

2
Reference Sources

• Software Quality Assurance A Management
  Perspective, by Robert H. Dunn, Textbook
  Software Engineering Project Management, 2nd ed.,
  edited by Thayer and Yourdon, IEEE Computer
  Society, 2000, pp433.
• Textbook Interpreting the CMMI, A Process
  Improvement Approach, Kulpa, Johnson, Auerbach
  Publications, 2003.
• Textbook Software Quality Assurance, practical
  approach, Tsun S. Chow, IEEE computer society,
  1985.

3
Overview

• Software Quality Assurance
• Need for SQA
• Techniques used in SQ
• Difference between CMM CMMI
• Application of SQA in CMMI

4
Software Quality Assurance

• IEEE Definition
• A planned and systematic pattern of all
  actions necessary to provide adequate confidence
  that the software conforms to established
  technical requirements
• CMM Definition
• The purpose of quality assurance is to
  provide management with appropriate visibility
  into the process being used by the software
  project and of the projects being built
• SQA Focuses on
• Continuous Improvement of the process (through
  audits)
• Defect prevention
• Analysis of the product
• Validation of the product

5
Rationale for SQA

• In earlier stages, the Quality of the S/w product
  was assessed on the basis of Back-end Testing.
• The above approach is no longer applicable .The
  reasons for wanting to improve S/w processes and
  Quality
• Project out of Control
• - Uncertainty in progress ,unreliable cost
  completion estimates
• Poor Performance of product
• - Frequent crashes, inconsistent results,
  excessive use of resources
• Maintenance difficulties
• - Complex software, code untraceable
• Difficulty of use
• - not matching customer expectations,
  abilities of the environment.
• SQA helps in Controlling projects by emphasizing
  on the suitability of the process, defect
  prevention activities, analysis, and support for
  testing.

6
Constituents of a SQA Program

• Making certain programming staff is adequately
  trained in new techniques, methods and tools.
• Evaluating the effectiveness of current
  development methods.
• Ensuring that project plans are on par with
  standards
• Early defect detection- using reviews, analysis
  tools and tests.
• Configuration control
• Analyzing defect data to improve processes.
• Validating the final product
• Gathering, analyzing, and evaluating user
  feedback.
• Evaluating potential software suppliers and
  monitoring their performance.
• Evaluating the fidelity with which plans and
  applicable standards are followed, and
  determining causes for deviations.
• Making Certain staff is empowered to prevent
  defective code, artifacts of development and user
  documentation.

7
Techniques of Software Quality Assurance

• Quality Control
• Audits
• To confirm compliance with agreed procedures
• Applied to both process and product
• Generally performed by SQE

8
Techniques of Software Quality Assurance Contd..

• Trend Data
• Identify process variables attached to quality
  and process control
• Control charts Completed items vs. time,
  Failures vs. time, etc.
• Generally performed by SQE

See Notes
9
Techniques of Software Quality Assurance Contd..

• Direct Measurements of Software products and
  Service
• Identifying the current state of the system
  (maintenance)
• Size and Complexity measures- Size (KLOC),
  Complexity (McCabe)
• Generally performed by SQE.

10
Techniques of Software Quality Assurance Contd..

• Failure Analysis
• - discovering sources of failure which cant
  be detected by a simple diagnosis
• - general root causes of failure poor SRS,
  design.
• Process Cause and Effect
• Attempts to find weak areas in the software
  process
• Based on Ishikawa scheme- man, machine, material,
  method and measurement.
• Four main control points- Caliber or staff,
  effectiveness of technology, effectiveness of
  tools, and maintainability

Fish Bone Diagram for Software Defects Source
Confluence of Six sigma Journal
11
Techniques of Software Quality Assurance Contd..

• Pareto Analysis (80/20 rule)
• Method of separating vital few from the
  trivial many
• Applying Pareto s theory , the quality of
  software mainly depends on few processes and
  these should be areas of concentration for
  failure analysis and improvement efforts.
• Can be applied to other areas like Improving
  quality of service.
• Source
• www.int.gu.edu.au/courses/7014int/qa.html

12
Techniques of Software Quality Assurance Contd..

• Efficacy of Defect Removal (EOR)
• The earlier a fault can be found , the less it
  costs to fix.
• EOR Number of faults actually found/ Number of
  faults theoretically
• found 100
• 50 EOR is bad.
• Task Entrance Criteria
• A task shouldnt be started until the
  implementation staff has a solid foundation .
• Diversities in Software life cycle definitions
  and programming paradigms makes it harder to
  choose any entrance criteria.
• Task Exit Criteria
• Making sure that a task is not considered
  complete until it is truly complete.
• Generally done using Spot Audits correlating
  sign-off sheets with the actual artifacts of
  completed tasks.

13
Techniques of Software Quality Assurance Contd..

• Test Control
• - SQA is greatly attached with the testing
  process
• Planning
• Main objectives of testing
• - Determining that code performs as it
  is expected to.
• Initial version- test cases, new version
  regression testing
• - Exposing as many faults as possible.
• Ensuring enough structural coverage.
• - Determining suitability of the code to
  the application
• Beta testing.
• Following the Plans
• Audits, esp. Spot Audits

14
Techniques of Software Quality Assurance Contd..

• Configuration Control
• Made easy with modern build systems coupled with
  library control tools.
• Many tools available for CC.
• Needs monitoring by SQE team.
• Test Exit Approval
• Special case of task exit criteria.
• Check-off lists
• SQA ensures that
• all testing tasks were completed,
• deviations are noted ,and tracked for closure.

See Notes
15
Software Quality and the CMM
Source Software Engineering Institute
(SEI) Software Capability Maturity Model (SW-CMM)
v1.1
16
Evolution of CMMI

• The first CMM (CMM v1.0) was developed for
  software and released in August 1990.
• Based on this success and the demand from other
  interests CMMs were developed for other
  disciplines and functions
• Systems Engineering
• People
• Integrated Product Development
• Software Acquisition
• Software Quality Assurance
• Measurement
• Others.

17
Evolution of CMMI contd..

• While organizations found these various CMMs to
  be useful they also found them to be
• Overlapping
• Contradicting
• Lacking clean, understandable interfaces
• Lacking standardization
• Displaying different levels of detail
• In addition, many organizations also had to deal
  with ISO 9001 Audits or TickIT audits based on
  ISO 9000-3
• This resulted in expensive, confusing and
  conflicting process improvement programs

18
Evolution of CMMI Contd..

• The CMM Integration Project was formed to
• Establish a framework to integrate current and
  future models
• Build an initial set of integrated models
• The source models that served as the basis for
  the CMMI include
• CMM for Software v2.0 Draft C
• EIA 731 Systems Engineering
• IPD CMM (IPD) v0.98a
• CMMI incorporates most of the current thinking on
  software management practices and corrects many
  of the shortcomings of the SW-CMM.

19
CMMI
20
Comparing CMMI to SW-CMM (v1.1)

• The major changes found in the CMMI fall into
  three categories
• disciplines covered
• CMMI Software Engineering, Systems
  Engineering,
• Integrated process and product development,
  Software Sourcing
• maturity levels and process areas
• Levels 2 and 4 are named Managed and
  Quantitatively Managed, respectively emphasize
  the evolution of the management processes from a
  qualitative focus to a quantitative focus.
• CMMI 25 PA, SW-CMM 18 PA
• model structure.
• Confusing terms Capability and Maturity
• CMMI has both representations Staged
  (maturity), Continuous (capability)

See Notes
See Notes
21
CMMI-Process and Product Quality Assurance

• Stresses the objective evaluation of products as
  well as processes!!
• Evaluation criteria must be established based on
  business objectives
• What will be evaluated?
• When or how often will a process be evaluated?
• How will the evaluation be conducted?
• Who must be involved in the evaluation?

22
Summary
Senior Management
Quality Assurance
Maintain QA Processes
Escalate (Corrective Action)
Facilitate Process Defn. and Metrics
Information System
Process Documentation
Reporting/ Records
Evaluations/ Audits
Projects
Customers (QAR)
Subcontractors (QA)
23
Conclusion

• Software Quality assurance cannot exist without
  management commitment.
• No standard automated tools exist to support
  Software Quality Assurance (except for
  configuration management) but metrics are useful
  tools for many SQA processes.
• SQA yields better results when performed by an
  independent SQA team.
• SQA (PPQA) directly applies to level 2 of SW-CMM
  (CMMI). However, more quantitative approach is
  followed in levels 3-5.
• Organizations using SW-CMM v1.1 should be able to
  transition to CMMI by focusing on changes in
  Measurement and Analysis at Level 2.