Software Quality Assurance

1
Software Quality Assurance

• What is quality software and how to achieve it?

2
Lecture Objectives

• To discuss what is meant by quality in software
• To understand the importance of quality in
  software engineering
• To describe the features of quality software
• To describe the measures used in assessing
  software quality
• To understand the international standards that
  are applied to assess software quality

3
What Is Quality Software?

• Is it software that works?
• Is it software that is error-free?
• Is it software that is very well-documented and
  has help facility for all its functions?
• Is it software that does not hang the machine?
• Is it software that does not bomb?

4
Group Project

• How do you ensure the quality of the software you
  are developing?
• Have you defined the standards and tasks to
  ensure the quality of the software?
• Will the customer say that your software is a
  quality product?
• Are you fulfilling the customers expectations?

5
Software Quality Definition

• Conformance to
• Explicitly stated functional and performance
  requirements
• Explicitly documented development standards, and
• Implicit characteristics that are expected of all
  professionally developed software
• (Pressman)

6
Software Quality Factors

• Low defects level when deployed
• Zero defect most preferably
• High reliability
• Capability of running without crashes
• Majority of the user satisfy with software when
  conducted the survey
• Effective customer support
• Rapid defect repair

7
Software Qualities

• Correctness
• Reliability
• Robustness
• Performance
• User friendliness
• Verifiability
• Maintainability
• Reparability
• Safety

• Evolvability
• Reusability
• Portability
• Understandability
• Productivity
• Size
• Timeliness
• Visibility

Select the critical attributes and plan how to
achieve them
8
Benefits Of Software Quality

• Reduced maintenance cost
• Stable and useful product
• Satisfy customer needs
• Better chances for continuing releases
• Build corporate culture and identity
• Better chances for software and design reuse

9
Software Quality Assurance

• Quality assurance a set of establish framework
  (procedures standard) used by an organization
  to achieve high quality software
• Essential activity for any business that produce
  product to used by others
• It need to be planned and systematic
• It does not just happen
• Assure each of the software qualities is met
• Goal set in requirements specification
• Goal realized in the implementation
• SQA is applied at every stage of the software
  process
• Quality standard is selected to apply to the
  software process

10
Why SQA Payoff ?
11
Principle Of SQA

• Set the standard and quality attributes that a
  software product must meet
• The goal to achieve
• Measure the quality of software product
• There is a way to determine how well the product
  conforms to the standards and quality attributes
• Track the values of the quality attributes
• It is possible to assess how well we are doing
• Use the information of software quality to
  improve the quality of future software product
• There is a feedback into the software development
  process

12
SQA Activities

• Application of Technical Methods
• Conduct of Formal Technical Reviews
• Software Testing
• Enforcement of Standards
• Control of Change
• Measurement
• Record keeping and Reporting

13
Application of Technical Methods

• Select appropriate tools and methods to capture
  system requirements, analyze system, design,
  implementation and testing
• Why use methods and tools
• To ensure high quality is achieve
• Analyst achieve high quality specifications
• Designer develop high quality designs
• Ability to measure quality in specifications and
  designs

14
Conduct of Formal Technical Reviews

• FTR is an activity to assess quality
• A stylised meeting conducted by technical staff
  with the sole purpose of uncovering quality
  problems
• Found to be effective in uncovering defects in
  software
• All major software items should be subjected to
  technical review

15
Software Testing

• Testing is a process of executing a program with
  the intent of finding errors, locate errors and
  to proof system correctness
• Testing is conducted based on the developed test
  cases
• Capture actual output
• Compare actual output with expected output
• Actual Expected Test case succeed
• Actual ! Expected Test case failed
• All test cases result must be recorded

16
Software Testing(Cont)

• To identify errors in software developed
• Test case design methods produces tests to be
  used on the software
• Development of test strategy for a set of tests
  that uncovers all possible errors
• Thorough testing is not as effective as expected
  in most cases

17
Enforcement of Standards

• Formal standards and procedures varies from
  company to company
• Could be dictated by customer, regulations, or
  self-imposed
• If formal (written) standards exist, they must be
  followed to ensure quality
• Assessment of compliance is done through FTR or
  audit

18
Control of Change

• An activity executed throughout the system life
  cycle to control change of products and life
  cycle artifacts
• Items that needs control changes
• Plans
• Specification
• Procedures
• Audit
• Report
• Support
• A software library need to be construct that
  store, manages and track these items

19
Control of Change (Cont)

• Every change has potential for introducing errors
  or creating side effects that propagate errors
• Change control process ensures quality
• Formalizing requests for change, evaluating
  nature of change, and controlling the impact of
  change

20
Measurement

• Measurement Numeric derivation for some
  attribute of a software product / process
• Compare the value with standard apply
• Often expressed in and days
• Integral part to any engineering discipline
• Software metrics must be collected to track
  quality
• Also to assess the impact of methodological and
  procedural changes on improved software quality

21
Record keeping and Reporting

• Historical record for project - results of
  reviews, audits, change control, testing, other
  SQA activities
• Provide procedures for collection and
  dissemination of SQA information
• Dissemination to development staff based on
  need-to-know basis

22
McCall Software Quality Model
Product Revision
Product Transition
Product Operations
23
McCalls Software Quality Factors

• Product Operations
• Operational characteristics
• Product Revision
• Ability to undergo changes
• Product Transition
• Adaptability to new environments

24
McCalls Software Quality Factors (Cont)
25
FCM By McCall Approach
Factors user concern oriented
Criteria software concern oriented
Metrics quantitative measures
26
Measuring the Factors

• The factors are difficult to measure directly
• McCall defined metrics that are assessed
  subjectively, on a scale of 0 to 10
• Includes auditability, accuracy, completeness,
  consistency,modularity, etc.
• Factors calculated as follows
• Fq c1 x m1 c2 x m2 cn x mn
• where
• cn Weighting coefficient (or importance
  factor)
• mn Criteria measured

27
Alternative Measurements

• Hewlett-Packard developed a set of quality
  factors - FURPS
• Functionality - features, capabilities, overall
• Usability - human factors, consistency, doc.
• Reliability - failure, accuracy, MTBF, recovery
• Performance -speed,response time, efficiency
• Supportability - maintainability, testability
• Quality metrics for each step in SE process

28
Measuring Quality Factors In SE Process
Investigations/ Specifications
Design
Implementation
Testing
Support
..... ...
F
U
R
P
S
29
Software Quality Assurance

• Quality assurance - essential activity for any
  business that produces products to be used by
  others
• SQA - a planned and systematic pattern of actions
  to ensure quality in software
• Quality is Job 1

30
Software Review

• like a filter for SE process
• must be applied at various points during
  development
• all technical work need reviewing
• also checks ones mistakes
• helps to discover defects early
• design can introduce 50 - 65 of errors
• review can uncover 75 of them
• cost of subsequent steps reduced

31
ISO 9000

• Set of standards that can be applied to a range
  of organisations from manufacturing through to
  service industries
• ISO 9001 - general standards, applies to design,
  development, maintaining products
• ISO 9000-3 - supporting document for software
  development
• Various countries also have own standards

32
ISO Certification

• Some countries have bodies which certify that the
  quality process in an organisation conforms to
  ISO requirements
• In Malaysia, SIRIM performs this function
• Customer looks for certification in an
  organisation to indicate the quality of products
  or services

33
Areas covered by ISO 9001/9000-3 STANDARD (1/3)

• A. A. Quality System framework
• 1. Management responsibility
• quality policy defined, documented, understood,
  implemented and maintained
• responsibilities and authorities for all
  personnel defined
• in-house verification resources defined, trained
  and funded
• a designated manager ensures that the quality
  program is implemented and maintained
• 2. Quality system
• requires a documented quality system be
  established
• should be an integral process throughout the
  entire life cycle
• 3. Internal quality system audits
• requires audits be planned and performed
• results communicated to management
• deficiencies corrected
• 4. Corrective action
• requires causes of non-conforming product be
  identified
• potential causes of non-conformance eliminated
• procedures changed as a result

34
Areas covered by ISO 9001/9000-3 STANDARD (2/3)
35
Areas covered by ISO 9001/9000-3 STANDARD (3/3)
C. Quality System Supporting Activities

• 1. Configuration management
• process of development and maintenance should be
  documented and controlled
• 2. Document control
• distribution and modification of documents should
  be controlled
• 3. Quality records
• quality records should be collected, maintained
  and dispositioned
• 4. Measurement
• where appropriate, adequate statistical
  techniques should be identified and used to
  verify the acceptability of process capability
  and product characteristics

• 5. Rules, practices and conventions
• are in place and followed
• 6. Tools and Techniques
• are applied appropriately to support the
  development process
• 7. Purchasing
• purchased products conform to their specified
  requirements
• 8. Included software product
• should be verified and maintained
• 9. Training
• training needs should be identified and training
  provided since related tasks may require
  qualified personnel
• training records should be maintained

36
Quality Management System
ISO 9000 Quality Models
instantiated as
Organisational Quality Manual
Organisational Quality Process
documents
is used to develop
instantiated as
Project Quality Management
Project 1 Quality Plan
Project 2 Quality Plan
Project 3 Quality Plan
37
ISO SQA

• An organisation that have SQA tasks in place and
  a quality management process would have most of
  ISO elements
• Ideally quality management should be separate
  from project management
• This is to ensure quality considerations are not
  compromised by concerns of budget and schedule

38
Process Improvement

• Understanding existing processes and changing
  these processes
• to improve product quality
• and/or to reduce costs and development time
• Assume that key factor to product quality is the
  quality of development process
• kaizen - process of continuous process improvement

39
Capability Maturity Model

• Developed by Software Engineering Institute (SEI)
  of Carnegie-Mellon U
• SEI was established to improve the capabilities
  of US software industry
• Focus on improving the process of software
  engineering
• Similar to ISO, organisations are assessed and
  certified to be on Level 1 to Level 5

40
Process Maturity Model
41
Level 1 Initial

• No effective management procedures or project
  plans
• No organisational mechanisms to ensure consistent
  use of formal procedures
• Successful software depends more on individual
  skill rather than organisation

42
Level 2 Repeatable

• Formal management, quality assurance and
  configuration control procedures
• Organisation can successfully repeat projects of
  the same type
• Lack of formal process model

43
Level 3 Defined

• Organisation has defined its process and has a
  basis for qualitative process improvement
• Formal procedures are in place to ensure that the
  defined process is followed in all software
  projects

44
Level 4 Managed

• Organisation has a defined process and a formal
  programme of quantitative data collection
• Process and product metrics are collected and fed
  into the process improvement activity

45
Level 5 Optimising

• Organisation is committed to continuous process
  improvement
• Process improvement is budgeted and planned and
  is an integral part of the organisations process

46
Key Process Area
47
Key Process Areas

• Level 1 Initial
• none
• Level 2 Repeatable
• Software configuration management
• Software quality assurance
• Software subcontract management
• Software project tracking and oversight
• Software project planning
• Requirements management

48
Key Process Areas (Continued)

• Level 3 Defined
• Peer reviews
• Intergroup coordination
• Software product engineering
• Integrated software management
• Training programme
• Organisation process definition
• Organisation process focus

49
Key Process Areas (Continued)

• Level 4 Managed
• Software quality management
• Quantitative process management
• Level 5 Optimising
• Process change management
• Technology change management
• Defect prevention

50
References

• Software Engineering A Practitioners Approach
  5th Ed. by Roger S. Pressman, Mc-Graw-Hill, 2001
• Software Engineering by Ian Sommerville,
  Addison-Wesley, 2001