Standards, Certification and Assessment

1
CHAPTER 7

• Standards, Certification and Assessment
• PART I

2
Learning Objectives

• To discuss issues on
• The benefits of use of standards
• The organizations involved in standards
  development
• The ways in which SQA standards contribute to SQA
• The classification of standards (i) Quality
  management standard and (ii) Project process
  standard.

3
The Benefits of Use of Standards

• The ability to apply software development and
  maintenance methodologies and procedures of the
  highest professional level
• Better mutual understanding and coordination
  among development teams but especially between
  development and maintenance teams
• Greater cooperation between the software
  developer and external participants in the
  project
• Better understanding and cooperation between
  suppliers and customers, based on the adoption of
  known development and maintenance standards as
  part of the contract.

4
Prominent Developers of SQA and SE Standards

• IEEE (Institute of Electrical and Electronics
  Engineers) Computer Science Society
• ISO (International Organization for
  Standardization)
• DOD (US Department of Defense)
• ANSI (American National Standard Institute)
• IEC (International Electrotechnical Commission)
• EIA (Electronic Industries Association)

5
Classification of SQA Standards

• SQA standards can be divided into 2 main classes
• SQA management standards, including certification
  and assessment methodologies (Quality management
  standards)
• Software project development process standards
  (project process standards).

6
Classification of SQA Standards - comparison
7
PART I

• Quality Management Standards

8
Learning objectives

• To discuss
• The benefits of using SQA standards
• The general principles underlying quality
  management according to ISO 9000-3
• The ISO 9000-3 certification process
• The principles embodied in the CMM
• The CMMI structure and process areas
• The principles underlying ISO/IEC 15504

9
The scope of quality management standards
certification standards

• The scope of certification standards is
  determined by the aims of certification, which
  are to
• Enable a software development organization to
  demonstrate consistent ability to assure
  acceptable quality of its software products or
  maintenance services. Certification is granted by
  an external body.
• Serve as an agreed-upon basis for customer and
  supplier evaluation of the suppliers quality
  management system. Accomplished by performance
  of a quality audit by the customer.
• Support the organization's efforts to improve its
  quality management system through compliance with
  the standards requirements.

10
The scope of quality management standards
assessment standards

• The scope of assessment standards is also
  determined by the aims of assessment, which are
  to
• Serve organizations as a tool for self-assessment
  of their ability to carry out software
  development projects.
• Serve for improvement of development and
  maintenance processes by application of the
  standard directions
• Help purchasing organizations determine the
  capabilities of potential suppliers.
• Guide training of assessor by delineating
  qualifications and training program curricula.

11

• ? certification standards external to support
  the supplier-customer relationships.
• ? assessment standards internal focuses on
  software process improvement.

12
ISO 9000-3 principles

• Customer focus organization must understand
  current and future customer needs.
• Leadership leaders establish the organizations
  vision.
• Involvement of people people are the essence of
  an organization.
• Process approach a desired result is achieved
  more efficiently when activities and resources
  are managed as a process.
• System approach to management identify,
  understand and managing processes.
• Continual improvement ongoing improvement of
  overall performance should be high on the
  organizations agenda.
• Factual approach to decision making effective
  decisions are based on the analysis of
  information.
• Mutually supportive supplier relationships
  organization and suppliers are interdependent.

13
ISO 9000-3 requirements
Galin, SQA from theory to implementation
14
The ISO 9000-3 certification process

• Organization wishing to obtain ISO 9000-3
  certification are to complete the following
• Develop the organizations SQA system
• Implement the organizations SQA system
• Undergo certification audits

15
The ISO 9000-3 certification process (Galin, 2004)
16
Planning the process leading to certification

• An internal survey of the current SQA system
  should supply information about
• Gaps between currently employed SQA and required
  procedures missing procedures in addition to
  inadequate procedures.
• Gaps between staff know-how and knowledge
  required regarding SQA procedures and SQA tools.
• Gaps regarding documentation of development as
  well as maintenance activities.
• Gaps or lack of parity regarding software
  configuration system capabilities and
  implementation.
• Gaps regarding managerial practices demanded for
  project progress control.
• Gap regarding SQA unit organization and its
  capabilities.

17
Planning the process leading to certification
(cont..)

• After completing the previous analysis, the plan
  for obtaining certification can be constructed.
  It should include
• A list of activities to be performed, including
  timetable
• Estimates of resources required to carry out each
  activity
• Organizational resources (a) internal
  participants SQA unit staff (including staff to
  be recruited) and senior software engineers (b)
  SQA consultants.

18
Development of the organizations SQA system

• Development of a quality manual and a
  comprehensive set of SQA procedures.
• Development of other SQA infrastructure
• Staff training and instruction programs,
  including staff certification
• Preventive and corrective action procedures,
  including the CAB committee
• Configuration management services, including a
  software change control management unit
• Documentation and quality record controls
• Development of a project progress control system.

19
Implementation of the organizations SQA system

• Efforts are shifted towards implementing the
  system.
• Includes
• setting up a staff instruction program and
  support services
• target team leaders and unit managers follow
  up and support the implementation.
• Internal quality audits verify the success in
  implementation of SQA system.

20
Undergoing the certification audits

• Review of the quality manual and SQA procedures
  developed by the organization.
• Verification audits of compliance with the
  requirements defined by the organization in its
  quality manual and SQA procedures. The main
  questions to be answer
• Have the staff been instructed on SQA topics?, do
  they display a satisfactory level of knowledge?
• Have the relevant procedures project plan,
  design reviews, progress reports, etc been
  properly and fully implemented by the development
  team?
• Have documentation requirements been fully
  observed?

21
Capability Maturity Models CMM and CMMI
assessment methodology

• Developed by Carnegie Mellon Universitys
  Software Engineering Institute (SEI) in 1986.
• Brief description of the maturity process
  framework.

22
The principles of CMM

• Quantitative management methods increases the
  organization's capability to control the quality
  and improve the productivity.
• Application of the five-level capability maturity
  model that enables to evaluate the achievements
  and determine the efforts needed to reach the
  next capability.
• Generic process areas that define the what
  not how enables the model's applicability to a
  wide range of implementation organizations
• It allows use of any life cycle model.
• It allows use of any design methodology,
  development tool and programming language.
• It does not specify any particular documentation
  standard.

23
The CMM key process areas (KPAs)
24
Capability Maturity Model Integration (CMMI)

• Started in the late 1990s.
• CMM - develop different sets of key processes for
  different departments that exhibited joint
  processes.
• Departments that applied different CMM variants
  in the same organization faced difficulties in
  cooperation and coordination.
• Three versions of CMMI
• CMMI-SE/SW SE and selected aspects of software.
• CMMI-SE/SW/IPPD/SS SE, software, and integrated
  product/process aspects.
• CMMI-SE/SW/IPPD SE, software, product/process,
  and supplier sourcing aspect.

25
The CMMI structure and processes area
26
The ISO/IEC 1554 the principles

• Harmonize the many existing independent
  assessment methodologies by providing a
  comprehensive framework model (what has to be
  accomplished rather than how it has to be
  done).
• Be universal to serve all or almost all
  categories of software suppliers, customers and
  software categories.
• Be highly professional.
• Aim at reaching international acceptance as world
  standard. To save suppliers' resources by
  eliminating the need to perform several different
  capability assessments in response to different
  customer requirements.

27
Structure of the ISO/IEC 15504 assessment model

• The model is composed of
• Capability levels and process attribute
  requirements for each level
• An achievement grade scale for process attributes
• Accumulative achievement requirements and their
  inputs and outputs.

28
ISO/IEC 15504 process assessment model
29
ISO/IEC 15504 - Capability levels and process
requirements
30
Achievement grades scale for ISO/IEC 15504
process attributes (Jung et al., 2001)
31
Accumulative achievement requirements for an
ISO/IEC 15504 capability model (Jung, et al.,
2001)
32
The ISO/IEC TR 15504 Standard - structure

• Part 1 Concepts and introductory guide
• Part 2 A reference model for processes and
  process capability
• Part 3 Performing an assessment
• Part 4 Guide to performing an assessment
• Part 5 An assessment model and indicator guide
• Part 6 Guide to competency of assessors
• Part 7 Guide for use in process improvement
• Part 8 Guide for use in determining supplier
  process capability
• Part 9 Vocabulary