Software Process Improvement

1
Software Process Improvement

• Software quality and process improvement with
  ISO-9000 and SEI CMM
• Kemerer-1997, Paulk-1996

2
Software Development Process

• Process A set of partially ordered steps
  intended to reach a goal with supporting inputs,
  outputs, constraints and resources (agents and
  tools), and possibly multiple levels of
  abstraction.
• Software development process goals
• High quality
• Low cost
• On-time delivery
• Other political/technological or market
  considerations
• IEEE 1074, ISO 12207, and IEEE/EIA 12207

3
Ideal Software Process

• Predictable
• Consistency in cost/schedule estimates
• Meeting quality and functionality requirements
• Defined/Standardized
• Independent of individuals
• Repeatable
• Statistically Controlled
• Measurement
• Insight
• Optimization

4
Quality and Process Standards

• ISO 9000 (general)
• Family of Quality Management standards from
  International Standardization Organization
• TQM (general)
• Total Quality Management
• Canada's CAE, Europe's BEA, and USA's MBNQA
• SEI CMM (software-specific)
• Software Engineering Institute set of Capability
  Maturity Models
• ISO 15504, SPICE (unofficial, software-specific)
• Common base for process improvement models like
  CMM
• Others (software-specific)
• Trillium, Bootstrap,

5
ISO 9000 Principles

• Customer focus
• Leadership
• Involvement of people
• Process approach
• System approach to management
• Continual improvement
• Factual approach to decision making
• Mutually beneficial supplier relationships

6
ISO 9000 Family

• ISO 9000
• Fundamentals and vocabulary
• ISO 9001
• Requirements
• The only standard in the ISO 9000 family against
  which third-party certification can be carried.
• ISO 9002
• Models for quality assurance in production,
  installation, and servicing
• ISO 9003
• Models for quality assurance in final
  specification and test
• ISO 9004
• Guidelines for performance improvements

7
ISO 9000-3 Software

• Guidelines for the application of ISO 9001 to
  development, supply, and maintenance of software
• ISO 9000-1 is the basic concepts
• ISO 9000-2 is Guidelines for application of
  9001/2/3
• Translating ISO 9001 guidelines to software
  development with specific considerations
• Sections do not always correspond to ISO 9001 in
  a one-to-one way but cross references are
  available

8
ISO 9000-3 Outline
4. Framework Management responsibility Quality
system Internal quality audits Corrective actions
5. Life-cycle Activities Contract Planning Design
and development Testing and acceptance Maintenance
6. Supporting Activities QA, Document
Control Training, Tools Measurement, Purchasing
1. Scope 2. References 3. Definitions
9
ISO 9000-3 Excerpts

• 5.4.2.1 Development Phases
• The development plan should define a disciplined
  process or methodology for transforming the
  purchasers requirements specification into a
  software product.
• 5.6.2 Design
• In addition to the requirements common to all the
  development phases, the following aspects
  inherent to the design activities should be
  examined
• Identification of design considerations,
• Design methodology,
• Use of past design experiences,
• Subsequent processes,

10
Total Quality Management

• Started in 80s and became popular in 90s
• Considered a superset of ISO 9001 with a process
  model/approach
• Total
• Quality involves everyone and all activities in
  the company.
• Quality
• Conformance to Requirements (meeting customer
  requirements).
• Management
• Quality can and must be managed.
• TQM
• A process for managing quality a philosophy of
  perpetual improvement in everything organization
  does.

11
TQM Principles

• Quality can and must be managed.
• Everyone has a customer and is a supplier.
• Processes, not people are the problem.
• Every employee is responsible for quality.
• Problems must be prevented, not just fixed.
• Quality must be measured.
• Quality improvements must be continuous.
• Life cycle costs, not front end costs.
• Management must be involved and lead.
• Plan and organize for quality improvement.

12
CMM History

• 1982, US DoD joint task force to review the
  software problems
• One of the resulting initiatives was the Software
  Engineering Institute (SEI)
• 1984, SEI was established at Carnegie Mellon
  University
• Its purpose to address the need for improved
  software in DoD operations
• SEI developed the Software Process Maturity Model
  for DoD and industry
• 1991, the Capability Maturity Model (CMM) was
  created

13
CMM/TQM Alignment
14
Deming Chain Reaction
15
Capability Maturity Models

• CMMI, Capability Maturity Model Integration
• SW-CMM, Capability Maturity Model for Software
• P-CMM, People Capability Maturity Model
• SA-CMM, Software Acquisition Capability Maturity
  Model
• SE-CMM, Systems Engineering Capability Maturity
  Model
• IPD-CMM, Integrated Product Development
  Capability Maturity Model

16
Maturity Concepts

• Software process
• a set of activities, methods, practices, and
  transformations that people use to develop and
  maintain software and the associated products
• Software process capability
• the range of expected results that can be
  achieved by following a software process
• Software process performance
• the actual results achieved by following a
  software process.
• Software process maturity
• the extent to which a specific process is
  explicitly defined, managed, measured,
  controlled, and effective (implies a potential
  for growth in capability and indicates both the
  richness of process and the consistency with
  which it is applied

17
Institutionalization

• As a software organization gains in software
  process maturity, it institutionalizes its
  software process via policies, standards, and
  organizational structures.
• Institutionalization entails building an
  infrastructure and a corporate culture that
  supports the methods, practices, and procedures
  of the business so that they endure after those
  who originally defined them have gone.

18
SW-CMM Levels
Software Process Maturity Levels
19
SW-CMM Levels

• Initial
• The software process is characterized as ad hoc,
  and occasionally even chaotic. Few processes are
  defined, and success depends on individual effort
  and heroics.
• Repeatable
• Basic project management processes are
  established to track cost, schedule, and
  functionality. The necessary process discipline
  is in place to repeat earlier successes on
  projects with similar applications.
• Defined
• Management and engineering activities are
  documented, standardized, and integrated into a
  family of standard software processes for the
  organization. Projects use a tailored version of
  the organizations standard software processes
  for developing and maintaining software.

20
SW-CMM Levels

• Managed
• Detailed measures of the software process and
  product quality are collected. Software processes
  and products are quantitatively understood and
  controlled.
• Optimizing
• Continuous process improvement is facilitated by
  quantitative feedback from the process and from
  piloting innovative ideas and technologies.
• Level 4 and 5 are usually considered high
  maturity levels
• 9 in SEI database of 1012 organizations, as of
  March 2001.

21
Process Capability vs. Maturity
22
Key Process Areas
23
CMM Structure
24
KPA Goals

• L-2, requirements management
• Controlled requirements as baseline for
  activities
• Consistent plans, procedures, and activities
• L-2, project planning
• Documented estimates
• Planned activities
• Agreed commitments
• L-2, software tracking and oversight
• Results tracked against plans
• Corrective actions
• Agreed changes to commitments

25
KPA Goals

• L-2, subcontract management
• Qualified subcontractors selected by prime
  contractor
• Agreed commitments
• Ongoing communications
• Performance tracking
• L-2, quality assurance
• Planned QA activities
• Verified adherence to procedures
• Informed groups and individuals
• Noncompliant issues addressed by project members
  or senior management
• L-2, configuration management
• CM activities planned
• Identified, controlled, and available items
• Change control
• Groups informed of status and content of baselines

26
KPA Goals

• L-3, organization process definition
• Standard software process
• Available information
• L-3, organization process focus
• Coordinated development and improvement
  activities
• Process standard used to identify strengths and
  weaknesses
• Planned organization-level improvement activities
• L-3, Training
• Planned training activities
• Necessary trainings received

27
KPA Goals

• L-3, integrated software management
• Tailored process for each project
• Projects planned and managed based on their
  process
• L-3, software product engineering
• Defined and integrated tasks
• Consistent work products
• L-3, inter-group coordination
• Agreed customer's requirements
• Agreed commitments
• Identified, tracked, and resolved inter-group
  issues.
• L-3, peer review
• Planned activities
• Identified and removed defects

28
KPA Goals

• L-4, quantitative process management
• Planned activities
• Quantitative performance control for project
  processes
• Quantitative identification of standard process
  capabilities
• L-4, software quality management
• Planned activities
• Measurable quality goals
• Quantified and managed progress

29
KPA Goals

• L-5, defect prevention
• Planned activities
• Identified common causes of defects
• Prioritized and removed causes
• L-5, technology change management
• Planned activities
• Evaluated effect on quality
• Transferred into normal practice if appropriate
• L-5, process change management
• Planned activities
• Organization wide participation in process
  improvement
• Standard and project processes continuously
  improved

30
KPA Common Features

• Commitment to perform
• Senior management policies
• Ability to perform
• Training, special skills tools
• Activities performed
• Plans and procedures
• Monitoring
• Measurement and analysis
• Verification
• Reviews and audits

31
Examining Maturity Level

• CMM Assessment
• Voluntary and confidential
• Inspects the organization-wide capability/maturity
  
• Done by senior professionals and one or two coach
• Coaches are from SEI or an SEI-licensed
  assessment vendor
• CMM Evaluation
• Open to external inspecting body
• Usually for getting a contract
• Done by external contractor
• Very judgemental and stressful

32
CMM Assessment

• Procedure
• Obtain commitment of organization
• Preparation
• Assessment conduct (interview, data collection,
  analysis)
• Usually in 3 or 4 days
• Briefing and reporting
• Action plan development
• Follow-up
• Teams/Personnel
• Assessment team
• Senior management
• Software project managers (SPMs)
• Functional area representatives (FARs)
• Developers, QA and CM engineers, members of
  Software Engineering Process Group (SEPG)

33
Maturity Levels in Industry

• Late 90s, US software sites
• 75 at level 1
• 15 at level 2
• 9 at level 3
• 1 at levels 4 or 5
• Oct. 2002 SEI list ( sites at level 4/5)
• Australia, 2/0
• Canada, 0/1
• China, 0/2
• India, 27/50
• USA, 39/20

34
CMM Criticism

• Subjective, binary, and intrusive evaluation
• Different approached for different organizations
• Answer CMM does not specify methods, just
  criteria
• No proper integration with TQM
• Unrealistic and skewed levels
• Cost of moving from one level to another
• Large companies with traditional process model
• See CMM-XP paper !

35
Quality Assurance Techniques

• Audits, evaluations, and corrective actions by
  SQA (CMM L-2)
• Error detection and defect removal via testing
  and peer reviews (CMM L-3)
• Fault tolerance and error recovery (CMM L-3)
• Repeatable and effective software engineering
  processes
• Measurement, and defect prevention (CMM L-4 and
  L-5)
• Analysis of defect trends for needed improvements

36
Software Review

• Data shows that it is much more efficient to find
  defects in reviews than in testing
• Unit test 2 to 4 defects / hour
• Code review 10 defects / hour
• Experienced reviewers can yield 70 or more
• Typical time to find and fix defects
• Integration test 10 to 40 hrs / defect
• Inspection typically less than 1 hr / defect
• Review does not have to be inspection
• Informal vs. formal
• Record time, defects found, and material reviewed
• Metrics defect density, detection rate, average
  time,

37
CMM and ISO-9000

• Different structure but strong correlation
• CMM emphasizes on continuous process improvement.
• ISO 9000 defines minimum requirement for an
  acceptable quality system.
• Documentation and meeting quality requirements
  are important similarities.
• Cross references exist.
• See comparison article !

38
CMM and Extreme Programming

• Extreme Programming (XP) is a methodology based
  on the concept of evolutionary software
  development.
• XP is mainly initiated and used for the
  high-speed, volatile world of Internet and Web
  software development.
• XP is used in arguments against rigorous
  software process improvement models like CMM.

39
XP Basics

• Small to medium-sized teams
• Fewer than 10
• Vague and/or rapidly changing requirements
• Iterative life cycle with short cycles
• Activities coding, testing, listening, and
  designing.
• Delivery of very small increments of
  functionality, a.k.a stories
• continual communication
• With the customer and within the team
• Simplicity (minimalist solutions)

40
XP Practices

• Planning game
• Small releases
• Metaphor
• simple shared view of system
• Simple design
• Continuous integration and testing
• Refactoring
• Restructure the system without changing its
  behavior
• Pair programming and Coding standards
• Collective ownership
• Onsite customer

41
XP and Level-2 KPAs

• Largely addressed by XP
• Requirement management (stories, on-site
  customer, and continuous integration)
• Planning (planning game and small releases)
• Tracking and oversight (project velocity and
  small releases)
• Partially addressed by XP
• Quality assurance (pair programming)
• Configuration management (collective ownership,
  small releases, and continuous integration)
• Not addressed by XP
• Subcontracting

42
XP and Level-3 KPAs

• Largely addressed by XP
• Software product engineering (metaphor, simple
  design, refactoring, coding standards, and
  testing)
• Intergroup coordination (on-site customer, pair
  programming)
• Peer reviews (pair programming)
• Partially addressed by XP
• Organization process focus (team level)
• Organization process definition (life cycle)
• Not addressed by XP
• Training program
• Integrated software management

43
XP-CMM Interplay

• XP is complementary to CMM concepts, even if they
  do not completely address them.
• CMM what-to-do in general terms,
• XP how-to for some specific cases
• XP generally focuses on technical work, whereas
  the CMM generally focuses on management issues.
• Lack of institutionalization in XP
• XP is hard to implement when project size grows.