Some CMM Assessment FindingsKey Practice Trouble Spots

1

• Some CMM Assessment Findings-Key Practice Trouble
  Spots
• Dr. Randy Walters

2
Key Process Areas (KPAs)
Continuously improving and business innovation
Quantitative management control focus
Process definition Integrated end-to-end process
architecture
Process discipline Basic management control
Ad hoc, chaotic
3
State of Practice (SEI-December 1998)
55.2
697 Organizations since 1993
25.8
15.6
2.7
0.6
2 Repeatable
3 Defined
4 Managed
5 Optimizing
1 Initial
4
Key Process Area Structure

• Goals supported by Key Practices (KPs)
• KPs
• State the fundamental policies, procedures, and
  activities for a KPA
• Describe what is to be done,
• Do not interpret them as a mandate for how
• CMM has 316 key practices
• 121 at Level 2
• 108 at Level 3

5
Key Process Area Structure

• KPs organized by Common Features
• Commitment to perform (CO-n)
• Org. policies for proj, Sr. mgt leadership
• Ability to perform (AB-n)
• Groups for, adequate resources, training for
  roles
• Activities performed (AC-n)
• Measurement and analysis (ME-n)
• Measurements made and used for...
• Verifying implementation (VE-n)
• SQA reviews/audits, sr. mgt review, compliance

6
Background

• Assessments- SPAs, CBA-IPI, CMM-based evaluation
  (Consortium developed)
• Data analyzed from 1991-1997 assessments done by
  Software Productivity Consortium
• 104 Assessments
• 2,221 analyzed findings
• 1,651 mapped to CMM Key Practices
• 270 duplicates and excluded findings
• 300 positive, general or neutral findings
• Military, aerospace, communications, business

7
Distribution of Weaknesses Found in Level 2 KPAs
8
Weakness Percentage () in Level 2 KPAs
Number of assessments finding weakness in
KPA / Number of times KPA assessed
9
Top Pitfall Practices in Level 2 KPAs
Number of assessments finding weakness in
Key Practice (KP) / Number of times KPA assessed
10
Top Pitfall Practicesin Level 2 KPAs
11
Distribution of Weaknesses Found in Level 3 KPAs
12
Weakness Percentage () in Level 3 KPAs
Number of assessments finding weakness in
KPA / Number of times KPA assessed
13
Top Pitfall Practices in Level 3 KPAs
Number of assessments finding weakness in
Key Practice (KP) / Number of times KPA assessed
14
Top Pitfall Practicesin Level 3 KPAs
15
Common Themes

• Following documented procedures
• SPP AC-9, AC-10, AC-12 Size/Effort/Cost
  procedures
• OPD AC-1, AC-2, ISM AC-1 Develop/maintain/document
  /tailor organization standard process
• ISM AC-4 Manage according to defined process

16
Common Themes

• Deriving and tracking estimates
• SPP AC-9, AC-10, AC-12 Estimation procedures
• SPTO AC-5, AC-6, AC-7, AC-8 Tracking of
  size/effort/cost/schedule/critical resources
• Demonstrating commitment by management
• SPP CO-2, SQA CO-1 Project and SQA Policy
• All level 3 CO-1 Policies
• Providing training

17
Common Themes

• Providing training
• SQA AB-4, SCM AB-5(SCM training)
• TP CO-1( Training Program Policy),
• TP AC-1, AC-2 Training plans and courses

18
Assessor Experiences

• Level 2 KPs most often missed
• Estimating work product sizes according to a
  documented procedure
• Estimating effort and cost according to a
  documented procedure
• Tracking work product sizes

19
Assessor Experiences

• Level 3 KPs most often missed
• Developing an organizational training plan
  according to a documented procedure
• Tailoring project processes from the
  organizational process
• Developing the organizational process according
  to a documented procedure

20
Assessor Experiences

• General pitfalls
• Visibility and communication of decisions,
  changes, results of negotiations among all groups
• Detailed and effective risk management
• Definitions and interpretations of
• Organization, project, roles, groups, members
• Who constitutes senior management
• Software Configuration Control Board
• Customer(s)
• Execution evidence and process assets retained