CHAPTER 8 : CMMBASED SOFTWARE PROCESS ASSESSMENT zahran

1
CHAPTER 8 CMM-BASED SOFTWARE PROCESS
ASSESSMENTzahran
2

• OBJECTIVE
•  to understand the use of CMM as a
• process model to assess processes
•  to know the usage of SEI questionnaire as an
  assessment instrument

3
CMM-Based SPA

• Introduction of CBA
• Context of CMM-Based Assessments
• SEI-Assisted Assessment Process
• SEI Software Capability Evaluation(SCE)
• SEI Maturity Questionnaire

4
CMM-BASED APPRAISALS

• The CMM is used as the basis for different types
  of software process assessment known as
  CMM-based appraisals
• CMM-based appraisals are any assessments based on
  the Capability Maturity Models key process area
  and the SEI questionnaire

5

• Generally, an assessment has a number of
  essential components
• A process model against which to assess the
  current state of your process
• An assessment method specifies the phases and
  steps of the assessment
• One or more assessments tools to assist in the
  activities of fact gathering, fact analysis and
  reporting

6
Contexts of CMM-bBased Assessments
SEI-assisted assessments -  Conducted by a
trained team of software professionals from both
the SEI and the organization being
assessed -  The scope usually a section or
division of an organization or possible an entire
site
7

• Software Capability Evaluations
• Subset of an assessments
• SEI does not perform evaluations, but it does
  offer training in conducting software capability
  evaluation
• Conducted as part of DoD software acquisition
  process
• May also be used by software organizations to
  evaluate subcontractors

8

• Self- assessments
• Conduct by a team of software professionals from
  the same organization undergoing the assessments
• SEI trains in-house assessment teams to conduct
  self-assessment
• Scope - division organization, an organization
  site or entire organization
• Provide a way for an organization to examine its
  own software process and monitor process
  improvement

9
SEI-ASSISTED ASSESSMENT PROCESS

• Have the following features
• Uses the CMM as a process maturity framework
• Uses a questionnaire as a tool for the assessment
  team

10

•   Objectives of SEI-assisted assessments
• To determine the current state of the software
  process that is practiced on a day to-day basis
  in an organization
• To identify the high-priority key areas for
  software process improvement on which the
  organization should focus its improvement efforts

11

• To enroll key software practitioners in the
  software process improvement and change process
• To help an organization focus on monitoring and
  improving its software process
• To identify areas of strength and spread them
  across the organization
• To send a copy of the assessments report back to
  the SEI to be stored in the assessments database

12
PHASES OF SEI-ASSISTED ASSESSMENT

• PRE-PLANNING
• PLANNING
• ASSESSMENT
• POST-ASSESSMENT

13
Selection phase

•  
• Involves identifying an organization as a
  candidate for assessment
• Setting up an executive-level briefing

14
Commitment phase

• Agreement is signed by the senior representatives
  of the organization and the SEI
• This commitment includes the personal
  participation of the senior site manager, site
  representation on the assessment team, and
  agreement to take action on the assessment
  recommendations

15
Activities

• Conducting an executive-level briefing
• Signing the assessment agreement
• Selecting the assessment team

Main
16
Preparation for PLANNING phases

• Preparing for the on-site assessment
• Assessment team receives training
• Plan on-site assessment
• Participants are selected and briefed about the
  assessments process, times, duration, and purpose
  
• The questionnaire can also be filled at this time

17
Activities

• Pre-assessment team training
• Project selection criteria
• Organizational briefing
• Assessment team training

18
Assessment team training

• Process management overview
• Organizational briefing
• Assessment process overview
• Maturity questionnaire
• Review of project responses
• Assessment discussions
• Team-building exercises
• Assessment findings
• Final report preparation
• Planning for the assessment

19
Preparing for the on-site period

• Selecting the assessment participant
• Briefing the assessment participant
• Assessment principles
• Assessment overview
• Detailed schedule
• Filling up the questionnaire by project
  representatives of the selected projects

Main
20
ASSESSMENT

• Fact Gathering and analysis assessment phase
• On-site assessment is conducted
• Takes a minimum of four days most assessments
  take 5-8 days

21
Assessment Structure

• 1st day
• brief senior management and assessment
  participants about the objectives and activities
• complete the assessment questionnaire
• data/information are received/analyzed
• hold discussion

22

• 2nd day
• conduct discussion with the key software
  practitioners provide further insight into the
  software proces
• 3rd day
• formulate findings
• get feedback from project representatives

23

• 4th day
• review finding to ensure assessment team
  understand the issue
• findings are revised and presented
• - ends with formulation of the recommendation
  that address the findings

24
Reporting- reporting phase

• assessment team prepares a formal written report
  and gives formal presentation of the
  recommendations
• usually include
• - scope of the assessment
• - how the assessment was conducted
• - composite organization status
• - examples of any strength found
• - summary of the findings
• - next steps (recommendations)

25

• guidelines for formulation recommendations
• address each key finding
• - limit the number of recommendations
• - make recommendations specific and concise
• - prioritize the recommendations
• - focus on what the recommendations are, not
  how
• they will be implemented
• - make the recommendations realistic
• the final report is the only permanent record of
  the assessment will be used as reference for
  the action plan formulation and execution

Main
26
Assessment follow-up phase

• After receiving the final report, the
  organization spends three to six months
  developing an action plan
• An action plan should be sufficiently detailed to
  provide a clear guide for execution by including
  the followings
• What recommendations will be implemented (action
  tasks)
• What recommendations will not be implemented, and
  why?

27

• How each action task will be implemented
• What kind of resources required to implement each
  action task
• Which person is responsible for each task
• How is the action task scheduled

28

• The action plan should be reviewed with senior
  management and should specify regular checkpoints
  for periodic management reviews
• After 18 months, a reassessment or
  self-assessment is done by the organization to
  determine progress

29

• The follow-up assessment has the following
  purposes
• To assess the progress that has been made
• To establish a target for completion of the most
  important actions
• To establish new priorities for continued
  improvement

30
SEI SOFTWARE CAPABILITY EVALUATION (SCE)

• SCE sometimes referred as capability
  determination
• A method for evaluating the software process of
  an organization to gain insight into its software
  capability
• Focus on the management aspects
• SCE team evaluates the organizations software
  project development practices against the key
  process area (KPAs) of the CMM to determine
  whether the organization follows a stable,
  predictable software process

31
PHASES OF SCE

• Consists of five phases
• 1.  Evaluation start
• Performed by the sponsoring organization
• Determining the attributes of the desired
  software product and the project required to
  produce it
• Determining the process capability that is most
  applicable for the planned development
• Selecting the SCE team

32

• 2. General Preparation
• Performed by the SCE team
• Identifying areas where the development
  organization lacks experience
• Defining the scope of the SCE to be investigated

33

• 3. Specific Preparation
• Performed by the SCE team
• Selecting projects for evaluation
• Preparing specific topics corresponding to the
  sub-process areas for evaluation
• Coordinating preparation for the site data
  collection activities 

34

• 4. Site Data Collection
• Performed by the SCE team
• Visiting the site and investigating each critical
  sub-process are in enough depth to determine
  their strengths, weaknesses and observable
  improvement activities
• Determining strengths, weaknesses, and
  improvement activities through interviewing and
  document reviews

35

• 5. Findings
• Performed by the SCE team
• Consolidating the decisions made in phase 4
  relating the findings to key process areas and
  sub-process area
• Expressing the findings as KPA strength and
  weaknesses and improvement activities
• Conclusion and completion

36
THE SEI MATURITY QUESTIONNAIRE

• Assessments instrument in the Fact Gathering
  phase
• Features in questionnaire CMM version 1.1
• - Focuses solely on process issues
• - Organized by CMM KPAs and cover the 18 KPAs of
  CMM version 1.1
• - Addresses the goals of each KPA, but not all
  of the key practices
• - Includes glossary of terms and KPA
  descriptions
• - Ample space is provided beneath each question
• - There is room for customizing the questionnaire

37
USE OF THE QUESTIONNAIRE

• Serves primarily to identify issues to be
  explored further though further interviews and
  discussion
• Analysis of the responses provides focus areas
  for further discussion and fact finding
  activities, to request supporting information
• Should be filled by in by the assessment
  participants
• They should be briefed on the structure, contexts
  and the guidelines for answering the questions

38

• The answer could be of the following
• YES
• The practice is well established and consistently
  performed
• NO
• The practice is not well established or is
  inconsistently performed
• DOES NOT APPLY
• You have the required knowledge about the project
  or organization and the question asked, but you
  feel the question does not apply to your project

39

• DONT KNOW
• You are uncertain about how to answer the
  question

40
BENEFIT OF USING THE QUESTIONNAIRE

• Provide a detailed framework for identifying key
  issues for discussion later in the assessment
• The questionnaire narrows the focus of the
  assessment team to selected key process area, and
  helps the team to focus on areas most important
  to the organization
• The questionnaire identifies areas about which to
  request supporting information

41

• Establish an initial rating of an organizations
  software process maturity level
• Uses as a springboard to start the software
  process improvement programme and set it in the
  right direction
• The assessment provides a baseline for the
  current state of the software process in the
  organization. This baseline will provide the
  foundation upon which to base the improvement
  actions

42

• Using consistent assessment tools and models
  across assessments is likely to produce ratings
  and result in a reliable, consistent and
  repeatable manner
• Provide an entry point through which the
  assessment team can get the truth about what
  really goes on in the software organization

43
CUSTOMIZING THE SEI QUESTIONNAIRE

• For self-assessment purpose
• Why customization? Need to align to
• Objective of assessment
• Overall direction of SPI Programme
• Business and software environment
• Package acquisition strategy

44

• Actions during customization
• Add question to cover additional key practices in
  KPA
• Add questions to cover non-process aspect.
• Customizing the number of KPA covered by
  questionnaire
• Also the CMM focuses solely on process-related
  issues. Sometimes it may be desirable to include
  other domains such as the organization or the
  technology.

45
Summary

• Assessments can be performed by using the CMM as
  a process model to assess the processes
• Use SEI Questionnaire as an instrument