Goddard Space Flight Centers

1

• Goddard Space Flight Centers
• Software Development Process Improvement Project
• CMMI Assessment Experiences
• Sally Godfrey
• Engineering Project Manager

2
Agenda

• Background
• GSFC Plan
• Phase 1
• CMMI Assessment -SE
• Goals/Scope
• Assessment Process
• Lessons Learned/Next Steps

3
Authority

• Directed by NASA Chief Engineer
• the SEWG is expected todefine and pilot a
  methodology for assessment of the systems
  engineering capability, which addresses knowledge
  and skill of the workforce, processes, and tools
  and methodology.
• Deputy Chief Engineer for Systems Engineering
  (Nov. 1, 2000)
• Promoted by the agency Software Working Group
  (SWG)
• Software Initiative being implemented across
  agency
• CMM and CMMI-SW programs at all Centers
• Studied by the agency Systems Engineering Working
  Group (SEWG)
• Assessment data from GSFC will be evaluated by
  the SEWG to determine if CMMI is appropriate for
  Systems Engineering implementation agency-wide.

4
GSFC Software Process Improvement Plan

• Purpose To define and detail the implementation
  of the Goddard Space Flight Center (GSFC) process
  improvement effort.
• Focus - Improving the processes and practices in
  use at GSFC using the Capability Maturity Model
  Integrated (CMMI) levels of maturity (ML) as a
  measure of progress.
• Scope - Process improvement effort that will be
  undertaken with the goal of raising GSFC from its
  current state to a CMMI Defined maturity level
  (level 3). All projects defined by NPG 7120.5
  and identified by GSFCs Center Director will
  participate in this effort.
• Goals -
• Increase percentage of projects that are on-time
  and within cost by at least 10
• Increase productivity by at least 5
• Decrease cycle time by 10-20
• Reduce error rate after delivery by at least 20

5
Implementation Phases in GSFCs Improvement Plan

• Phase 1(Year 1) Assessment phase to assess the
  gaps and determine what it would take to improve
  capability (both cost and time). Conduct few
  pilot project(s) to assess the implementation
  approach of the process improvement effort.
• Phase 2 Staged implementation of process
  improvement effort on all NPG 7120.5 projects
  with a goal of improvement in areas of greatest
  benefit (CMMI Defined maturity level (level 3)?)
• Phase 3 Sustain and continue to improve the
  process improvement gains in the previous phases.

6
Phase 1 (Current Year) Goals

• Benchmark several areas against the CMMI model
  (Where are we?)
• Learn what is involved in trying to use CMMI as a
  model for improvement (How hard is it? Does it
  make sense?)
• Get a basis for estimating the cost of a process
  improvement program that achieves CMMI Level 3
  (How expensive is it?)
• Assess our planned implementation approach (Are
  we doing this the right way?)

7
Pre-Assessment Areas Selected
Pre-Assessments -Flight Software
(11/01) -Project Level-Focus on Systems
Engineering Acquisition (4/02) -Ground
Software (8/02)
8
Appraisal Goals for Systems Engineering
Pre-Assessment

• Determine the applicability of the CMMI Model for
  evaluating systems engineering and acquisition
  activities at Goddard
• Baseline the systems engineering organization
  against the requirements in the model
• Gain experience in the use of the model as a
  baselining tool

9
Key Points for Pre-Assessment

• The pre-assessment used the CMMI SW/SE/IPPD/SS
  V1.1 as the appraisal reference model.
• Pre-assessment conducted less formally than
  SCAMPI -more reliance on interviews, less on
  verification of information and document review.
  No level ratings reported
• As prescribed by the CMMI appraisal method,
  findings are the result of team consensus,
  supported by multiple data points from multiple
  sessions.
• Results of a Capability Maturity Model Integrated
  (CMMI) pre-assessment are reported as findings
  of strengths and improvement opportunities in the
  CMMI Process Areas.

10
Pre-Assessment Process Flow
Day 1
Day 4
Pre On-Site
Post On-Site
Day 5
Day 2 - 3
Analyze Requirements
CMMI Overview Training
Conduct Interviews
Prepare Final Findings
Produce Reports and Support Follow-on Activities
Conduct Interviews and Review Documents
Develop Appraisal Plan
Review Documents
Deliver Final Findings
Review Presentations
Consolidate Information
Select and Prepare Team
Consolidate Information
Conduct Interviews
Deliver Draft Findings
Obtain Organizational Information
Consolidate Information
Select and Prepare Participants
Prepare for Data Collection
11
Pre-Assessment Scope

• The scope was limited to in-house activities and
  the management of external contractor activities.
• Per the assessment process, three projects have
  been evaluated as representative of the product
  development capability of the Goddard Space
  Flight Center.
• CMMI Components Reviewed
• Maturity Levels 2 and 3 Process Areas
• Specific Goals
• Specific practices are evaluated to determine
  specific goal coverage based on evidence of
  weaknesses, improvement activities, strengths and
  alternative practices.

12
CMMI Components (per Process Area) Not Reviewed

• Actual documented process being used on
  projects
• Activities, process inputs outputs,
  deliverables, roles responsibilities,
  measurements, work instructions, templates,
  tailoring, why when, etc.
• Training for use of process
• Use of process and adherence to process
• Planning and monitoring of process
• Providing resources for process

13
Appraisal Participants -Not Just Systems Engineers

• Senior Systems Engineers
• Project Managers
• Systems Engineers
• IVV Managers
• Requirements Managers
• Software Managers
• Instrument Managers
• Quality Assurance Representatives
• Configuration Management Representatives
• Engineering Process Group (EPG) Representatives
• Training Coordinators
• Schedulers
• Contracting Officers COTRs

14
Level 2 Process Areas

• Requirements Management
• Project Planning
• Project Monitoring Control
• Supplier Agreement Management
• Measurement Analysis
• Process Product Quality Assurance
• Configuration Management

15
General Process Requirements for Each Process
Area at Level 2

• Document project level processes so that all
  projects have a starting point for these
  activities.
• Plan and manage these process activities,
  including
• Institute an organizational policy
• Plan the process
• Provide resources
• Assign responsibility
• Train people
• Manage configurations
• Identify involve relevant stakeholders
• Monitor control the process
• Objectively evaluate adherence
• Review status with higher level management

16
Level 3 Process Areas

• Requirements Development
• Technical Solution
• Product Integration
• Verification
• Validation
• Organizational Process Focus
• Organizational Process Definition
• Organizational Training
• Integrated Project Management
• Risk Management
• Integrated Teaming (not assessed)
• Integrated Supplier Management
• Decision Analysis and Resolution
• Organizational Environment for Integration (not
  assessed)

17
General Process Requirements for Each Process
Area at Level 3

• Document organization level processes (and
  tailoring guidelines) so that all projects have a
  starting point for all process activities.
• Plan and manage these process activities,
  including
• Institute an organizational policy
• Plan the process
• Provide resources
• Assign responsibility
• Train people
• Manage configurations
• Identify involve relevant stakeholders
• Monitor control the process
• Objectively evaluate adherence
• Review status with higher level management
• Collect information for process improvement

18
Initial Impressions from CMMI Use with Systems
Engineering

• CMMI Pre-Assessment provided fairly accurate
  bench-mark of state of system engineering
  activities
• Pre-assessment was aquick-look -Provided a
  wealth of information in a short period of time
  (1 week)
• External involvement of assessors helps
  facilitate cooperation from projects
• Pre-assessment was excellent training for
  internal assessors involved (one SE, one S/W, one
  QA- all EPG members)
• Future pre-assessments and bench-marking could
  now be done by internal assessors (Have
  experience base)

19
Next Steps

• Develop and report CMMI process recommendations
  to the NASA Systems Engineering Working Group and
  others.
• Decide which improvement opportunities should be
  worked based on the Goddard business direction.
• Develop action plans for continuing process
  improvement.

20
Back-up Slides
21
Software Working Group Software Initiative
Implementation Plan

• Goal - To advance software engineering and
  systems engineering practices to effectively
  deliver the highest level of capability to NASAs
  programs and projects.
• Strategies - Process improvement, as well as
  items related to research, safety, reliability,
  and quality, attraction and retention of
  engineers, and improving NASA's software and
  systems engineering knowledge and skills.

22
Goddards Matrix Structure
GSFC
Code 100 Sr. Mgmt. Training
Code 200 Procurement
Code 300 Quality Assurance, IVV
Code 400 Projects
Code 500 Applied Engineering
Code 600 Space Science
Code 900 Earth Science
Project 1
Project 2
Project 3
Software
Electrical
Mechanical
Science Instruments
QA
Contract Officer
23
Requirements Flowdown
IVV NPG 8730
IVV NPD 8730.4
IEEE/EIA 12207
Legend
NPD 7120.5a
NPD 2820.1 NASA SW Policies
Capability Maturity Model Integ. (CMMI)
Existing
NPG 2820
NASA
In process
GSFC
Institutional Policies
PGs
GSFC Lessons Learned
Goddard GPGs Project Plan
ISO 9000
Industry Standards
Domain- Specific Procedures

Systems Eng. Mgmt Plan
Software Mgmt Plan
24
Infrastructure
Dr. Linda Rosenberg Arthur F. Obenschain
(500) Charles S. Vanek (100) Judy Bruner
(300) Joseph Hennessy (580) Dolly Perkins CIO
(400) John Dalton (600) Jerome Bennet
(900) Nelson Keeler (IVV) Jim Andary (530) Sally
Godfrey (ad hoc)
Sara (Sally) Godfrey
25
Within GSFC
26
EPG Training

• Training received
• Overview NASA SW Initiative and GSFC
  Implementation Plan
• 3 day SEI CMMI course
• Assessment expectations (CSC, AF)
• Metrics (GSFC)
• 2 day SEPG (NASA HQ/ SEI approved)
• Risk Mgt (GSFC NASA)
• Documentation structure of GPGs translated to
  CMMI
• Tools evaluations
• Additional
• Sally Godfrey 5 day SEI CMMI Intermediate
  course
• 2 members EPG SEI Mastering Process Improvement

27
MOG

• Provide oversight and direction to the EPG and
  AMG and assist in establishing priorities
• Work with the EPG in communicating process issues
  and industry practices to GSFC senior management
• Represent their constituent organizations in
  reaching consensus on GSFC institutional software
  policies and standards for both in-house and
  contractor-supplied software
• Review and concur on all GSFC software and system
  policies and guidelines prior to final
  publication

28
EPG

• For the pilots and during the rollout to other
  GSFC entities the EPG will
• Lead the continuous definition, maintenance and
  improvement of software process policies
  procedures and best practices including the
  development and maintenance of the GSFC software
  development process improvement plan
• Facilitate software process assessments
• Arrange for and support training and continuing
  education related to process improvements for
  engineers, line managers, project management, and
  GSFC senior management
• Define and maintain metrics to track, monitor,
  and assess the status of focused improvement
  efforts and pilot studies
• Provide status information and evaluations of
  the improvement activities to all levels of
  management
• Lead the institutional response, where
  appropriate, to software/systems-related
  Nonconformance Reports
• Maintain a collaborative working relationship
  with practicing software/systems engineers to
  obtain, plan, and install new practices and
  technologies
• Provide software engineering consultation to
  development projects and management

29
AMG

• Develop and maintain the GSFC Develop Software
  and Systems Products web site which includes the
  software development process improvement library,
• Develop and maintain a database of GSFC software
  process and product metrics,
• Act as the clearinghouse for software metrics
  reported to NASA HQ,
• Develop insights into the metrics sources that
  will enhance the consistency and effectiveness of
  interpretation,
• Maintain a database of GSFC software product
  characteristics in order to understand process
  metrics, encourage software reuse, and assist in
  identifying special expertise, and
• Establish and manage a service that provides
  software engineering tools to projects in cases
  where a single GSFC vendor interface and
  institutional supplier is appropriate.

30
References

• CMMI Software Engineering Institute at
  www.sei.cmu.edu
• International Council of System Engineering
  www.incose.org
• Project Management Institute www.pmi.org
• Quality Assurance Institute www.qaiusa.com
• American Society for Quality www.asq.org
• Crosstalk www.stsc.hill.af.mil
• Data Analysis Center for Software
  www.dacs.dtic.mil
• Process Asset Library Example sepo.spawar.navy.mi
  l/sepo/