Systems Engineering Revitalization Through CMMI

1
Systems Engineering Revitalization Through CMMI
Michael T. Kutch, Jr. SPAWAR Systems Center
Charleston (SSC-C) Head, Intelligence
Information Warfare Systems Engineering
Department National Competency Lead for I/A 5.8
Deputy National Competency Lead for ISR/IO 5.6
CDCA Government and Industry Conference November
7, 2007
Improving operational effectiveness through C4ISR
common integrated solutions
2
Presentation Outline

• Vision and Strategy
• SE Revitalization
• Why
• How
• Tools
• Lessons Learned
• Results

3
Process Improvement and Systems Engineering
Strategy - 2003

• Vision
• Develop and maintain a World Class Systems
  Engineering Organization
• Approach
• Achieve Command-wide operational consistency
• Based on ISO 15288 systems engineering
• Based on ISO 12207 software engineering
• Measure using best practices of CMMI
• Goals
• CMMI Maturity Level 2 by April, 2005
• CMMI Maturity Level 3 by April, 2007

Both Goals attained on schedule 1st SPAWAR
Systems Center to Achieve ML2 and ML3 New Goal
Maturity Level 4 by 2010
4
Which one is World Class?
When you want it done right, Who do you want
working on it ?
Cutting corners, undisciplined, untrained
Rigorous processes, Skilled resources
Permission to use Redneck Mechanic photo received
from Dave Lilligren, 3/9/2007 Permission to use
NASCAR Technical Institute photo received from
Popular Mechanics, 3/16/2007
5
The Need for ChangeTop Five Systems Engineering
Issues

• Lack of awareness of the importance, value,
  timing, accountability, and organizational
  structure of SE on programs
• Program teams driven by cost and schedule, not
  disciplined SE
• Emphasis on speed fix in next spiral
• SE inadequately considered in decisions
• Risk poorly defined and managed
• Qualified resources are generally not available
  within government and industry for allocation on
  major programs
• Insufficient SE tools and environments to
  effectively execute SE on programs
• System complexity is ever increasing
• Need a structured methodology and adequate tools
  to understand and manage the complexity
• Poor initial program formulation
• Requirements definition, development, and
  management is not applied consistently and
  effectively

From NDIA Study, January 2003 Adapted from
Technical Excellence through Systems
Engineering by Mark Schaeffer - Principal
Deputy Director, Defense Systems Director,
Systems Engineering OUSD (ATL)
6
The Overrun Problems
USAF Study - 2004

• 3 main reasons for overruns
• Unstable requirements
• Inadequate Systems Engineering
• Program budget instability

Adapted from NDIA SE Conference 2004 Opening
Remarks by Bob Rassa, General Chair
7
Plan to Revitalize SE

• Government needs to be more involved and
  accountable for the Systems/Software Engineering
  process
• Policy and Guidance
• Training and Education
• Assessment and Support
• Industry partners need more rigor in their SE
  SW processes
• Necessary in order to
• Properly define and verify requirements
• Identify and analyze risk factors
• Evaluate alternate solutions
• Accurately estimate and schedule work

Planning Communication Execution -
Implementation
8
CMMI Capability Maturity Model Integration

• SSC-Charleston chose to implement CMMI because
  it provides a structured model for process
  improvement and is used to measure and improve an
  organizations ability to successfully manage
  complex systems engineering and software
  projects.
• The model recognizes excellence in business
  practices, as measured against a set of demanding
  criteria.

SEI has reported quantitative evidence showing
how CMMI-based process improvement can result in
improvements in cost, schedule, quality, customer
satisfaction and return on investment.
9
Classic System Engineering Vee DiagramEmbodies
the ISO/IEC 15288 Standard
CMMI Process Areasalign with SE Vee
Understand User Requirements, Develop System
Concept and Validation Plan
Demonstrate and Validate System to User
Validation Plan
RD
Val
TS
Ver
Develop System Performance Specification And
System Validation Plan
Integrate System and Perform System Verification
to Performance Specifications
PI
Systems Engineering
Expand Performance Specifications into CI
Design-to Specifications and CI Verification
Plan
Assemble CIs and perform CI Verification to CI
Design-to Specifications
Decomposition and Definition
Integration and Qualification
Evolve Design-to Specifications into Build-to
Documentation and Inspection Plan
Inspect to Build-to Documentation
Fabricate, Assemble, and Code to Build-to
Documentation
time
Design Engineering
RSKM
DAR
PP
PMC
CM
REQM
PPQA
MA
SAM
IPM
10
Critical Success Factors Identified
11
Process Improvement Infrastructure Organization
Business Board
Vision
Mike Kutch SE/CMMI Champion

External Liaison
Bruce Carter Dir. Engr. Operations
Strategy
Management Steering Group (MSG)
Staff
Engineering Process Office (EPO)
Tactical Implementation
Define and ManageStandard Processes
Enterprise Process Group (Ent PG)
AD
SE IPT
WFO IPT
RISK
PM IPT
Facility IPT
CM IPT
RDTE IPT
LSS
PPQA IPT
Comms Netw Dept EPG
Cmd/Ctrl Dept EPG
ISR/IO Dept EPG
NetCentric SE Dept EPG
LOG IPT
Corporate Engineering Process Group (EPG)
Corporate Business Process Group (BPG)
TecInn IPT
12
Engineering Process Office (EPO)
Engineering Process Office (EPO)

• Supports the Director of Engineering Operations
• Developed Policies
• Policy for each CMMI Level 2, 3, 4, 5 Process
  Area
• Developed Standard Process Manuals
• Top Level
• Systems Engineering
• Software Development
• Software Maintenance
• Supporting Processes
• Process Manual for each CMMI Level 2, 3, 4, 5
  Process Areas
• Additional process documentation as needed
  Reviews, Tailoring, etc
• Develop plan templates
• Coach and mentor selected projects
• Build tools
• Develop and deliver training
• Perform interim assessments

13
Typical IPT
Integrated Process Teams focus on specific
process areas ie Systems Engineering or Logistics
EPO Rep(s)
Code 09K Rep(s)
Process x IPT
Code 09A Rep(s)
Code 09C Rep(s)
Code 50 Rep(s)
Code 80 Rep(s)
Code 60 Rep(s)
Code 70 Rep(s)

• Representatives selected by Dept EPG Chairperson
• Subject matter experts from each department or
  division involved in specific IPT process area
  including contractors

14
Selecting Initial Projects

• Start with interested projects
• High Sponsor interest
• Strong need/desire to improve
• Met Guidelines that would yield benefits
• Systems or software engineering effort
• SSC-C performs the Project Management function
• Multi-year
• Over 2M per year

EPO provided frequent mentoring and coaching and
participated in projects CMMI status meetings
Industry partners were highly integrated into
projects process planning, implementation, and
assessment process
15
Tools and Resources Developed

• Training
• Web-based training modules
• On-site, instructor led training
• CMMI/SE Website
• Engineering Process repository
• Documentation
• Tools
• Samples and Templates
• ePlan Builder
• TurboTax for Generating Plans

Tools and resources available to all SSC-C
employees and contractors via the SSC-C intranet
(CAC card)
16
SE PI Training Architecture
SEI Intro to CMMI 3-day
PI WBT
All WBTs available from SSC-C intranet
Foundation of PI and CMMI
Engineering Project Process Mgmt Workshop
SE Fundamentals
SE 101 WBT
Core SSC-C project and engineering
processes (Level 2 and 3)
Architecture Dev. WBT
Intro to Software Engineering
SE for Managers
Risk Management WBT
SEMP Workshop
Subject Matter Experts - Use commercially availabl
e on-site classes
Quality Engineering
Requirements Analysis
Risk Management
Existing
Existing - revise
Appraisal Assessment Workshop
New - develop
Prepare Projects for BSC or SCAMPI
New - buy
17
EPO website provides access to all of SSC-Cs
organizational process assets
Approximately 100 pages of content over 1000
documents available
18
ePlan Builder Tool

• ePlan Builder tool
• An interactive, web-based application that leads
  the user through a structured interview process
  (like TurboTax) to generate a CMMI-compliant
  plan
• Includes standard, consistent text
• Generates an initial project-specific document
• Project Management Plan (with Work Breakdown
  Structure)
• Configuration Management Plan
• Process and Product Quality Assurance Plan
• Requirements Management Plan
• Measurement and Analysis Plan
• Supplier Agreement Management Plan (by end of
  2007)
• Systems Engineering Plan (DoD SEP Format)

19
What We Have Accomplished

• Process Focus
• Defined Policies and Processes
• Aligned with DoD and SPAWAR guidance
• Aligned with industry standards and CMMI model
• Built organization structured around processes
  and process improvement
• Training is Critical
• Providing Fundamentals of Engineering for new and
  old professionals
• Developed web-based training for self-paced and
  refresher training
• Defining a structured technical career
  development path for engineers
• Tools for the Engineers
• Developed ePlan Builder application to generate
  planning documents
• Developed templates, checklists, and web-based
  document repositories to link standards and DoD
  guidance to day-to-day tasks and processes

Early and persistent Systems and Software
Engineering applied to programs and projects
20
Lessons Learned

• Senior Management support is critical to success
• Training
• Everyone needs to be engaged train the masses
• Specific training for process owners/subject
  matter experts
• Utilize Teams (IPTs) as champions of specific
  processes
• Multi-department representation
• Change agent mentality
• Process-focused charters
• Resource Properly
• Implement with projects that want to improve, can
  benefit from efforts, and that recognize own
  weaknesses
• EPO staff provided skilled coaching, resources,
  support, and tools
• Project members learned by doing and maintaining
• Goals and Publicity
• Keep goals to sizable bites (projects)
• Publicize successes Share best practices

21
Is the SE Revitalization Working ?

• Recognition of SE and CMMI effort
• 1st SPAWAR Systems Center to achieve Maturity
  Level 2 (2005)
• 1st SPAWAR Systems Center to achieve Maturity
  Level 3 (2007)
• Multiple presenter at NDIA SE and CMMI
  conferences
• High interest in Tools, Training, and
  Implementation

22
Is the SE Revitalization Working ?

• Business Results
• SCN They see us as a model and want to increase
  our efforts.
• Automation Program We had hundreds of sites and
  there was a need for a structured organization to
  put a wrapper around that and control it. CMMI
  became the wrapper.
• CICS CMMI was key to achieving the project
  goal.
• VIDS The VIDS failure (2000) motivated
  implementing CMMI because the team needed to
  change course or the customer would have no
  confidence in system development. It was a
  tremendous success
• Others Asking for Help
• PMS 408 CREW program
• SESG / NAVAIR / NAVSEA
• Marine Corp Quantico
• Air Armament Center, Eglin AFB

23
Going Forward

• Increase usage of tools across departments/project
  s
• Continue internal CMMI Level 3 mini assessments
• Enhance/Expand Organizational Measurement
  Repository (OMR)
• Command and Department Project Reviews process
• Look at quality of plans and implementation of
  best practices
• Reviews of project status by management driven by
  project metrics
• More Peer Reviews to measure saves
• Better tailoring guidance for smaller projects
• Require/Encourage process maturity from industry
  partners

Begin CMMI Maturity Level 4/5 implementation
24
Any Questions for me?
Questions for you !
Do you have a documented Systems Engineering or
Software Development process? How aware are you
of the CMMI model for process improvement? How
are you developing and refining requirements with
your customers? What can we do to help you help
us?
Contact Information Michael T. Kutch,
Jr. SPAWAR Systems Center Charleston Email
michael.kutch_at_navy.milPhone 843-218-5706