Shared Knowledge and Information Flow in Systems Engineering

1
Shared Knowledge and Information Flow in Systems
Engineering
NASA Goddard Space Flight Center Systems
Engineering Seminar March 3, 2009

• Socio-Cognitive Analysis of the GSFC Mission
  Design Laboratory

Mark S. Avnet Ph.D. Candidate Engineering Systems
Division Massachusetts Institute of Technology
2
Who Am I and Why Am I Here?

• S.B. in Physics, MIT, 2001 M.A. in Space Policy,
  GWU, 2005
• Software Engineer, 2001 2003 NASA HQ, 2004
  2005
• Observed an Interesting Phenomenon
• Decisions in space systems development require
  integration of perspectives policy, scientific,
  engineering, public, etc.
• Systems engineering takes into account the unique
  views of each, but the engineer is taken to be
  outside of the stakeholder framework.
• Ph.D. in Engineering Systems, MIT, 2009
• Research addressing this issue
• Focus of this talk contributions to SE here at
  GSFC

3
Perspectives on Space Systems Design
Source Robinson, G.L., Systems Engineering
Initiatives at NASA, Goddard/SMA-D Education
Series, 25 Sept 2008.
4
Structure of the Presentation
1
2
3
4
5
Overview of the Mission Design Lab
Part 1
6
GSFC Integrated Design Center
Integrated Design Center (IDC)
Focus of this Talk
Mission Design Lab (MDL)
Instrument Design Lab (IDL)
7
The Mission Design Lab
8
The MDL Structure and Products
http//idc.nasa.gov/mdl/products.cfm
http//idc.nasa.gov/idc/services.cfm
9
The MDL Roles and Facility
10
MDL Design Study Observations

.
Typical Studies
11
Analysis of the Design Process
Part 2
12
The Design Structure Matrix (DSM)
Task A depends on information from Task G
Tasks D and E must be done concurrently
13
Design Process Analysis
Series
Coupled
Phases of the Design Life Cycle
Starting Assumptions
14
Modeling the MDL Design Process
15
Partitioning the DSM The Conceptual Design
Lifecycle
Requirements Definition Phase
Engineering Design Phase
Integration Phase
Maintenance and Support Phase
Costing Phase
16
Critical Design Trades and Interdependent
Disciplines
13 Core Loop Types
Spacecraft Bus Loop
Propulsion Sizing Loop
Stabilization Loop
Ground Segment Loop
Data Loop
Power System Electronics Loop
Power Loop
Electrical Heating Loop
Propulsion Thermal Control Loop
Radiator Operation Loop
Reentry Loop
Computing Reliability Loop
Radiation Shielding Loop
17
Tearing the DSM Indentification of Starting
Assumptions
Tear the Design Budgets
Power Budget
Mass Budget
Reliability Budget
18
The Torn DSM MDL Process with Starting
Assumptions Made
Requirements and Assumptions Phase
Orbit Determination Phase
Sequential Engineering Design Phases
Iterate
Integration Phase
Costing Phase
19
The Core of Interdependent Disciplines
Flight Dynamics
Location
Mission Operations
Avionics
Location
Communications
Electrical Power
Location
Mechanical
Thermal
Ground Segment Loop
Data Loop
20
Insights from DSM-Based Analysis
The Design Structure Matrix is a powerful tool
for describing and analyzing the space systems
design process.
(Results for your system may vary.)
21
A Model of Shared Knowledge
Part 3
22
Mental Models of the System
Mental Models
Mechanisms whereby humans are able to generate
descriptions of system purpose and form,
explanations of system functioning and observed
system states, and predictions of future system
states
Rouse, W.B. and N.M. Morris (1986). On Looking
Into the Black Box Prospects and Limits in the
Search for Mental Models. Psychological
Bulletin 100(3) 349363.
Shared Mental Model (SMM)
SMM
Condition in which two people utilize the same
underlying mechanisms or at least utilize
mechanisms that lead to similar descriptions,
explanations, and predictions
Team Member
Team Member
23
Measuring Mental Models
Survey Question on Major Design Drivers
24
Measuring Shared Mental Models
Mental Model Sharedness, Sx,y , is defined as
Ratio of common choices to total choices
Dx of drivers selected by person x Dy of
drivers selected by person y Dx,y of drivers
selected by both x and y
Sx,y
Team Member x
Team Member y
25
Social Network Analysis

• A set of tools and techniques for analyzing a
  large group of entities (nodes) and the structure
  of interactions and/or relationships among them
  (edges).

Node
Edge

• Node Design Team Member x or y
• Edge Shared Mental Model between x and y
• Edgeweight Value of Sharedness, Sx,y

26
Dynamics of Shared Knowledge
Post-Session
Pre-Session
CSMM structural similarity (edge-by-edge
correlation)
Change in Shared Knowledge
?
27
Dynamics of Shared Knowledge Relationship to
System Attributes
28
Integrated Analysis People and Process
Part 4
29
Content of Shared Knowledge Perceived Importance
of Drivers
IP,Comm proportion of team checking
Communications
30
The Communications Subsystem An Indicator of
Shared Knowledge
Recall the Central Role of Communications in the
Design Process
31
Measuring Team Coordination
Expected Interaction Matrix Based on Core Loop
Types in the Partitioned DSM
Actual Interaction Matrix Based on Survey Data
of Interactions for Each Study (Study 3 Shown
Here)
32
Socio-Technical Congruence
Congruence Matrix Overlay of Expected and
Actual Interactions
N number of cells Nb number of blank
cells N total number of cells
33
Dynamics of Shared Knowledge Relationship to
Team Coordination
34
The Typical MDL Process Recommendations in
Discussion
People
Process
Tools
Period of learning and consensus building
Resolve orbit determination trades
DSM-based process automation software
Determine starting assumptions
Facility
Sub-teams based on interdependent disciplines
Lab layout based on interdependent disciplines
Design sequentially then iterate
Proposed Standard Design Process Model under
Development in Conjunction with the MDL
35
(No Transcript)
36
The People Behind This Work
Annalisa Weigel, MIT, Thesis Advisor NASA
Graduate Student Researchers Program
(GSRP) Deborah Amato, Former IDC Systems
Engineer Jennifer Bracken, IDC Systems
Engineer Tammy Brown, IDL Team Lead Bruce
Campbell, IDC Manager Anel Flores, MDL Systems
Engineer Gabriel Karpati, Former IDC Systems
Engineer John Martin, MDL Team Lead Mark Steiner,
SESAC Branch Head
IDC Support Staff Felicia Buchanan-Jones, Dawn
Daelemans, Elfrieda Harris, Erica Robinson, Ed
Young
12 MDL Customer Teams
And, of course, the MDL engineers, whose
sustained participation made this work possible.
37
Thank You
38
Backup
39
Building the DSM for the MDL
Although collocation accelerates the pace of
design activity, it also presents an obstacle to
formal analysis and process improvement. DSM
construction must account for this.

• Parameter-Based DSM

• Steps of DSM Construction in the MDL
• Preliminary Interviews
• Surveys on Design Sessions
• Structured Interviews
• Verification and Validation

• Guiding Principles for DSM Construction in the
  MDL

• Document maximal flow for a typical design
  session
• Include only deliberate and purposeful
  information flow
• Abstract two-way negotiation-type interactions

40
Data Collection on Mental Models

• Survey Data on Major Design Drivers
• Team members indicate whether each of a set of
  issues drives the ultimate design.
• Simple Example with Only Four Possible Drivers
• Cost
• Schedule
• Performance
• Science

24 16 Possible Mental Models
41
Filtering Out Random Responses A Cutoff
For Shared Mental Models
SMMx,y 0
SMMx,y 1
x and y do not share mental models to any
greater extent than two people with no prior
knowledge of the task answering at random
35 Possible SMMs
42
Quantifying Shared Knowledge Edge Weights in a
Social Network

• Surveys Distributed 20 Drivers and 1,771
  Possible SMMs
• Network Edge Weights on a 1-4 Scale
• Time Dependence of Shared Knowledge
• 12 Design Sessions Observed
• Pre- and Post-Session Data Collected for Each

.
43
Dynamics of Shared Knowledge Relationship to
System Attributes
44
Propulsion Subsystem and Mission Type
45
Team Coordination and Shared Knowledge in the
Team
46
Proposed Standard Design Process
Model
47
Contributions to the Research

• Product Development
• Guiding Principles for Building a Design
  Structure Matrix in a Rapid Collaborative Design
  Environment
• Method for Converting a Parameter- to a
  Team-Based DSM
• Cross-Functional Teams and Shared Mental Models
• Scalable Network Model of Shared Knowledge in
  Engineering Design
• Metric that Captures Dynamics of Shared Knowledge
• Systems Engineering and Space Systems Design
• System-Level Representation of the Entire Design
  Process
• Analysis of the Role of People in the Process
• Standardized Design Process Based on Both of the
  Above
• Explicit Connection between Organizational/Social
  Psychology and Systems Engineering Best Practices

48
Future Work

1. Apply Methods to the Instrument Design Laboratory
   and to Other Similar Design Centers Apply Both
   DSM and SMM Work to Longer Development Programs
2. Build DSM with Types and Strengths of
   Dependencies
3. Time Series Analysis 1 to 2 Surveys Each Day
   Tracking the Evolution of SMMs Over Time
4. Measure SMMs Based on Other Forms of Knowledge in
   Addition to Task Team, Process, Context,
   Competence
5. Network Analysis of Design Sessions
6. Experimental Approach with a Learning Period
   Structured in Various Ways and Several
   Combinations of Number and Length of Design
   Iterations