Safety Management Systems for Design and Manufacturing Organizations

1
Federal Aviation Administration
Concepts and Implementation for the Pilot Project
Participant Part II - Implementation
MSMS Team Member FAA Aircraft Certification
Service
2
Agenda

• Related SMS Activities
• SMS Fundamentals
• SMS Pilot Project Implementation
• Guidance Tools
• Preliminary Gap Analysis meeting
• Working together during the Pilot Project
• Questions

3
Pilot Project Objectives

• Provide recommendations and analysis to support
  of development of FAA regulations and associated
  policies
• Validate or refine the DM SMS Framework
• Validate or refine guidance materials
• Propose the appropriate applicability and
  scalability of SMS
• Refine assessment and oversight policies

4
MSMS Organizational Structure
AIR-150
GAP Analysis Framework Team
MSMS Team Lead
Communication Lead
ONE Team
MSMS Pilot Project Lead
Share point Focal
Data Collection Analysis Focal
Pilot Project
FAA Participant Management Teams (PMT)
FAA Participant Management Teams (PMT)
FAA Participant Management Teams (PMT)
FAA Participant Management Teams (PMT)
FAA Participant Management Teams (PMT)

• PMT Includes
• MSMS Team Member as FAA PMT Lead
• Manufacturing representatives (Field)
• Engineering representatives (Field)

5
(No Transcript)
6
Roles Responsibilities

• DM SMS Pilot Project Lead
• Serves as the point of contact for the DM Pilot
  Project
• Coordinates and manages the FAA Participant
  Management Teams
• Provides final acceptance to exit a Level
• Pilot Project Lead Kurt.Krumlauf_at_faa.gov

7
Roles Responsibilities (Continued)

• FAA Participant Management Team (PMT)
• Consists of members of the MSMS Team FAA field
  personnel from MIDOs and ACOs
• Participate in meetings / telecons and site
  visits
• Participate in procedure and process assessments
• Provide status/updates to the oversight
  organization throughout the Pilot Project
• Company PMT
• MSMS Team Kurt Krumlauf (lead) and Steve Slagle
• LAACO Cheyenne DelCarmen and Joe Costa
• Phoenix MIDO Don Henchar

8
Roles Responsibilities (Continued)

• FAA PMT Lead
• Leads the Participant Management Team
• Provides direction, guidance and assistance to
  PMT and participant
• Coordinates formal interaction with the
  participant, such as meetings, telecons, output
  deliverables, reviews of exit criteria
• PMT Lead for Company Kurt Krumlauf
  kurt.krumlauf_at_faa.gov [phone]

9
Roles Responsibilities (Continued)

• Participant Point of Contact (POC)
• Company SMS Project Manager (or similar)
• Serve as POC for SMS activities
• Serve as primary contact to the FAA

10
AIR SMS Implementation Strategy

• SMS will be implemented in 4 progressive Levels
• This approach enables an organization to
  establish the basic building blocks (levels 1 and
  2) before implementing the more complex
  activities of SRM and SA (levels 3 and 4)
• Development and implementation of the SMS is more
  manageable

11
AIR SMS Implementation Strategy

• Level 1 focuses on an agreement with the FAA for
  a completed gap analysis and development of an
  implementation plan.
• Levels 2 3 will be subject to an assessment
  process by the FAAs PMT
• Level 4 is for the organization to verify the
  performance and effectiveness of their SMS
  management practices and operational processes

12
Defining the Scope of Your Project

• For an SMS to be fully effective it must be
  implemented organization-wide
• However, for the pilot project you may want to
  scale down the project
• Consider the companys products, processes, and
  departments that you would like to include in the
  pilot project
• The scope should be of appropriate magnitude with
  the goal of exiting Level 1 within six months.

13
Implementation Levels
1
Planning Organization
14
1
Planning Organization
15
Level 1 Output Documents
1
Planning Organization

• Management Commitment definition (Letter)
• Organization Safety Policy
• Comprehensive SMS Implementation Plan for entire
  organization through level 4
• SMS Training Plan for all employees

16
1
Planning Organization
Level 1 Exit Criteria

• Evidence of Accountable executives commitment to
  implement SMS, define safety policy and convey
  safety expectations and objectives to its
  employees
• Evidence of adequate resources to implement SMS
• Designate a management representative responsible
  for SMS development
• Define safety-related positions for those who
  will participate in SMS development and
  implementation
• Obtain agreement with FAA on your Detailed Gap
  Analysis and Implementation Plan
• Identify safety competencies and develop a
  training plan
• Complete a exit level one meeting with the FAA

17
Level 2 Output Documents
2
Reactive Processes

1. Objective evidence that SRM processes and
   procedures have been applied to at least one
   existing hazard and the mitigation process has
   been initiated
2. Updated comprehensive SMS implementation plan for
   all elements to take the participant through
   Level 4
3. Updated SMS training plan for all employees

18
Level 2 Exit Criteria
2
Reactive Processes

1. Processes procedures documented for operating
   the SMS to the level of reactive analysis,
   assessment and mitigating actions
2. Develop documentation relevant to SMS
   implementation plan and SRM components (reactive
   processes)
3. Initiate and document voluntary non-punitive
   employee reporting and feedback program
4. Conduct SMS training for staff directly involved
   in the SMS process to at least the level
   necessary for the SMS reactive processes

19
Level 2 Exit Criteria (Continued)
2
Reactive Processes

1. Apply SRM processes and procedures to at least
   one known (existing) hazard and initiate the
   mitigation process to control/mitigate the risk
   associated with the hazard
2. Update the Detailed Gap Analysis on the entire
   organization (pending scope) for all elements of
   the SMS Framework
3. Update the comprehensive SMS Implementation Plan
   for all elements to take the organization through
   Level 4

20
Level 3 Output Documents
3
Proactive Processes

1. Objective evidence that SRM processes and
   procedures have been applied to all Element 2.1
   (1) Hazard Identification and Analysis items
2. Objective evidence that SRM processes and
   procedures have been applied to at least one
   existing hazard and that the mitigation process
   has been initiated
3. Updated comprehensive SMS implementation plan
   for all elements
4. Updated SMS Training Plan for all employees

21
Level 3 Exit Criteria (Continued)
3
Proactive Processes

• Objective evidence that Hazard Identification and
  Analysis has been accomplished
• Objective evidence that SRM processes and
  procedures have been applied to at least one
  existing hazard and mitigation process has been
  initiated
• Updated comprehensive SMS implementation plan for
  all elements
• Updated SMS training plan for all employees

22
3
Level 3 Exit Criteria (Continued)
Proactive Processes

1. Objective evidence of compliance with Element 4.1
   Competencies and Training
2. All applicable SMS processes and procedures have
   been applied to at least one existing hazard and
   mitigation process have been initiated
3. Complete SMS Training for Staff directly involved
   in SMS process to the level of accomplishing all
   SMS processes
4. Complete employee training commensurate with this
   level of implementation maturity

23
Level 3 Exit Criteria (Continued)
3
Proactive Processes

• Objective evidence of compliance with
  Competencies Training
• All applicable SRM processes and procedures have
  been applied to at least one existing hazard and
  that the mitigation process has been initiated
• Complete SMS training for the staff directly
  involved in the SMS process to the level of
  accomplishing all SMS processes
• Complete employee training commensurate with this
  level of implementation

24
Level 4 Expectations
4
Continuous Improvements

• The complete SA process, continuous monitoring
  other SRM processes are functioning
• Verify the performance and effectiveness of the
  SMS management practices and operational
  processes

25
Agenda

• Introductions
• Related SMS Activities
• SMS Fundamentals
• SMS Pilot Project Implementation
• Guidance Tools
• Preliminary Gap Analysis meeting
• Working together during the Pilot Project
• Questions

26
Guidance and Tools

• DM SMS Pilot Project Guide, referred to as The
  Guide
• Developmental Guidance (DG) for DM SMS Framework
• Tools
• Gap Analysis Tool
• Assessment Tool
• Implementation Planning Tool
• AIR SMS Website (SharePoint)

27
DM SMS Pilot Project Guide

• SMS Background and Introduction
• Roles, Responsibilities and Relationships
• Implementation Processes
• Pilot project process for implementation of level
  1
• Pilot project process for implementation of level
  2 3
• FAA Assessment of the SMS levels
• DM SMS Framework
• Information on SRM SA integration
• System Description and Hazard Identification
• Definitions and Acronyms

28
System Description

• Assists an organization to
• Define its system and identifying hazards
• Establish the scope and boundaries of the company
  to which SMS applies
• Appendix D of The Guide
• Currently it provides a summary of a system
  description
• A more comprehensive system description is under
  development and will be included
• Data and information received from the Pilot will
  be used to improve The Guide

29
Developmental Guidance (DG) for DM SMS
Framework

• Purpose
• Provides assistance to an organization in
  developing their SMS using the DM SMS Framework
• Clarifies the intent and explains the
  expectations in the DM SMS Framework

30
Developmental Guidance (Continued)

• Overview
• Provide examples and ways an organization may
  choose to develop its SMS
• Formatted to follow the Framework in The Guide
• Comparable to an Advisory Circular (imbeds
  Framework expectations and guidance)
• Focuses on the areas that may not be
  self-explanatory to the participant
• Designed to be a living document

31
Developmental Guidance (Continued)
32
Gap Analysis Tool

• Purpose
• Assists participants in documenting their gaps
  and status toward meeting the DM SMS
  expectations
• Provides a mechanism to learn more about the
  expectations
• Provides a basis for a participants
  Implementation Plan

Gap tool
33
Gap Analysis Tool (Continued)

• Structure
• Excel Spreadsheet with multiple tabs
• Combines Preliminary and Detailed Gap Analysis
  tools in one file
• Follows the structure of the DM SMS Framework
• Provides step-by-step instructions on how to use
• Provides an Executive Summary
• Automatically populates system segment ratings
• Records Lessons Learned and comments for each
  Element and Sub-Element

Gap tool
34
Assessment Tool

• Structure
• Assessment tab is within the Gap tool
• Assessment tab will be used by PMT to assess
  policies and procedures
• Tracks and documents the PMT's assessment of an
  organization's SMS
• Documents an organization's evidence for
  achieving specified ratings for each DM SMS
  Framework Expectation

Gap tool
35
Assessment Tool (Continued)

• Structure
• Provides a single repository for PMT assessment
  outputs
• Provides a Lessons Learned mechanism to collect
  feedback on the assessment process

Gap tool
36
Implementation Plan Tool

• Structure
• Implementation plan tab is in the Gap tool
• Identifies tasks and activities that are needed
  in order to close the gaps
• Identifies the anticipated timeframe and point of
  contacts for the planned tasks and activities
• Provide a high-level view of the timeline
• Provides a narrative that describes how the
  organization intends to close all expectation
  gaps and achieve a fully functioning SMS.

Gap tool
37
AIR SMS Website (SharePoint)

• Project information / data will be shared with
  the FAA on the MSMS SharePoint site
• Each participant places deliverable documents in
  their own file location to share with FAA
• Files are protected from viewing from other
  participants
• Only selected FAA users will have access to view
  participant information
• Data will not be kept beyond PMT assessment

38
AIR SMS Website (SharePoint)

• Other Information to be shared with all
  participants
• FAA SMS guidance documents
• Gap analysis and assessment tools
• Frequently Asked Questions (FAQs)
  (non-proprietary)
• Generic (non-proprietary) Lessons Learned
  information
• Best Practices
• Other SMS information

39
Pilot Project - Data collection

• Specific data, lessons learned, feedback will be
  collected to assist the MSMS team in the
  development of the SMS framework, policy and
  processes
• Data and information will be general (i.e.
  procedures) and not technical (i.e. Design) in
  nature
• If proprietary data is received, it will not be
  preserved or shared outside the MSMS team (data
  will be deleted / returned when FAA assessment is
  complete)
• Data collection and analysis tools will be used

40
Pilot Project - Data Collection

• Specific data and information to be collected is
  being developed by MSMS team
• Examples of data and information that will be
  requested are
• Lessons learned data and feedback on the SMS
  framework and pilot project processes
• Number of days to complete gap analysis and to
  exit each level
• Number of procedures and processes (new and
  existing)
• Types and number of hazards identified
• SMS processes and procedures that are also part
  of quality system

41
Protection of Participant Information

• The FAA Recognizes the importance of your
  participation in the pilot project and protection
  of data
• FAA will exercise due diligence to not share data
  outside FAA
• FAA will limit the types of data it retains
• Limit data within the FAA
• Please note
• We may share de-identified information
• SMS information that is not specifically
  protected under Part 193 and may be
  subject to FOIA

42
Agenda

• Introductions
• Related SMS Activities
• SMS Fundamentals
• SMS Pilot Project Implementation
• AIR SMS Guidance Tools
• Preliminary Gap Analysis meeting
• Working together during the Pilot Project
• Questions

43
Preliminary Gap Analysis

• Over the next two days we will work together on a
  Preliminary Gap Analysis
• The Gap analysis tool will be used
• The meeting will be facilitated by the POC and
  PMT

44
Purpose of the preliminary gap analysis

• Ensure the organization and FAA have a common
  understanding of the Gap Analysis Process and
  framework
• Develop a common understanding of the DM SMS
  Framework Expectations in the context of the
  organization and its operations
• Preliminary Gap Analysis Tool assists the
  organization in conducting an initial high-level
  first look / best guess in assessing
  compliance with the SMS
• Provides a starting place for the organization to
  conduct a detailed evaluation of the gaps

45
Agenda

• Introductions
• Related SMS Activities
• SMS Fundamentals
• SMS Pilot Project Implementation
• AIR SMS Guidance Tools
• Preliminary Gap Analysis meeting
• Working together during the Pilot Project
• Questions

46
Working together during the Pilot Project

• Pilot project should not interfere with the FAAs
  Certificate Management (CM) responsibilities and
  Type Certification activities (e.g. ACSEP, DOIP,
  Designee Mgt)
• Changes to a PAHs FAA approved quality system
  will continue to be approved per Part 21 and FAA
  policy
• The PMT will work with the participant for the
  duration of the pilot
• Communication process is outlined in The Guide
• Meetings / telecons are anticipated
• Formal communications / decisions will be between
  the POC and PMTL
• Etc.

47
Working together during the Pilot Project
(continued)

• Deviations, clarifications (interpretation),
  improvements concerning the framework, guidance
  material, tools will be coordinated per
  instructions in The Guide
• Feedback and lessons learned will be collected
  throughout the pilot project
• Changes to pilot project guidance will be posted
  on SharePoint

48
Working together during the Pilot Project
(continued)

• After this meeting
• After the preliminary gap analysis the PMT will
  continue working with you
• Within the next few weeks the POC and PMT Lead
  will schedule ongoing meetings / telecons
• The PMT will work closely together in initial
  discussions and activities
• Participants should complete a Detailed Gap
  Analysis and have an Implementation Plan within
  six months
• Schedule an Exit Level 1 validation meeting with
  PMT

49
What happens After the Pilot Project?

• Company should continue developing their SMS to
  get to Level 4
• Sustaining support by the FAA will continue per
  instructions in The Guide
• Local FAA Offices will continue to work with
  company until SMS rule and FAA policy is
  finalized
• POCs should continue working with the local FAA
  offices

50

• Questions ?

• Thank you

• for participation

• in the Pilot Project