Certification, Maintenance, and Operations: Strengthening the Process Links Dan Cheney, Carol Giles,

1
Certification, Maintenance, and Operations
Strengthening the Process Links Dan Cheney,
Carol Giles, andRuth HarderFederal Aviation
Administration
2
19th Annual FAA/JAA International
Conference Phoenix, Arizona June 5, 2002
3
Workshop Item III

• Review of CPS Study
• CPS Overview
• Key Findings and Observations
• Conclusions

4
CPS Overview

• The remarkable safety of todays commercial
  aviation system is the product of continuously
  identifying and applying improvements to all
  aspects of safety by manufacturers, operators,
  and governments
• Continuous improvements involving
• Requirements
• Processes
• Technology
• Training

5
CPS Overview

• During the course of investigation of the January
  2000, Alaska Airlines Flight 261 MD-83 accident,
  particular attention has been given to the
  process associated with how critical airplane
  systems are certified and eventually maintained
  and operated
• Although the NTSB investigation of the Alaska 261
  accident is not yet complete, in December 2000,
  the FAA initiated a review of major processes
  being used in the U.S. to certify commercial
  transport airplanes

6
CPS Overview

• The Commercial Airplane Certification Process
  Study (CPS) began its work in January 2001, and
  has focused on key aspects of the airplane
  certification process and the interrelationship
  of these processes to certain operation and
  maintenance processes
• 34 team members from government, industry,
  academia, and consulting organizations

7
CPS Overview
8
CPS Overview

• The CPS review represents a critical
  self-examination of current processes in order
  to identify areas for improvement
• In order to identify improvements in an already
  very safe system, this kind of introspection is
  considered necessary
• CPS was not an accident investigation activity
  accidents and incidents were reviewed only to
  provide insight into process issues

9
CPS Overview

• CPS review timetable of one year to study and
  report on key aircraft certification,
  maintenance, and operational processes
• Tasked to
• Provide a report to senior FAA management with
  findings
• Identify areas for process improvements

10
CPS Overview

• The CPS report was completed in early March 2002,
  and submitted to the Associate Administrator for
  Regulation and Certification, Mr. Nick Sabatini

• The study paid particular attention to the
  adequacy of the processes related to the arrows
  in this figure

11
CPS Overview

• The CPS team met formally 15 times, averaging
  approximately one week a month
• Some groups met between formal meetings
• A very large amount of information was analyzed
  during the study
• 68 case studies (accidents/incidents)
• 42 presentations
• 12 interviews
• 10 historical reports

12
CPS Overview
13
CPS Overview

• Five focus areas were identified through multiple
  screening and selection processes
• Safety Assurance Processes
• Aviation Safety Data Management
• Maintenance/Operations/Certification Interface
• Major Repairs and Modifications
• Safety Oversight Processes
• Findings and observations resulted from an
  in-depth review in these five areas

14
Key Findings and Observations

• There are 15 findings and 2 observations in the
  CPS report
• These represent areas where the CPS team felt
  that existing processes could be improved.
• Although close examination of any of the 15
  findings could lead one to believe the current
  system has significant deficiencies, it should be
  noted that the study team deliberately looked for
  problem areas so as to improve the system

15
Key Findings and Observations

• Rather than go into detail on all 15 findings, I
  would like to mention certain key findings, some
  from each of the five focus areas
• Most findings have strong international ties
• An in-depth discussion of all 15 findings and 2
  observations can be found in the final report

16
Finding 1

• Human performance is still the dominant factor in
  accidents
• The process used to determine and validate human
  responses to failures and methods to include
  human responses in safety assessments need to be
  improved.
• Design techniques, safety assessments, and
  regulations do not adequately address the subject
  of human error in design or in operations or
  maintenance.

• 80 of all accidents have a human error
  contribution
• 66 of all accidents identify flight crew errors
  as primary cause
• Rates unchanged despite technological advances

17
Finding 1

• Human errors involve many different factors and
  issues - difficult to identify
• Represents major opportunity for safety
  improvements
• Better understanding of range of human skills
  needed
• Lessons learned knowledge needed by all those
  with human factors responsibility
• May benefit from human engineering best
  practices sharing within industry
• FAA Human Factors Team report currently
  addressing many of these issues

18
Finding 4

• Processes for identification of safety critical
  features of the airplane do not ensure that
  future alterations, maintenance, repairs, or
  changes to operational procedures can be made
  with the cognizance of those safety features.

• Many critical safety features of complex
  transport airplane designs are not readily
  obvious. Examples
• Check valves, shear links
• Environmental capability features (e.g.,
  lightning)
• Seals, drain lines, vapor barriers
• Wire routing, electrical grounding paths
• Secondary structural load paths, energy
  absorption devices

19
Finding 4

• Increased awareness of safety critical features
  of the airplane is needed by those operating and
  modifying it
• Strongly linked to several other findings in
  study, e.g., Finding 7, which is discussed next
• Represents a high-leverage opportunity for
  improvement
• EAPAS and recent FAR 25 (i.e., fuel system
  safety) change cited as good examples for raising
  awareness

20
Finding 7

• There is no widely accepted process for
  analyzing service data or events to identify
  potential accident precursors.

• All safety data systems should have common
  objective - to identify the need for safety
  intervention prior to an accident
• Some government/industry safety data management
  programs have shown promise in accident precursor
  capability
• May represent best practices approach for the
  rest of industry
• CAAM, COSP, FOQA and ASAP are examples

21
Finding 8

• Adequate processes do not exist within the
  FAA or in most segments of the commercial
  aviation industry to ensure that the lessons
  learned from specific experience in airplane
  design, manufacturing, maintenance, and flight
  operations are captured permanently and made
  readily available to the aviation industry. The
  failure to capture and disseminate lessons
  learned has allowed airplane accidents to occur
  for causes similar to those of past accidents.

• Lessons learned information needs to be an
  important part of accident precursor
  recognition/intervention strategy
• Currently, no formal lessons learned database
  exists

22
Finding 8

• Most safety lessons learned are still relevant
  today despite technology advances
• Corporate memory fades with time and loss of
  experienced personnel
• Some informal OJT attempts to pass critical
  safety lessons from one generation to the next,
  but becoming increasingly difficult
• Accident investigation findings often narrow
  focused while important high level lessons
  usually have broad applicability

23
Finding 10

• There are currently no industry processes or
  guidance materials available that ensure
• Safety related maintenance or operational
  recommendations developed by the OEM are
  evaluated by the operator for incorporation into
  their maintenance or operational programs.
• Safety related maintenance or operational
  procedures developed or modified by the operator
  are coordinated with the OEM to ensure that they
  do not compromise the type design safety standard
  of the airplane and its systems.

• Challenge is for safety related or
  significant material to be coordinated rather
  than all material
• Not all safety related information can/should be
  the subject of airworthiness directives

24
Finding 10

• This is not an easy subject to solve and may
  benefit from industry partnership, both
  manufacturers and operators
• Method for ensuring distribution and review of
  safety material is needed rather than resorting
  to airworthiness directives (ADs)
• If quickly implemented, the solution to this
  finding could have an immediate impact on safety

25
Finding 13
Inconsistencies exist between the safety
assessments conducted for the initial Type
Certification (TC) of an airplane and some of
those conducted for subsequent alterations to the
airplane or systems. Improved FAA and industry
oversight of repair and alteration activity is
needed to ensure that safety has not been
compromised by subsequent repairs and alterations.

• Vast majority of repairs/alterations are done
  consistently with the safety assessment of the
  original design
• However, cases exist of inadequate repairs and
  alterations
• FAAs STC Process Review and Field Approval
  Process are examples of efforts to improve this

26
Finding 15

• Processes to detect and correct errors made by
  individuals in the design, certification,
  installation, repair, alteration, and operation
  of transport airplanes are inconsistent allowing
  unacceptable errors in critical airworthiness
  areas.

• Transport airplane safety achieved, in large
  part, through fail-safe, redundant, and
  fault-tolerant design concepts
• These concepts are not extended consistently to
  single-point human error potential during
  maintenance and alteration
• Strongly linked to Finding 4 (Safety Critical
  Features)

27
Conclusions

• Four common areas were identified, which appeared
  to link the findings and observations
• Information Flow Barriers to critical
  information flow may exist
• Human Factors Failure of the human machine
  interface
• Lessons Learned Significant safety issues
  learned through accident/incident analysis
• Accident Precursors Significant incidents that
  are indicators of a serious service problem
  requiring intervention in order to prevent an
  accident

28
Conclusions
29
Conclusions

• CPS findings and observations, as well as many of
  the accidents analyzed during this study, concern
  interface issues involving certification
  processes and operation/maintenance processes

30
Conclusions

• Cross discipline consideration will be
  essential in order to achieve airplane level
  safety awareness during safety decision making

31
Conclusions

• Improvements in safety will require coordinated
  initiatives involving the manufacturer, operator,
  and FAA
• Accident precursor recognition and intervention
  will require greater airplane level safety
  awareness during all aspects of the commercial
  airplane life cycle

32
Workshop Item IV

• CPS Response Team Strategic Plan
• Overview of CPS Change Areas
• CPS Change Areas International Links
• Potential for Future Direction

33
Change Area Overview

• CPS Response Team identified four Change Areas
  from CPS Report
• I. Safety Information Awareness
• II. Human Factors Integration
• III. Repairs Alterations
• IV. AIR/AFS Integration
• The Change Areas provide a framework for CPS
  implementation in the Strategic Plan

34
CPS Change Area Overview

• The Change Areas are separate initiatives for
  safety improvements, but have strong
  interrelationships with each other.
• Work on Change Areas can be initiated in
  parallel.
• Breaking Change Areas into smaller packages would
  not provide necessary integration during
  implementation

35
I. Safety Information Awareness

• Issue Safety information processes used
  throughout industry are fragmented and
  uncoordinated, causing inefficient identification
  and mitigation of safety issues
• Key Topics to be addressed in this Change Area
• A. Safety Critical Design Information (F 2, 3, 4)
• B. COS Data Management (F 5, 6, 7, 9)
• C. Lessons Learned/Airplane System Awareness (F
  8)
• D. OEM/Operator Safety Info Transfer (F10, O1)

36
I. Safety Information Awareness

• To raise awareness of safety issues and mitigate
  them, safety information must be managed in an
  integrated way
• Identify important safety critical features in
  the design
• Define how to communicate this information
• Timely identify safety issues from in-service
  data and mitigate the safety concern
• Capture and retain Lessons Learned

37
I. Safety Information Awareness

• The Response Team believes this Change Area
  provides the strongest opportunity for
  improvements to safety
• Safety information should be addressed at the
  global level to be most effective otherwise key
  information may be missed
• Plan for Change Breakout teams will developed
  detailed work plans for each Information Topic.

38
II. Human Factors Integration

• Issue Human errors continue to dominate as a
  contributing factor in accidents (F1,15)
• Multiple initiatives underway today
  internationally to address human factors issues,
  are fragmented uncoordinated
• Plan for Change Urgent need for a comprehensive,
  joint industry/government plan that integrates
  these separate efforts.

39
II. Human Factors Integration

• HF Integrated plan should
• Focus on human factors issues that have resulted
  in accidents in the past and/or that could result
  in accidents in the future.
• Bring together the various certification,
  operations, and maintenance efforts currently
  underway within the FAA and industry.
• Address human factors inputs into the ongoing
  operational safety decision process for the
  existing fleet.

40
III. Repairs Alterations

• Issue Lack of clear requirements for major
  repairs and alterations, combined with inadequate
  oversight (F12, 13, 14, O2)
• Changes made without understanding of the
  original certification assumptions add risk
  because the modifier, maintainer, or operator may
  not understand safety implications.
• Global impact due to the number and complexity of
  repairs and alterations being accomplished
  worldwide on a daily basis

41
III. Repairs Alterations

• Plan for Change
• Review industry best practices to develop a model
  and issue guidance material for establishing
  repair and alteration decision logic.
• Provide guidance to insure alterations consider
  original OEM safety critical information
• Review existing policies and procedures related
  to the oversight of consultant DERs who approve
  complex repairs, alterations, or STCs

42
IV. AIR/AFS Integration

• Issue Lack of effective communication and
  coordination between FAA offices responsible for
  certification (AIR) and maintenance and
  operations (AFS) may compromise safety (F11)
• Affects the FAAs ability to address industry
  safety issues effectively and industrys ability
  to comply fully

43
IV. AIR/AFS Integration

• Plan for Change
• Define AVR-level policy directed towards improved
  internal and external communication and
  coordination between AIR and AFS.
• Develop business processes to enhance
  coordination between AIR and AFS.
• Improved FAA in-house communication between these
  organizations will achieve safety benefits from
  more timely and consistent communication with
  global counterparts and industry

44
CPS Future Direction

• Review of CPS planned safety improvements in
  relation to other major safety initiatives
• - identify and manage the relationships between
  change initiatives
• CPS Strategic Plan Phase II work will include
  detailed plan development, using breakout teams
  comprised of people from all involved FAA and
  industry organizations

45
CPS Future Direction

• All parties (manufacturers, operators, and
  authorities) must be fully involved in and
  committed to the CPS safety solutions.
• Change will be necessary in many or all involved
  organizations, not just within the FAA or any
  single organization.
• Airplane System Awareness must continue to
  improve throughout FAA and industry worldwide.