Title: Certification, Maintenance, and Operations: Strengthening the Process Links Dan Cheney, Carol Giles,
1Certification, Maintenance, and Operations
Strengthening the Process Links Dan Cheney,
Carol Giles, andRuth HarderFederal Aviation
Administration
219th Annual FAA/JAA International
Conference Phoenix, Arizona June 5, 2002
3Workshop Item III
- Review of CPS Study
- CPS Overview
- Key Findings and Observations
- Conclusions
4CPS Overview
- The remarkable safety of todays commercial
aviation system is the product of continuously
identifying and applying improvements to all
aspects of safety by manufacturers, operators,
and governments - Continuous improvements involving
- Requirements
- Processes
- Technology
- Training
5CPS Overview
- During the course of investigation of the January
2000, Alaska Airlines Flight 261 MD-83 accident,
particular attention has been given to the
process associated with how critical airplane
systems are certified and eventually maintained
and operated - Although the NTSB investigation of the Alaska 261
accident is not yet complete, in December 2000,
the FAA initiated a review of major processes
being used in the U.S. to certify commercial
transport airplanes
6CPS Overview
- The Commercial Airplane Certification Process
Study (CPS) began its work in January 2001, and
has focused on key aspects of the airplane
certification process and the interrelationship
of these processes to certain operation and
maintenance processes - 34 team members from government, industry,
academia, and consulting organizations
7CPS Overview
8CPS Overview
- The CPS review represents a critical
self-examination of current processes in order
to identify areas for improvement - In order to identify improvements in an already
very safe system, this kind of introspection is
considered necessary - CPS was not an accident investigation activity
accidents and incidents were reviewed only to
provide insight into process issues
9CPS Overview
- CPS review timetable of one year to study and
report on key aircraft certification,
maintenance, and operational processes - Tasked to
- Provide a report to senior FAA management with
findings - Identify areas for process improvements
10CPS Overview
- The CPS report was completed in early March 2002,
and submitted to the Associate Administrator for
Regulation and Certification, Mr. Nick Sabatini
- The study paid particular attention to the
adequacy of the processes related to the arrows
in this figure
11CPS Overview
- The CPS team met formally 15 times, averaging
approximately one week a month - Some groups met between formal meetings
- A very large amount of information was analyzed
during the study - 68 case studies (accidents/incidents)
- 42 presentations
- 12 interviews
- 10 historical reports
12CPS Overview
13CPS Overview
- Five focus areas were identified through multiple
screening and selection processes - Safety Assurance Processes
- Aviation Safety Data Management
- Maintenance/Operations/Certification Interface
- Major Repairs and Modifications
- Safety Oversight Processes
- Findings and observations resulted from an
in-depth review in these five areas
14Key Findings and Observations
- There are 15 findings and 2 observations in the
CPS report - These represent areas where the CPS team felt
that existing processes could be improved. - Although close examination of any of the 15
findings could lead one to believe the current
system has significant deficiencies, it should be
noted that the study team deliberately looked for
problem areas so as to improve the system
15Key Findings and Observations
- Rather than go into detail on all 15 findings, I
would like to mention certain key findings, some
from each of the five focus areas - Most findings have strong international ties
- An in-depth discussion of all 15 findings and 2
observations can be found in the final report
16Finding 1
- Human performance is still the dominant factor in
accidents - The process used to determine and validate human
responses to failures and methods to include
human responses in safety assessments need to be
improved. - Design techniques, safety assessments, and
regulations do not adequately address the subject
of human error in design or in operations or
maintenance.
- 80 of all accidents have a human error
contribution - 66 of all accidents identify flight crew errors
as primary cause - Rates unchanged despite technological advances
17Finding 1
- Human errors involve many different factors and
issues - difficult to identify - Represents major opportunity for safety
improvements - Better understanding of range of human skills
needed - Lessons learned knowledge needed by all those
with human factors responsibility - May benefit from human engineering best
practices sharing within industry - FAA Human Factors Team report currently
addressing many of these issues
18Finding 4
- Processes for identification of safety critical
features of the airplane do not ensure that
future alterations, maintenance, repairs, or
changes to operational procedures can be made
with the cognizance of those safety features.
- Many critical safety features of complex
transport airplane designs are not readily
obvious. Examples - Check valves, shear links
- Environmental capability features (e.g.,
lightning) - Seals, drain lines, vapor barriers
- Wire routing, electrical grounding paths
- Secondary structural load paths, energy
absorption devices
19Finding 4
- Increased awareness of safety critical features
of the airplane is needed by those operating and
modifying it - Strongly linked to several other findings in
study, e.g., Finding 7, which is discussed next - Represents a high-leverage opportunity for
improvement - EAPAS and recent FAR 25 (i.e., fuel system
safety) change cited as good examples for raising
awareness
20Finding 7
- There is no widely accepted process for
analyzing service data or events to identify
potential accident precursors.
- All safety data systems should have common
objective - to identify the need for safety
intervention prior to an accident - Some government/industry safety data management
programs have shown promise in accident precursor
capability - May represent best practices approach for the
rest of industry - CAAM, COSP, FOQA and ASAP are examples
21Finding 8
- Adequate processes do not exist within the
FAA or in most segments of the commercial
aviation industry to ensure that the lessons
learned from specific experience in airplane
design, manufacturing, maintenance, and flight
operations are captured permanently and made
readily available to the aviation industry. The
failure to capture and disseminate lessons
learned has allowed airplane accidents to occur
for causes similar to those of past accidents.
- Lessons learned information needs to be an
important part of accident precursor
recognition/intervention strategy - Currently, no formal lessons learned database
exists
22Finding 8
- Most safety lessons learned are still relevant
today despite technology advances - Corporate memory fades with time and loss of
experienced personnel - Some informal OJT attempts to pass critical
safety lessons from one generation to the next,
but becoming increasingly difficult - Accident investigation findings often narrow
focused while important high level lessons
usually have broad applicability
23Finding 10
- There are currently no industry processes or
guidance materials available that ensure - Safety related maintenance or operational
recommendations developed by the OEM are
evaluated by the operator for incorporation into
their maintenance or operational programs. - Safety related maintenance or operational
procedures developed or modified by the operator
are coordinated with the OEM to ensure that they
do not compromise the type design safety standard
of the airplane and its systems.
- Challenge is for safety related or
significant material to be coordinated rather
than all material - Not all safety related information can/should be
the subject of airworthiness directives
24Finding 10
- This is not an easy subject to solve and may
benefit from industry partnership, both
manufacturers and operators - Method for ensuring distribution and review of
safety material is needed rather than resorting
to airworthiness directives (ADs) - If quickly implemented, the solution to this
finding could have an immediate impact on safety
25Finding 13
Inconsistencies exist between the safety
assessments conducted for the initial Type
Certification (TC) of an airplane and some of
those conducted for subsequent alterations to the
airplane or systems. Improved FAA and industry
oversight of repair and alteration activity is
needed to ensure that safety has not been
compromised by subsequent repairs and alterations.
- Vast majority of repairs/alterations are done
consistently with the safety assessment of the
original design - However, cases exist of inadequate repairs and
alterations - FAAs STC Process Review and Field Approval
Process are examples of efforts to improve this
26Finding 15
- Processes to detect and correct errors made by
individuals in the design, certification,
installation, repair, alteration, and operation
of transport airplanes are inconsistent allowing
unacceptable errors in critical airworthiness
areas.
- Transport airplane safety achieved, in large
part, through fail-safe, redundant, and
fault-tolerant design concepts - These concepts are not extended consistently to
single-point human error potential during
maintenance and alteration - Strongly linked to Finding 4 (Safety Critical
Features)
27Conclusions
- Four common areas were identified, which appeared
to link the findings and observations - Information Flow Barriers to critical
information flow may exist - Human Factors Failure of the human machine
interface - Lessons Learned Significant safety issues
learned through accident/incident analysis - Accident Precursors Significant incidents that
are indicators of a serious service problem
requiring intervention in order to prevent an
accident
28Conclusions
29Conclusions
- CPS findings and observations, as well as many of
the accidents analyzed during this study, concern
interface issues involving certification
processes and operation/maintenance processes
30Conclusions
- Cross discipline consideration will be
essential in order to achieve airplane level
safety awareness during safety decision making
31Conclusions
- Improvements in safety will require coordinated
initiatives involving the manufacturer, operator,
and FAA - Accident precursor recognition and intervention
will require greater airplane level safety
awareness during all aspects of the commercial
airplane life cycle
32Workshop Item IV
- CPS Response Team Strategic Plan
- Overview of CPS Change Areas
- CPS Change Areas International Links
- Potential for Future Direction
33Change Area Overview
- CPS Response Team identified four Change Areas
from CPS Report - I. Safety Information Awareness
- II. Human Factors Integration
- III. Repairs Alterations
- IV. AIR/AFS Integration
- The Change Areas provide a framework for CPS
implementation in the Strategic Plan
34CPS Change Area Overview
- The Change Areas are separate initiatives for
safety improvements, but have strong
interrelationships with each other. - Work on Change Areas can be initiated in
parallel. - Breaking Change Areas into smaller packages would
not provide necessary integration during
implementation
35I. Safety Information Awareness
- Issue Safety information processes used
throughout industry are fragmented and
uncoordinated, causing inefficient identification
and mitigation of safety issues - Key Topics to be addressed in this Change Area
- A. Safety Critical Design Information (F 2, 3, 4)
- B. COS Data Management (F 5, 6, 7, 9)
- C. Lessons Learned/Airplane System Awareness (F
8) - D. OEM/Operator Safety Info Transfer (F10, O1)
36I. Safety Information Awareness
- To raise awareness of safety issues and mitigate
them, safety information must be managed in an
integrated way - Identify important safety critical features in
the design - Define how to communicate this information
- Timely identify safety issues from in-service
data and mitigate the safety concern - Capture and retain Lessons Learned
37I. Safety Information Awareness
- The Response Team believes this Change Area
provides the strongest opportunity for
improvements to safety - Safety information should be addressed at the
global level to be most effective otherwise key
information may be missed - Plan for Change Breakout teams will developed
detailed work plans for each Information Topic.
38II. Human Factors Integration
- Issue Human errors continue to dominate as a
contributing factor in accidents (F1,15) - Multiple initiatives underway today
internationally to address human factors issues,
are fragmented uncoordinated - Plan for Change Urgent need for a comprehensive,
joint industry/government plan that integrates
these separate efforts.
39II. Human Factors Integration
- HF Integrated plan should
- Focus on human factors issues that have resulted
in accidents in the past and/or that could result
in accidents in the future. - Bring together the various certification,
operations, and maintenance efforts currently
underway within the FAA and industry. - Address human factors inputs into the ongoing
operational safety decision process for the
existing fleet.
40III. Repairs Alterations
- Issue Lack of clear requirements for major
repairs and alterations, combined with inadequate
oversight (F12, 13, 14, O2) - Changes made without understanding of the
original certification assumptions add risk
because the modifier, maintainer, or operator may
not understand safety implications. - Global impact due to the number and complexity of
repairs and alterations being accomplished
worldwide on a daily basis
41III. Repairs Alterations
- Plan for Change
- Review industry best practices to develop a model
and issue guidance material for establishing
repair and alteration decision logic. - Provide guidance to insure alterations consider
original OEM safety critical information - Review existing policies and procedures related
to the oversight of consultant DERs who approve
complex repairs, alterations, or STCs
42IV. AIR/AFS Integration
- Issue Lack of effective communication and
coordination between FAA offices responsible for
certification (AIR) and maintenance and
operations (AFS) may compromise safety (F11) - Affects the FAAs ability to address industry
safety issues effectively and industrys ability
to comply fully
43IV. AIR/AFS Integration
- Plan for Change
- Define AVR-level policy directed towards improved
internal and external communication and
coordination between AIR and AFS. - Develop business processes to enhance
coordination between AIR and AFS. - Improved FAA in-house communication between these
organizations will achieve safety benefits from
more timely and consistent communication with
global counterparts and industry
44CPS Future Direction
- Review of CPS planned safety improvements in
relation to other major safety initiatives - - identify and manage the relationships between
change initiatives - CPS Strategic Plan Phase II work will include
detailed plan development, using breakout teams
comprised of people from all involved FAA and
industry organizations
45CPS Future Direction
- All parties (manufacturers, operators, and
authorities) must be fully involved in and
committed to the CPS safety solutions. - Change will be necessary in many or all involved
organizations, not just within the FAA or any
single organization. - Airplane System Awareness must continue to
improve throughout FAA and industry worldwide.