Inventing the Operational Safety Assessment ATN

1
Inventing the Operational Safety Assessment
ATN99
Steve Paasch Federal Aviation Administration Aircr
aft Engineering Division Avionics Branch AIR-130
c/o ANM-100S 1601 Lind Avenue SW Renton, WA
98055-4056 phone [phone] fax
425-227-1181 email steve.paasch_at_faa.gov
2
Contents

• Operational Safety Assessments - How they became
  to be
• The purpose of conducting formal safety
  assessments
• The processes and methods involved in performing
  safety assessments
• Controller Pilot Data Link Communications (CPDLC)
  program Operational Safety Assessment (OSA)

3
Operational Safety Assessments - How they became
to be
Technology Aging Evolution
Aviation Expansion Globalization
Need for New Operational Capabilities
Safety
Airspace Modernization
RTCA TASK FORCE IV
Need For Certification Efficiencies
End-to-End Certification
OSA
Digital Communications Requirements
(SC-189/WG-53)
4
RTCA SC-189 / EUROCAE WG-53

• Chartered to develop safety, performance,
  interoperability requirements for air traffic
  services supported by communications
• Subgroup 2 is working on methods and examples for
  developing operational environment descriptions
  and performing operational safety assessments
• air-ground end-to-end safety assessment from an
  operational viewpoint

5
Website for SC-189

• http//www.mews.org/atssir//

6
RTCA Task Force IV

• Opportunities to Reduce the Time, to Reduce the
  Cost, and to Provide Better Certification Service
• Achieving Operational Benefits
• Human Performance
• End-to-End Aviation Systems Considerations
• Regulation, Policy, and Guidance Development
• Authority Organization, Processes, and Industry
  Interface

7
End-to-End Aviation Systems Considerations

• The Task Force heard many concerns that systems
  were not being properly considered overall, or
  "from end to end." The introduction of new
  elements into the ground or airborne parts of the
  system are not generally preceded by appropriate
  systems engineering practices, including
  definition of operations concepts and
  requirements. It is clear that overall system
  performance is rarely specified and that
  authorities often do not take a structured
  approach to establishing the requirements for
  International Airspace System (INAS) systems and
  components. It is common for new ground or
  airborne components to have specifications or
  performance that are not matched to the other
  elements of the system with which they work to
  perform their function. One consequence may be
  that the new system element is over-specified,
  and therefore more expensive than it should be to
  achieve the incremental improvement in
  performance. Another possible consequence is that
  the new system element is not properly specified
  in light of the performance of other system
  elements, and the expected improvement in
  efficiency from the new system element is not
  attained.

8
Task Force IV Recommendations

• Recommendation 2 The authorities should
  establish and maintain a systems engineering
  capability. This function should be used to
  establish overall performance requirements for
  all advanced systems and their subsystems, in
  conjunction with the user community. As part of
  this effort, the authorities should consider
  developing clear approval standards and processes
  for ground system elements that are integrated,
  to the degree necessary, with airborne system
  element certification. (Section 3.4)
• Recommendation 5 The authorities should broadly
  implement a process where the regulators and
  applicants come to an early and clear agreement
  on their respective roles, responsibilities,
  expectations, schedules, and standards to be used
  in certification projects. The process should
  apply broadly across airborne and ground systems,
  allow non-applicant equipment suppliers to engage
  in certification programs, and provide greater
  opportunity to approve components or processes
  independent of the airplane. (Section 3.6)

9
The purpose of conducting formal safety
assessments
10
Starting from what we have and going to what we
need

• We have a traditional aircraft-related system
  safety assessment process
• What is it?
• What is for?

11
What is the traditional, aircraft related system
safety assessment process?

• It is a systems engineering activity to assure
  that safety objectives are met......by
  identifying where systems requirements are needed
  to eliminate or mitigate potential safety
  problems
• Systems engineering is a two sided coin -
  optimistic vs pessimistic
• SSA turns the systems engineering perspective of
  performance, functionality, form, etc., around
• Do this vs what if it doesnt do this?

12
What is the traditional, aircraft related system
safety assessment process for?

• In a nutshell--
  To have a systematic way to
  analyze aircraft and aircraft systems
  function-related failure conditions, as well as
  failure condition contributors and mitigators, in
  order to
• Set safety objectives for failure conditions
• Identify systems safety requirements to meet
  safety objectives
• Assure systems safety requirements (and thus
  safety objectives) are met

13
A Systems Engineering Discipline

• The System Safety Assessment side of the systems
  engineering coin
• has its own methods for discovering requirements
• has its own processes to organize the methods
• has its own vocabulary to facilitate the
  processes
• has its own guidance materials for passing
  knowledge on

14
System safety assessments are tied to aircraft of
a type, and the installed systems and equipment,
or engines and engine systems.
15

• ...But the aircraft isnt the only player in the
  airspace game...

16
Broadening our horizons beyondan aircraft.....to
the airspace system
multiple aircraft multiple capabilities
ground systems signal networks operational
procedures ad hoc evolution modernization
program
17
What is the Operational Safety Assessment Process
for?

• In a nutshell-- A systematic way to analyze
  airspace and air traffic management
  service-related operational hazards, and
  operational hazard contributors and mitigators,
  in order to
• Set safety objectives for operational hazards
• Identify systems and procedural safety
  requirements to meet safety objectives
• Assure systems and procedural safety requirements
  (and thus safety objectives) are met

18
An Airspace Planning Discipline

• The Operational Safety Assessment side of the
  airspace planning coin
• should have its own methods for discovering
  requirements
• should have its own processes to organize the
  methods
• should have its own vocabulary to facilitate the
  processes
• should have its own guidance material for passing
  knowledge on

19
The processes and methods involved in performing
safety assessments
20
Inventing a vocabulary

• Starting with the system safety assessment
  vocabulary

21
What to say when good systems go bad

• Failure Condition
• Failure
• Failure Mode
• Fault
• Error

22
What can we do with our specialized vocabulary?

• We can organize our concepts into relationships

23
Aircraft designers view
Aircraft
FAILURE CONDITION
ERROR
FAILURE
FAILURE
FAILURE MODES
FAULT
FAULT
FAULT
FAULT
ERROR
PHYSICS
PHYSICS
ERROR
PHYSICS
ERROR
PHYSICS
ERROR
24
Terminology comparison
OSA
SSA

• Functions
• Failure conditions
• Failures
• Failure Modes
• Faults
• Errors

• Air Traffic Services
• Operational hazards
• Failures
• Failure Modes
• Faults
• Errors

25
Airspace designers view
26
Inventing a process

• Starting with the system safety assessment process

27
A metaphor for systems engineering?
28
System Safety Assessment Process

• Identify aircraft or systems functions
• Identify failure conditions
• Determine failure condition severity
• Set safety objectives based on failure condition
  severity
• Determine system safety requirements to meet
  safety objectives
• Allocate safety requirements across systems and
  components
• Assure safety requirements are met

29
What can we do with our specialized process?

• We can organize our activities to be systematic
  and thorough

30
System Safety Assessment Process - discovering
safety requirements
as-built
System Safety Assessments
strategies refinement
Preliminary System Safety Assessments
objectives
Functional Hazard Assessment
Common Cause Analyses
Aircraft or System Function Definition
31
Process comparison
SSA
OSA

• Identify aircraft or systems functions
• Identify failure conditions
• Determine failure condition severity
• Set safety objectives based on failure condition
  severity

• Identify air traffic services
• Identify operational hazards
• Determine oper. hazard severity
• Set safety objectives based on operational hazard
  severity

32
Process comparison (continued)
SSA
OSA

• Determine systems safety requirements to meet
  safety objectives
• Allocate safety requirements across systems and
  components
• Assure safety requirements are met

• Determine operational safety reqts to meet safety
  objectives
• Allocate safety reqts across institutions and
  airspace components
• Assure safety requirements are met

33
Operational Safety Assessment Process
-discovering safety requirements
as-built
Ground System Safety Assessments
Aircraft System Safety Assessments
Institutional Safety Assessments
strategies refinement
Allocation of Safety Objectives and Requirements
objectives
Operational Hazard Assessment
Operational Environment Definition (OED --
Services and airspace characteristics that may
affect hazard severity)
Common Cause Analyses
34
How do operational safety assessments and system
safety assessments relate?
35
(No Transcript)
36
Inventing methods?

• Starting with system safety assessment methods?

37
System Safety Assessment Methods

• Inverse relationship for classifying failure
  conditions and setting assurance levels
• Fail Safe Principles
• Fault Tree Analysis
• Failure Modes and Effects Analysis
• Markov Analysis
• Dependence Diagramming
• Mathematics of failure rates, probability, and
  Boolean algebra

38
What can we do with our specialized methods?

• We can discover cause and measure effect in a
  relatively precise fashion with tabular,
  graphical, mathematical, logical means

39
System Safety Assessment hazard classification
scheme
40
(No Transcript)
41
Methods comparison
SSA
OSA

• Inverse relationship for classifying operational
  hazards and setting assurance levels
• Otherwise, were working on it
• Matrix and templates
• Institutional methods at institutional levels
• CPDLC
• OED
• Hazard table
• FTA
• Reqts allocation

• Inverse relationship for classifying failure
  conditions and setting assurance levels
• Fail Safe Principles
• Fault Tree Analysis
• Failure Modes and Effects Analysis
• Markov Analysis
• Dependence Diagramming
• Mathematics of failure rates, probability, and
  Boolean algebra

42
Operational Safety Assessment hazard
classification scheme
43
Inverse relationship
44
Inventing guidance material

• Adding to system safety assessment guidance

45
Safety assessment guidance material

• AC 23.1309-1C for Normal, Utility, Acrobatic,
  Commuter Airplanes
• AC/AMJ 25.1309-1B for Transport Airplanes
• AC 27-1A for Normal Rotorcraft
• AC 29-2B for Transport Rotorcraft
• SAE ARP 4754 for all
• SAE ARP 4761 for all

46
What can we do with our specialized guidance?

• We can pass the vocabulary, processes, and
  methods on to the community of airspace planners,
  developers, service providers, and users

47
Guidance comparison
SSA
OSA

• System Safety Assessments
• AC 23.1309-1C
• AC 25.1309-1B
• AC 27-1A
• AC 29-2B
• SAE ARP 4754
• SAE ARP 4761

• Operational Safety Assessment
• RTCA SC-189 ED-DO docs
• Guidance
• Methodology
• SPR
• FANS

48
SC-189/WG-53 Summary
49
Publication overview
ED/DO-GUID
Cross-regional/area planning

• Implementation
• Aircraft certification
• ATS system commissioning
• ATS operational approval
• User operational approval
• Airspace approval

ED/DO-SPR
CNS/ATM Service Operation
CNS/ATM System, Procedures, Airspace Development
Homogeneous ATM Area Planning Objs/rqmts
ED/DO-METH

• ED/DO-INTEROP
• ARINC 622
• ARINC 623
• ATN
• MIX

50
ED/DO-GUID
Planning
Requirements Determination
Operational capability Air traffic
services Functions
Objective To agree on approach To establish
requirements Activities Definition,
Assessment, Allocation Validation Evidence
Approval plan(s) Assessments Requirements Traceabi
lity
OED
TechChoice
OSA
RCP
Allocated requirements
Interop
Coordination
Aircraft Ops App
Aircraft Cert
ATS Prov Sys App
ATS Prov Ops App
Airspace App
51
189/53 pub group

• Objective To produce first drafts of all
  publications with high level of maturity on
• PUB-4 D Outline for the RTCA / EUROCAE Documents
• PUB-20 (METH) Method for Operational
  environment description and for evaluating
  operational environment
• PUB-22 (GUID) Guidelines for qualifications and
  operations of advanced ATS
• PUB-23 (INTEROP (622)) A622 Interoperability
  Document
• PUB-24 (SPR (Procedural Control Airspace))
  Characterization of operational environment -
  safety and performance requirements

52
189/53 pub group

• Serge Bagieu 189-53 A/C mfgr France
• Tom Kraft 189-53/CAG A/C cert USA
• Wil Struck 189-53/CAG A/C cert USA
• Lionel Bertin SG1 A/C mfgr France
• Kevin Grimm SG1 ATS provider USA
• Bob Granville SG2 ATS provider UK
• Steve Paasch SG2 A/C cert USA
• Gilles Surlaive SG3 ATS provider France
• Roy Oishi SG3 Com provider USA
• Gary Morton CAG/SG2 ATS regulator UK
• Don Streeter CAG/SG1 A/C ops app USA
• Chester Studzinski CAG/SG2 ATS regulator Canada
• Jim Coyne CAG/SG2 A/C cert Australia
• Frank Cheshire PUB A/C Oper USA
• Mike Cuddy PUB A/C Oper USA

53
189/53 pub group schedule
Apr May Jun Jul Aug Sep Oct
54
Controller Pilot Data Link Communications (CPDLC)

• Items being developed
• Operational Environment Description (OED)
• Operational Hazard Assessment (OHA)
• Allocation of Safety Requirements (ASOR)
• Includes fault tree analysis and requirements
  allocation matrix
• SPR (Safety Performance Requirements)

55
(No Transcript)
56
Wrap Up and Questions?

• Thank You!