Normal Operations Safety Survey NOSS

1

• Normal Operations Safety Survey (NOSS)
• Chris Henry
• The University of Texas Human Factors Research
  Project
• The University of Texas at Austin
• 2nd ICAO TEM NOSS Symposium
• Washington DC
• February 7, 2007

2
Presentation Objectives

• Why conduct NOSS?
• Factors that lead to a successful NOSS
• Frequently asked questions

3
Aviation Safety Envelope
Accidents
Incidents
Normal Operations
4
Safety Data Coverage
Accidents
Incidents
Normal Operations
5
Aviation Safety Envelope
Accidents
Incidents
Normal Operations
NOSS Rationale Proactive snapshot of system /
controller performance strengths and weaknesses
in normal operations (just like a health checkup)
6
What can NOSS provide?

• Identify threats within the operating environment
• Check the quality/usability of procedures
• Identify problems with controller/equipment
  interfaces
• Assess the degree of transference of training to
  the line
• Understand controller shortcuts and workarounds
• Make comparisons across units or facilities
• Assess safety margins

7
NOSS Success Factors

• NOSS success is dependent upon methodology and
  execution

Formal Check
Regulator
Nobody
Angel Performance
Natural Performance

• Low controller trust Low quality data because
  there will be no differentiation between NOSS and
  proficiency checks

8
NOSS Operating Characteristics

• Over-the-shoulder observations during normal
  shifts
• No observations of controllers who are undergoing
  training
• No observations while checks are being conducted
• Voluntary controller participation
• Controllers have a right to decline a NOSS
  observation
• Denial rates in NOSS trials have been low
• Joint management / association sponsorship
• Letter signed by management association
  representatives sent to all controllers

9
NOSS Operating Characteristics

• Trusted and trained observers
• Trust and Credibility
• Selection process management / association list
  of candidates
• Diversity of observers increases buy-in
• Training
• Classroom training, test observations,
  recalibration session
• Curriculum
• TEM principles and exercises
• NOSS observation protocols
• Narrative guidelines

10
NOSS Operating Characteristics

• Anonymous, confidential, and non-punitive data
  collection
• No names, employee numbers, dates, experience, or
  other identifying information
• Data used for safety purposes only no punitive
  actions
• Observers identity is known only by the third
  party facilitator
• Trusted and secure data collection site
• Third party or controller association gatekeeper
• Controllers and observers must be comfortable
  with the data collection site
• Systematic data collection instrument
• No judgments observers record data based on TEM
  events

11
NOSS Safety Metrics

• What do controllers do to manage their airspace?

The Threat and Error Management Model serves as
the NOSS Safety Metrics
12
NOSS Operating Characteristics

• Data verification process
• TEM data checked to ensure coding accuracy and
  consistency with SOP
• Data analysis does not begin until verification
  has been completed
• Data-derived targets for safety enhancement
• Serve as benchmarks for safety change
• Initial NOSS, safety change process, follow-up
  NOSS
• Feedback results to controllers
• Results summarized for controllers
• Information on how organization intends to
  respond to the data

13
NOSS Gaining Controller Trust

• Over-the-shoulder observations during normal
  shifts
• Joint management / association sponsorship
• Voluntary Participation
• Trusted and trained observers
• Anonymous, confidential, and non-punitive data
  collection

• Trusted and secure data collection site
• Systematic data collection instrument
• Data verification process
• Data-derived targets for safety enhancement
• Feedback results to controllers

14
NOSS Defined

• The 10 characteristics that differentiate NOSS
  (LOSA) from other methodologies have been
  endorsed by the
• Federal Aviation Administration
• International Civil Aviation Organization
• International Air Transport Association
• International Federation of Airline Pilots
  Association
• International Federation of Air Traffic
  Controllers (6)
• The LOSA Collaborative
• US Airline Pilots Association
• University of Texas
• Programs that omit one or more of the
  characteristics might be useful and beneficial,
  but they are not NOSS
• NOSS must have all ten characteristics

15
Frequent Questions

• How long are NOSS observations?
• Observations approximately one hour in duration
  have given us good data
• Do we observe individual controllers/positions or
  teams?
• Team dynamics captured, but from the perspective
  of an individual operating position
• Should NOSS be continuous or a snapshot?
• Snapshot allows organization to assess and act on
  data
• Other data sources can inform between snapshots
• Little benefit for the increased cost of doing
  continuous NOSS
• What should the scope of a NOSS be?
• Depends on the size of the facilities and the
  comparisons to be made