Line Operations Safety Audit LOSA

1
Line Operations Safety Audit (LOSA)

• Captain Dan Maurino
• Flight Safety and Human Factors ICAO
• First ICAO Global Symposium on TEM NOSS in ATC
• Luxembourg
• 9 10 November 2005

2
Line Operations Safety Audit

• Ten Operating Characteristics (ICAO LOSA Manual,
  Doc 9803)
• Jump seat observations during normal flights
• Voluntary crew participation
• Confidential, safety-minded data collection
• Joint airline-pilot sponsorship
• Targeted observation instrument
• Trusted, trained, standardised observers
• Trusted data collection site
• Data cleaning round tables
• Data-derived targets for enhancement
• Feedback of results to line pilots

Capture an airline systems real
performance through the eyes of its flight crews
3
A Day at the Office

• To achieve daily safety and production goals,
  flight crews must

4
Threats
Factors external to the flight deck that increase
the complexity of flight operations

• Environmental Threats
• Adverse WX
• Terrain
• Airport conditions
• Heavy traffic / TCAS events
• Command events / errors
• Language difficulties
• Aircraft Threats
• Aircraft malfunctions
• Automation events

• Airline Threats
• Dispatch events / errors
• Ground events / errors
• MX events / errors
• Time Pressures
• Irregular Operations
• Reception / congestion
• Cabin events / FA errors

5

Errors
Deviations from flight crew or organizational
expectations

• Handling errors deficiencies in stick and
  rudder skills
• e.g. inability to maintain proper
  speed/configuration during ILS approach
• Procedural errors deficiencies in execution
• e.g. correctly selecting an incorrect altitude in
  the altitude alerter
• Communication errors deficiencies in the
  exchange of operational information
• e.g. misunderstanding an altitude clearance

6
Undesired Aircraft States
Flight crew induced aircraft states where
existing safety margins are reduced

• Lateral deviation
• Vertical deviation
• Speed too high
• Speed too low
• Incorrect aircraft configuration
• Unstable approach

7

• What does LOSA bring to the airline community?
• (that the airline community does not already
  have)

8
The Extreme Value of the Narratives
Threat runway switch. We had expected and had
briefed 17Left from the ATIS broadcast. No reason
for the switch given by ATC. Scramble to brief
new runway plus linked to error of not finding
the proper approach chart and automation error of
not setting up computer for the landing runway.
FO left the approach and arrival route for 17Left
in the FMC instead of the new runway. Based on
this the aircraft reached DIETZ and started a
turn to heading 350 which was still the active
route instead of the 300 heading assigned.
9
Almost immediately after takeoff, the Captain
started a turn up the river, and did not wait
until 400 feet, per SOP. About 300 feet, captain
reached over to MCP and selected Flight Path
Angle. He did not tell FO of this selection,
which is very unusual of a setting. In fact, the
FO later asked me if I had ever seen anyone use
this selection and stated that he has never seen
it in his 2 years of being on the Ford Trimotor.
Because the Captain did not tell the F/O what he
was doing, when the FO noticed this setting, he
reselected the more common mode (Heading).
Captain then reached over and then reselected
Flight Path Angle. This led to a brief discussion
of whether of not the aircraft would capture NAV
in this setting. As a result of all of this, the
FO missed ATC handoff to departure control, a
call that was not made until the aircraft leveled
off at 5000 feet. Both Captain and FO were
involved in heads-down stuff as the aircraft
leveled off. If the autopilot would not have
leveled off, then the crew would have not caught
it, because they were doing extraneous things.
(FO, programming MCDU Captain, looking at
weather).
10
SCP Data Transformed Into Action
Re-measure
11
SCP Example

• First LOSA Targets for improvement by the SCP
• Energy management issues during
  descent/approach/land
• 18 of flights speed deviation undesired
  aircraft state
• Unstable approaches
• 11 of flights unstable approach
• Checklist errors
• 50 of flights checklist errors
• Monitor/cross-checking
• 38 of errors undetected (not responded to by the
  flight crew)
• ATC threat management
• 22 of ATC threats mismanaged (linked to flight
  crew error)

12
SCP Airline Response to Data

• Committees to address targets for enhancement
• Implemented a leadership training module in RT
• Modified checklist philosophy
• Established bottom lines for unstable approaches
• Implemented error management training course for
  all pilots
• Check airmen were trained in evaluating error
  management

13
SCP Did it Work?
11 of 182,500 flights/year 20,075 unstable
approaches 4 of 182,500 flights/year 7,300
unstable approaches
14
How Realistic is LOSA Data?

• Observers changing flight crews behaviour
  Flight crews faking behaviour
• Average three errors per flight 25 are
  mismanaged
• Average 32 of flights have an UAS
• Substandard CRM or procedural violations
• Using cell phone during taxi-out
• Offering cigarettes to LOSA observer

15
Observer Bias

• Sources and countermeasures
• Sensitivity Observers sensitive to certain
  areas of flight operations (Human Factors experts
  only see HP issues)
• Observers with different backgrounds
• Observers gathering facts No judgment of causes
  
• Proficiency Lack of TEM knowledge to collect
  data
• Test for TEM knowledge following training
• Data verification
• Observer burnout Less data collected/less
  precise
• Cap the number of observations per observer

16
Continuous or Snapshot LOSA?

• Snapshot approach
• Continuous LOSA
• Costs
• Questionable value over snapshot LOSA
• A snapshot approach fosters accountability
• There is a start and end for data collection that
  marks a point in time to derive targets and get
  to the SCP
• Snapshot approach can complement other continuous
  safety monitoring tools (FDA/self-reporting)

17
Five Effective Ways of Killing LOSA

• Pretending to do LOSA
• Senior management not briefed and/or prepared for
  the possibility of undesirable results
• Results not communicated to pilots Affects
  pilot acceptance for the second LOSA
• Unmotivated observers
• No safety change process in place to act on LOSA
  findings

18
LOSA Airlines September 2005
28 airlines 15 countries
Continental Airlines Continental
Express Continental Micronesia Delta Airlines EVA
Air Frontier Airlines Futura LACSA LanChile QA
NTAS

• AeroMexico
• Asiana Airlines
• Alaska Airlines
• Air New Zealand
• Air Transat
• Braathens
• Cathay Pacific
• China Airlines
• COPA

Malaysia Airlines Mt. Cook Airlines Regional
Express Singapore Airlines Silk Air TACA / TACA
Peru UNI Air US Airways VARIG