Enhancing Safety in Flight Operations

1
Enhancing Safety in Flight Operations

• Bob Helmreich
• Human Factors Research Project
• The University of Texas at Austin
• ALPA, San Antonio
• May 9, 2000

2
Current Research at UT

• Line Operations Safety Audit (LOSA)
• Validated by major carriers
• Recently endorsed by ICAO as a central focus for
  its Flight Safety and Human Factors Program
• Threat and error management model
• Analysis of normal flights, incidents, and
  accidents
• Aviation Safety Action Program (ASAP)
• Reporting form design for incidents
• Investigate effects of national, organizational,
  professional culture on crew performance and
  safety
• Data from airlines in 26 countries
• Underlying motivation
• Be proactive improve safety before an accident
• Explore human factors issues using threat and
  error management model
• Identify latent systemic threats

3
Measuring Threat, Error, and ResponseThe Line
Operations Safety Audit (LOSA)
4
LOSA

• Six years of research ten airlines - 3800
  flight segments
• Work in progress - still developing LOSA
  methodology
• FOQA tells what happen, LOSA looks for why it
  happened
• What is it?
• Non-jeopardy, jump seat observations of flight
  crews
• Data highlight strengths and weaknesses of
  airline procedures and crew member behavior
• How is it implemented?
• One to three months of observations
• Team of observers make cross-fleet observations
• UT staff and airline personnel from HF, safety,
  standards, union

5
LOSA Data Flight Crew

• Flight crew
• Demographics
• Attitudes/perceptions (FMAQ)
• Comments from safety interview
• Flight description
• Overt threats
• Operational complexity
• Observer narrative

• Performance
• CRM practice
• Flight crew errors
• Undesired aircraft states
• Technical data for approaches
• Threat management
• Error management
• Undesired state management

6
Overt Threats

• Overt threats are the building blocks of
  operational complexity

• Adverse Weather
• Terrain
• Aircraft malfunctions
• Automation anomalies
• Poor airport conditions
• Abnormal operations
• Operational pressures

• ATC event or error
• Maintenance event or error
• Ground event or error
• Dispatch event or error
• Cabin event or error

• Overt threat profiles differ greatly by airline
  and operation

• Training should be customized to the airlines
  threat profile

7
Human Error is the Downside of Having a Brain
Error is action or inaction that leads to a
deviation from intentions or expectations
Loosely paraphrased from Rene Amalberti
8
A Typology of Flight Crew Error

• Intentional Noncompliance violations
• ex) Performing a checklist from memory
• Procedural Followed procedures but wrong
  execution
• ex) Wrong altitude setting dialed into the MCP
• Communication Missing information or
  misinterpretation
• ex) Miscommunication with ATC
• Proficiency Error due to a lack of knowledge
• ex) Lack of knowledge with automation
• Decision Crew decision unbounded by procedures
  that unnecessarily increased risk
• ex) Unnecessary navigation through adverse
  weather

9
Consequential Outcomes -Undesired Aircraft States

• Undesired Aircraft States Flightcrew induced
  deviations
• from normal flight that threaten safety

• Lateral deviation
• Vertical deviation
• Speed too high or low
• Unstable approach
• Abrupt aircraft control
• Incorrect configurations

• Long landing
• Hard landing
• Landing off centerline
• Wrong taxiway or ramp
• Wrong runway
• Wrong airport or country

10
LOSA Data Flight Crew

• For each flight segment, observers collect data
  on
• Threats to safety and external errors (i.e., ATC,
  ground, or maintenance)
• Flight crew errors and violations
• Threat and error management strategies and
  outcomes
• CRM practice
• Technical data on approaches flown (type and
  stability)
• Undesired aircraft states (lateral and vertical
  nav and speed deviations, etc.)
• Crew comments from a systematic survey and
  interview
• Attitudes and perceptions of the organizational
  culture
• Events that have occurred, but are unknown to
  management
• Suggestions to improve flight operations and
  safety
• Data collected are confidential and de-identified
  to ensure anonymity
• Protected as part of quality assurance
• LOSA is a safety program

11
Some Data from Threat and Error LOSAs
12
LOSA External Threat Results

• 72 of the flights had at least one external
  threat
• Two external threats per flight
• Most external threats on a flight 11
• Most Frequent Threats for one airline (focus on
  Latin American operation)
• Adverse weather 34 of the flights
• ATC events or errors 34
• Aircraft malfunctions 15

13
Flight Crew Error
14
LOSA Error Results

• 68 of the flights had an observed error
• Average of two errors per flight
• Most errors on a flight 14
• Most common flight crew errors
• Automation errors 31 of all errors
• Wrong MCP and FMC modes and settings
• Failure to cross-verify
• Checklist errors 21 of all errors
• Checklist performed from memory
• Nonstandard usage
• Missed items

15
Threat and Error by Phase of Flight
Summary - The most threatening phase (descent /
approach / land) also contains the most error
16
(No Transcript)
17
Error Frequencies and Consequences
18
Violations matter

• 40 of accidents in global fatal accident
  database had violations

Flight Safety Foundation Approach and Landing
Accident Reduction Task Force Report
R. Khatwa R. Helmreich November, 1998
19
(No Transcript)
20
Violators pose more risk! LOSA Data

• Crews that commit at least one violation are
• 1.5 times more likely to commit unintentional
  errors
• 1.8 times more like to commit errors with
  consequential outcomes (additional error or
  undesired aircraft state)

21
Four Ways to Reduce Violations

• Dont hire male pilots
• Dont hire U.S. pilots
• Get rid of SOPs
• Stop collecting data

22
Threat and Error Management

• After conducting observations on 3800 flights
  across seven airlines......
• Fifteen behavioral markers have been determined
  to be the most relevant in threat and error
  management
• The fifteen markers can be divided into four
  groups
• Team Climate and Leadership
• Planning
• Execution
• Review/Modify Decisions

23
Error Outcomes

• Inconsequential 85 of all errors
• Consequential Outcomes 15 of all errors
• Additional Error 3
• Undesired Aircraft States 12

24
Undesired Aircraft State Management

• When an aircraft enters an undesired aircraft
  state, flightcrews manage the state not the error
• Undesired aircraft state responses
• Mitigate 79 of all error induced aircraft
  state responses
• Exacerbate 2
• Fail to respond 12

25
Variability between Airlines
Airline Airline Airline A
B
C Threats per segment 3.3
2.5 0.4 Errors
per segment .86 1.9
2.5 Error Management consequential
18 25
7
26
Variability Within One Airline
Aircraft Intentional
Procedural
Noncompliance AdvancedTech Fleet 1
40
31 Advanced Tech Fleet 2
30
44 ConventionalTech Fleet 1 17

55 ConventionalTech Fleet 2
53 20
27
Using LOSA Error Data

• Violations - suggest poor procedures, weak
  captain leadership and/or a culture of
  non-compliance
• Procedural errors - may indicate poor workload
  management or poor procedures
• Communications error - may reflect inadequate CRM
  (monitoring and challenging) or complacency
• Proficiency errors - suggest pressures to train
  and/or need for higher standards and better
  monitoring by check airmen
• Decision errors - may indicate need for more CRM
  training on expert decision making and risk
  assessment

28
LOSA Success Story

• Major American carrier Two LOSAs, two years
  apart
• 1st LOSA results
• Major problems identified
• Substandard ratings for Captain leadership
• Checklist protocols and adherence
• Unstable approach problem
• Company response
• Formed committees to address problems identified
  by LOSA
• Incorporated leadership training
• Revised bottom lines for unstable approaches
• Implemented error management training for pilots
• Check airmen culture change training in
  evaluating error management

29
LOSA Success Story - Outcomes

• 2nd LOSA results
• Captain leadership ratings improved
• 40 reduction in the checklist errors
• 77 reduction in unstable approaches at 1000ft.
  ATDZ
• 42 reduction in unstable approaches at 500 ft.
  ATDZ
• Increased upper management and pilot buy-in for
  human factors and safety initiatives

30
Using LOSA data to build a model of threat
and error management in aviation (UT-TEMM)
31
Need for a Model of Threat and Error

• We need to
• understand the threats and errors that occur
  during normal operations
• Determine the underlying factors that contribute
  to error (Reasons latent failures)
• have a teaching tool that stresses threat and
  error management as the central focus of CRM

32
Latent systemic threats

• Latent systemic threats
• Culture (National, Professional, and
  Organizational)
• Airports and navigational aides
• Aircraft design, automation, and maintenance
• Regulations, policies, and procedures
• Training curriculum and implementation
• Flight crew support (ATC, MX, Ground, Dispatch,
  Cabin)
• Latent threat often detected only after an
  incident or accident
• Normal operations data identify latent systemic
  threats before they become consequential

33
Threat and Threat Management
Latent Threats National Culture, Organizational
Culture Policies, Professional culture,
Regulation
Overt Threats
Environmental Factors
Individual Factors
Team/Crew Factors
Aircraft Factors
Organizational Factors
External Error
Behavioral Marker Countermeasures
Safety
34
The Full UT Model
Overt Threats
Threat Induced Incident or Accident
External Threat Management
Inconsequential
Error
Error Response
Recovery
Additional Error
Undesired Aircraft State
Undesired Aircraft State Response Mitigate Exacerb
ate Fail to Respond
35

• Applying TEMM to a pilot error incident
• Landing at the wrong airport

36
The Flight Scenario

• Houston to Corpus Christi second leg of the first
  day of a trip scheduled 20 minutes
• FO is PF to IAH-CRP first PF leg after initial
  training
• Captain experienced check airman conducting IOE
• CA not to CRP for 3 years, FO never
• CRP ATIS ..good, 6,000 broken 10 miles, wind
  010, 10kts. Expect loc app to Rwy 31.
• Descent started expecting VORTAC vectors to Rwy
  31 localizer. .. On autopilot until
  intercepting final approach course

37
The Scenario continued

• After CRP Vortac, App Cont. gives vectors and for
  localizer and Cleared Approach Rwy 31 Localizer
• Turning final, the Capt sees the airport and
  reports to tower Runway in sight.
• Crew has slowed and configured early to allow
  time to reach landing configuration and be stable
  by 500 AGL. They are surprised runway so close.
  RED FLAG!!
• A normal landing was completed but runway shorter
  than expected. Landing completed with 500
  remaining
• There was no terminal, only hangers. Crew taxis
  off Rwy and realizes they are not at CRP

38

• Expected Overt Threats
• 2nd leg of first day as crew
• IOE being given
• 1st EFIS leg for FO, only exposure to EFIS in
  Level 5 device
• FOs 1st trip to CRP
• Capt had not been to CRP in 3 years had never
  landed to North at CRP

39

• Unexpected Overt Threats
• ATIS doesnt mention 2000 scattered cloud layer
  that blocks fwd vision. (Crew cannot see CRP)
• ATIS gives no warning of Cabaniss Field parallel
  runway
• A/C on autopilot, has problem intercepting
  localizer

40

• External Error
• ATC has localizer set to Rwy 13 instead of 31 for
  A/C doing training

41
Error 3 Decision
Crew did not go around when they discover airport
too close
No response
No response
Approach to Wrong airport
Incident
Outcome
Crew Response
Crew Action
Final Outcome
Error
42

• Latent Threats
• Geographic Cabaniss Field same layout 6nm on
  Corpus localizer
• Navigational chart depiction of Cabaniss hard to
  read
• ATC no ATIS warning about Cabaniss on weekend
• ATC MSAW warning for Cabaniss disabled
• Mothers Day Sunday Navy usual drag racing
  cancelled
• System information on other near approaches to
  Cabaniss existed, but not available
• Organization no mention of Cabaniss on10-7 page
  in manual
• Professional pressure to complete approach and
  make landing

43
Threat and Error Totals
5 Overt, 8 Latent Threats, 1 External error
Threat Induced Incident or Accident
External Threat Management
Inconsequential
3 Crew Errors
Error Response Not detected
Recovery
2 Additional Error
1 Undesired Aircraft State
Undesired Aircraft State Response 1 Fail to
Respond
44
ASAP Background

• ASAP is another leg of the Voluntary Air Safety
  Partnership along with FOQA
• American Airlines has had an ASAP program since
  1994
• Clinton administration and Jane Garvey endorse
  ASAP
• What does the FAA get in return?
• Feedback on ATC performance and procedures
• Better data on airline operational problems
• Reduced number of certificate actions
• More than a dozen airlines are in line for ASAP

45
Our Contribution to ASAP

• Designed a report form that goes beyond a pilot
  narrative to focus on human factor issues
• Sample of the form can be downloaded from our
  website
• Uniqueness of the UT ASAP form
• Gets at the why an event occurred
• Question format allows for quick categorization
  and trend monitoring
• Designed for usage on company Intranet
• Uses the conceptual framework of threat and error
  management

46
The Questions Report Form Sections

• Event demographics When and Where did the event
  occur?
• Crew member demographics What was the
  background of the pilots involved?
• Flight conditions In what context did it occur?
• Narrative of the event What happened?
• Contributing factors to the event What factors
  caused and helped mitigate the event?
• Recommendations from the reporter How can
  similar incidents be prevented in the future?

47
CRMs as Threat and Error Management

• The Latest CRM programs
• Recognize the inevitability of error
• Provide a set of countermeasures against threat
  and error
• Are effective in a supportive organizational
  culture

48
HF Training Issues

• Human limitations as sources of error
• The nature of error and error management
• Expert decision making for threat and error
  management
• Conflict resolution
• Training in using specific behaviors as
  countermeasures against threat and error
• Formal review of accidents and incidents error
  model
• Reinforcement for threat recognition and error
  management

49
Does Medicine Need CRM

• Common Concepts
• Safety is super-ordinate goal
• But cost cannot be ignored
• Teams interacting with technology
• Risk varies from low to high
• Threat error from multiple sources
• Second guessing after adverse events
• UT group has collected data for 7 years in the OR
  and now the ICU

50
Does CRM Export?

• One of the biggest mistakes in the history of CRM
  was the assumption that CRM programs would work
  at another airline or even worse in another
  culture
• CLR in Korea
• Flight Safety in China
• The assumption that airline CRM training will fix
  the problems of medicine is dangerously wrong
• The professional and organizational cultures are
  different
• The system is more complex and more screwed up
• The regulatory climate is different
• The legal issues are different
• Conceptual issues are similar, but the
  misapplication of CRM can do more harm than good
• Destroying confidence in the importance of team
  training when change is not effected an problems
  persist

51
Final Words
52
University of Texas Human Factors Research Project
www.psy.utexas.edu/psy/helmreich/nasaut.htm