Human Factors and Collaborative Decision Making

1
Human Factors and Collaborative Decision Making

• Andrew J. Ryan
• (ajryan_at_gmu.edu)
• http//mason.gmu.edu/ajryan/research.html
• Human Factors Engineering
• Dr. L. Adelman

2
Overview

• Overview of Collaborative Decision Making (CDM)
• Examine Flight Schedule Monitor (FSM) as it
  relates to human factors principles
• Introduce the concept of feedback in FSM and
  discuss user attitudes towards this feature
• QA

3
Importance of CDM

• CDM Working Group
• Operational and technical experts
• Federal Aviation Administration
• 1 of 4 core technologies as part of the National
  Airspace System modernization
• White House
• Announcement on 10 March 2000

4
FAA Air Traffic Management
Air Traffic Management (ATM)
Traffic Flow Management (TFM)
Air Traffic Control (ATC)

• Safe Separation

• Safe and EfficientFlows

5
Flight Schedule Monitor (FSM)

• FAA and Airline Industry shareware which
  provides common situational awareness
• Provides traffic flow management modeling tools
  for issuing ground delay programs and ground stops

6
What is a Ground Delay Program?
A ground delay program (GDP) is implemented at an
airport when arrival demand exceeds capacity
7
Ground Delay Program Goal
Balance capacity and demand at impacted airport.

• Delays taken on the ground, not enroute
• Reduces airborne congestion
• Delivers smooth and reduced arrival rate to the
  airport

8
Impact of Ground Delay Programs

• GDPs in use since early 1980s
• 500 - 1000 GDPs per year
• 1999 GDP statistics
• 705 GDPs over 23 airports (average of 2 per day)
• 361,246 flights controlled (987 per day)
• 32.8 Mil minutes of delay assigned (89,537 per
  day)
• Average of 90 min. assigned delay per flight

9
Other GDPE Benefits

• Improved data quality and system predictability
• Enhanced GDP performance
• Increased arrival slot utilization
• Smoother arrival flows into the airport
• Improved decision making
• Avoided GDPs and early GDP cancellations
• Increase system flexibility
• Increased cooperation and trust

10
How does FSM fare against Human Factors
Principles?
11
Situational Awareness . . . YES!
12
Graph Consistency

• According to Wickens
• The graph designer should be concerned with the
  relations among various graphs . . . The
  consistency should allow readers of the graph to
  perceive related measures across various
  situations (121).
• How does FSM fare against this principle?

13
(No Transcript)
14
(No Transcript)
15
About Consistency . . .
Someone preferred ABC while the next person was
partial to XXX
16
Ad hoc Abbreviations
17
Directing Attention

• As Wickens states
• It is sometimes possible to advise an operator
  in advance where direction should be directed
  (83).
• Does anything jump out at you in the following
  slide?

18
(No Transcript)
19
(No Transcript)
20
Alerts warn of Non-compliance
21
Menu Structure

• Wickens
• Menu designers would like to structure a menu in
  such a way that the target items are reached in a
  minimum average time (82).
• Frequently used items can be positioned toward
  the top of the menu.
• High cognitive distance how subjectively
  related two pieces of information are for the
  user (178).
• Organizational distance the distance between
  two items in the menu hierarchy (178).

22
This menu would not raise many flags . . .
23
Conversely, the depth of this menu is
mind-boggling
24
Partitions group selections . . .
25
Violating consistency, ellipses are introduced
26
What would Miller Think? (WWMT)
27
Proximity Compatibility (again)
28
Are the Subs Really Off?
The airlines pay the price
29
Ground Stops

• A ground stop is used to ease airborne holding
  and internal congestion.

30
Taking the Auto out of Automation

• After all parameters are entered, FSM generates
  the following message
• This creates a level of undertrust in the system.

31
What is Undertrust?

• Undertrust defined
• When an operator distrusts automation either
  because of its complexity or because if its
  unreliability, there is a possibility that the
  system may be trusted less than is warranted
  (543).
• Solutions
• Simplify design
• Explicate algorithms

32
Undertrust in FSM

• Specialists invoke non-scientific heuristics to
  fool the system.
• Chicago OHare (ORD) GDP rates are constantly
  inflated to garner expected results.
• Confirmation bias/positive reinforcement sets in
  as there is no feedback to refine their method.
• As Wickens notes . . . general practice in
  decision making does not lead to improved
  performance (327).

33
Solution Create a feedback loop

• Spoke to several user types
• Specialists, computer support advocates, airline
  representatives, supervisors
• Overwhelming support for feedback (either
  real-time or post operations)
• Less experienced operators felt uncertain of
  their performance as there was no gold or silver
  standard
• Feedback will focus on process and not just
  output (a good outcome does not equate to a good
  decision making process)

34
Proposed Solution

• Either real-time or post-ops feedback
• Need to examine if specialists will appreciate
  help in the heat of battle (could take
  attention from task)
• Real time feedback can provide insight into
  algorithms
• Airport specific feedback based on stochastic
  modeling and user input (they must invoke
  heuristics for a reason)
• Only applicable to high volume, dynamic airports
  (BOS, ORD, LGA, MSP, DFW)

35
Benefits of Feedback

• Provides the learner with knowledge of results
• Needed to improve performance
• Sets a silver standard for performance
• Specialists cry workload when performance is
  questioned, feedback can (loosely) quantify
  performance
• Allows all parties involved to see into the
  process as opposed to playing Monday morning
  quarterback.

36
Wrap-Up

• Overview of Collaborative Decision Making (CDM)
• Examine Flight Schedule Monitor (FSM) as it
  relates to human factors principles
• Introduce the concept of feedback in FSM and
  discuss user attitudes towards this feature
• QA

37
Human Factors and Collaborative Decision Making

• Andrew J. Ryan
• (ajryan_at_gmu.edu)
• http//mason.gmu.edu/ajryan/research.html
• Human Factors Engineering
• Dr. L. Adelman