Cognitive Engineering PSYC 530 Automation and Human Performance

1
Cognitive EngineeringPSYC 530Automation and
Human Performance

• Raja Parasuraman

2
Overview

• Characteristics of Automation
• Human Performance in Automated Systems
• Designing for Effective Human-Automation
  Interaction

3

• Automation Definitions and Characteristics

4
Automation is Ubiquitous

• Aviation
• Air traffic control
• Ground and maritime transportation
• Process control and manufacturing
• Military command and control
• Medicine and health care
• Intelligent agents
• Home automation
• Robotics
• Drug design/Molecular genetics

5
What is Automation?

• A machine or system that accomplishes (partially
  or fully) a function that was previously carried
  out (partially or fully) by a human operator

Source PARASURAMAN, R., RILEY, V. (1997).
Humans and automation Use, misuse, disuse,
abuse. Human Factors.
6
Advanced Automation Boeing 777
EICAS
NAV DISPLAY
PAPER!
PRIMARYFLGHT DISPLAY
7
Reasons for the March Towards More Automation

• Cost
• Safety?
• Technical Capability
• Human Factors?

X
8
Grounding of the Cruise Ship Royal Majesty,
Nantucket, 1995
9
Grounding of the Cruise Ship Royal Majesty,
Nantucket, 1995

• Accident Grounding of passenger ship on Rose and
  Crown shoal near Nantucket Island, MA
• Losses 2 million structural damage 5 million
  lost revenue no injuries or fatalities
• Automation Autopilot Automatic Radar Plotting
  Aid (ARPA) Global Positioning System (GPS)

10
Grounding of the Cruise Ship Royal Majesty,
Nantucket, 1995
11
Grounding of the Cruise Ship Royal Majesty,
Nantucket, 1995

• NTSB Probable Cause Over-reliance on automated
  features of the integrated bridge system
  management failure to ensure officers adequately
  trained in automated features
• Human-Automation Issues automation complacency
  crew resource management training

12
NTSB Report Conclusions (Extracts)

• the GPS receiver antenna cable connection
  separated enough that the GPS switched to dead
  reckoning mode, and the autopilot.no longer
  corrected for the effects of wind, current or
  sea.
• the watch officers monitoring of the status of
  the vessels GPS was deficient throughout the
  voyage
• .deliberate cross-checking between the GPS and
  the Loran-C to verify positionwas not being
  performed.
• .all the watchstanding officers were overly
  reliant on the automated position display.and
  were, for all intents and purposes, sailing the
  map display instead of using navigation aids or
  lookout information

13

• Human Performance in Automated Systems

14
Human Performance

• Cognitive Processes
• Visual Attention
• Mental Workload
• Vigilance and Monitoring
• Working Memory
• Situation Awareness
• Decision Making
• Social Processes
• Trust in Automation
• Attitudes

15
Levels of Automation

• HIGH 10. The computer decides everything, acts
  autonomously, ignoring the human.
• 9. informs the human only if it, the
  computer, decides to
• 8. informs the human only if asked, or
• 7. executes automatically, then
  necessarily informs the human, and
• 6. allows the human a restricted time to
  veto before automatic execution, or
• 5. executes that suggestion if the human
  approves, or
• 4. suggests one alternative
• 3. narrows the selection down to a few, or
• 2. The computer offers a complete set of
  decision/action alternatives, or
• LOW 1. The computer offers no assistance
  human takes all decisions and actions.

Source SHERIDAN, T. B. (1992). Telerobotics,
Automation, and Supervisory Control. Cambridge,
MA MIT Press.
16
Human-Automation Interaction Some Empirical
Methods

• Human-in-the-loop Simulation
• Human Performance Modeling
• Quantitative Models
• Field Studies

17
A Field Study?
18
Automation and Human Performance

• Automation can fundamentally change the nature of
  the cognitive demands and responsibilities of the
  human operators of system--often in ways that
  were unintended or unanticipated by designers

19
Automation and Human Performance Benefits

• Improved precision of performance
• Operational flexibility
• Reduced mental workload
• Enhanced safety (automated warning systems)

20
Automation and Human Performance Potential Costs

• Unbalanced mental workload
• Automation complacency
• Loss of situation awareness
• Mode error/confusion
• Manual skill degradation
• Degraded teamwork/communication

21
Automation The Double-Edged Sword

• Automation often provides clear benefits
• Automation can also lead to novel, unanticipated
  problems and performance costs
• Which tasks should be automated and to what
  level for optimal control, performance, and
  safety?
• Technologists Automate tasks as fully as
  technically possiblethe technological
  imperative
• Human factors engineers Automate to an extent
  that balances efficiency with safety and
  ensures a proper role for the human in the
  resulting system

22
Automation Can But Does Not Always Reduce Mental
Workload

• Clumsy AutomationIncreases mental workload
  during high task load, reduces it during low task
  load
• Cognitive OverheadAutomation is difficult to
  engage, adjust, or turn off

Sources WIENER, C. E. (1988). Cockpit
automation. In E. L. Wiener D. C. Nagel (Eds.)
Human factors in aviation. San Diego Academic
Press. KIRLIK, A (1993). Modeling strategic
behavior in human-automation interaction Why an
aid can (and should) go unused. Human Factors,
35.
23
Effects of Level of Automation on Situation
Awareness

• Levels of SA
• Level 1 Perception
• Level 2 Comprehension
• Level 3 Projection

Source Endsley, M., Kiris, E. (1995). The
out-of-the-loop performance Problem and level of
control in automation. Human Factors, 37, 390-398.
24
EFFECTS OF LEVEL OF AUTOMATION ON OPERATOR
SITUATION AWARENESS
100
90
SA LEVEL 2 ( Correct)
80
70
Manual
Decision Support
Consensual AI
Monitored AI
Full Automation
LEVEL OF AUTOMATION BEFORE AUTOMATION FAILURE
25
Trust Affects Automation Usage

• Over-trust (Complacency)Inappropriate use and
  over-reliance on automation
• Under-trust (Distrust)Disuse or turning off of
  automation

The goal is to achieve calibrated trust that is
matched to the situation
Source LEE, J., MORAY, N. (1992). Trust,
control strategies, and allocation of function In
human-machine systems. Ergonomics. PARASURAMAN,
R., MOLLOY, R., SINGH, I. L. (1993).
Performance consequences of automation-induced
"complacency." International Journal of
Aviation Psychology.
26
Automation Trust and Complacency Study

• 24 Experienced General Aviation Pilots
• 2 Levels of DifficultySingle and Multiple-Task
• 2 Levels of Automation (Manual, Automated)
• Task Carry out primary flight and fuel
  management tasks manually, monitor automated
  engine-systems task

27
Multi-Attribute Task Battery (MAT)
28

• Human Operators Are Poor at Monitoring Automated
  Systems When They Are Simultaneously Engaged in
  Other Manual Tasks

MANUAL
AUTOMATED
100
Cost of Automation Complacency
80
DETECTION RATE ()
60
40
20
0
SINGLE-TASK
MULTI-TASK
29

• Designing for More Effective Human-Automation
  Interaction

30
Make Automation State Indicators and Behaviors
More Salient

• Use Display Integration to Improve
• the Observability of Automation Behaviors

31
Engine Indicator and Crew Alerting System (EICAS)
EPR
Detecting a Malfunction Requires
Manual Integration over Several Engine
Parameters
N1
EGT
32
Engine Monitoring and Control System (EMACS)
33
Effects of Display Integration on
Human-Automation Interaction
EICAS (Non-Integrated)
EMACS (Integrated)
34
Use of Multi-Modality Feedback to Enhance
Human-Automation Interaction
Glass Cockpit Simulator
Tactile Feedback System
Roll Mode Transition
Autothrottle Mode Transition
35
(No Transcript)
36
Summary of Human Performance

• Certain automation designs can lead to unbalanced
  mental workload, reduced situation awareness,
  and miscalibrated trust and complacency
• The irony of automation (Bainbridge,
  1983)highly reliable but imperfect automation
  has a greater cost than less reliable automation
  when the automation fails
• Some of these costs can be mitigated using
  integrated displays, multi-modality feedback,
  ecological interface design, and adaptive
  automation

37
Evaluative Criteria Human Performance

• Mental models
• Communication and coordination
• Mental workload
• Situation awareness
• Trust and complacency
• Cognitive skills
• Teamwork

38
Additional Evaluative Criteria

• Production and Operating Costs
• Automation Reliability
• Costs of Decision/Action Consequences
• Efficiency/Safety Tradeoffs
• Ease of System Integration
• Liability Issues

39
Implementing Human Factors in Automation Design
HUMAN-MACHINE SYSTEMS APPROACH
Design
Development
Fielded System
Operations
Human Factors Science and Engineering
40
Automation design should not consist of cleaning
up the designers mess afterwards.