Designing for Situation Awareness in Complex Systems

1
Designing for Situation Awareness in Complex
Systems
Presentation to Virginia Tech January 2002

• Mica R. Endsley, Ph.D.
• SA technologies, Inc.

2
How Do People Make Decisions?
World Cues
Situation Awareness
Recognize Known Class of Situation
Ascertain Situation
Retrieve Script of Appropriate Behavior from
Memory
Determine Course of Action that will work
Carry out Actions
Naturalistic Models of Decision Making
3
Naturalistic Models of Decision Making

• Dreyfus, 1981
• Chess (deGroot, 1965)
• Managers (Mintzburg, 1973)
• Science (Kuhn, 1970)
• Experts use pattern matching to quickly
  understand situations and select appropriate
  course of action
• Klein, 1988
• Fire Ground Commanders
• Recognition Primed Decision Making (RPD)
• Kaempf, et al. (1993)
• Tactical Commanders - 95 used RPD
• Feature Matching - 87
• Story Building - 13

4
Situation Awareness Drives the Decision Process
The Key Factor Determining Decision Quality is
SA
5
Why Talk about SA?

• Leading Causal Factor in Review of 175 Military
  Aviation Mishaps
• (Hartel, Smith Prince , 1991)
• Major Causal Factor in 88 of Accidents
  Associated with Human Error in Review of Major
  Aircarrier Accidents (1989-1992)
• (Endsley, 1994)
• Portion of the task that takes up the majority of
  decision makers time and effort

6
SA is Critical for Decision Making in Many
Domains

• Aviation
• ATC
• Process Control
• Driving
• Train Operations
• Advanced Manufacturing
• Space
• Medicine
• Education
• Business Operations

7
Situation Awareness
Situation Awareness is the Perception of the
Elements in the Environment within a Volume of
Time and Space, the Comprehension of their
Meaning, and the Projection of their Status in
the Near Future.

• Spatial/Temporal SA
• attitude
• altitude
• heading
• velocity
• vertical velocity
• Gs
• flight path
• actual values relative
• to assigned
• projected flight path
• projected landing time

• System SA
• system status
• functioning and settings
• radio
• altimeter
• transponders
• flight modes automation
• deviations from correct settings
• ATC communications present
• fuel
• impact of degrades settings
• on performance
• time and distance available on fuel

• Environmental SA
• weather formations movement
• temperature
• icing
• ceilings
• fog
• turbulence, winds
• sun
• visibility
• IFR/VFR conditions
• areas to avoid
• flight safety
• projected weather conditions

Endsley, 1988
8
How do we get SA?

• System

9
Situation Awareness the Product of the Processes
Processes
Product
10
Theory
Endsley, 1988, 1995
11
Mechanisms of SA
Directs Attention
Provides Comprehension Projection
SA Guides Selection of Active Goal
Active Goal Selects Model
12
Alternating Data Driven and Goal Driven Processing

• Data Driven Processing (Bottom-up)
• Salient Cues Catch Attention
• Cues Interpreted
• Options Generated/Evaluated
• Option Selected
• Appropriate Actions taken
• Goal Driven Processing (Top-down)
• Goals Direct Attention
• Goals Determine Development of Level 2 SA
• Interpretation of information
• Direct Selection of Actions to bring Environment
  into line with Goals and Plans
• Goals Determine Selection of Model for
  Interpreting Information
• Model directs selection of plan generation of
  actions
• Scripts if available may be used

13
Key Features

• Novices Novel Situations
• Largely Data Driven
• Limited by Working Memory and Attention
• Expertise
• Develop Goal Driven Processing
• Develop Mental Models
• Pattern Matching to Prototypical Situations in
  Memory
• Direct Attention to Relevant Information
• Critical cues, key information, integration of
  information
• Provides for Comprehension Projection
• Develop Automaticity
• low attention demand
• limited awareness of novel cues

Rapid Situation Comprehension and Decision
Making Low Load on Working Memory
14
Factors Affecting SA
Stress Workload Fatigue
15
Designing for SA
16
SA Requirements Analysis

• Goal-directed Task Analysis
• Establish major goals subgoals
• Determine major decisions needed to meet each
  goal/subgoal
• Identify dynamic information needs associated
  with each decision subgoal
• Data required
• How data is used, integrated, combined to address
  each decision

17
SA Requirements Analysis

• Goals
• Sub-Goals
• Decisions
• Projection Requirements (Level 3 SA)
• Comprehension Requirements (Level 2 SA)
• Data Requirements (Level 1 SA)

18
Example of SA RequirementsAir Traffic Control
1.1 Separate aircraft 1.1.1 Assess aircraft
separation vertical separation meets or
exceeds limits? vertical distance between
aircraft along route (projected) vertical
distance between aircraft (current) aircraft
altitude (current) altitude accuracy altitude
(assigned) altitude rate of change
(climbing/descending) lateral separation
meets or exceeds limits? will aircraft
cross? will aircraft overtake? amount of
divergence present? time until diverge?
lateral distance between aircraft along route
(projected) lateral distance between aircraft
(current) aircraft position projected
aircraft route (current) (2.1.1) aircraft
capabilities (2.1.2)
Goals
Decisions
SA - Projection
SA - Comprehension

SA - Data
19
Designing for SA
SA Requirements Analysis
Evaluation of Impact on SA
20
Technology-Centered Design Philosophy
Design Technologies Let Human Adapt
21
Information Gap
Find
Sort
Integrate
Process

• More Data ? More Information

22
Challenge of the Information Age

• Overcoming the Information Gap to Develop an
  Understanding of the Information Available

23
Human-CenteredDesign Philosophy
Design technology to fit capability of humans
24
SA Oriented Designs

• Provide SA not Data
• focus on integrated system
• integrate according to goals
• higher levels of SA
• Take into Account Limitations of Attention,
  Sampling, Stressors
• beware of tradeoffs
• Support Global SA
• detailed information for specific goals on demand
• Trigger Critical Cues to Activate Situation
  Classification
• information salience
• Reduce Extraneous Data
• Support Parallel Processing

25
Example

• To develop an operator interface for the FAAs
  Maintenance Control Centers (MCC) that maximizes
  operator performance in terms of
• Monitoring system status
• Detecting system failures
• Diagnosing system failures
• Predicting preventing system failures

26
Problem

• Technology-Centered Interface Design
• Non-integrated data
• Data does not relate to operator goals
• Requires extra processing
• Dispersed data
• Requires extra, time-consuming tasks
• Induces Errors

27
Interface Integration Standardization

• Systems Need to be Used in Conjunction to Support
  OCC Activities
• Monitoring
• RMM based maintenance diagnosis
• Maintenance tasking
• Coordination logging
• Proactive maintenance
• Tracking scheduled unscheduled activities
• Event tracking to determine new OCC actions
• Interfaces Not Sufficiently Integrated
• Interfaces Inconsistent
• Display of information
• Functions interaction methods
• Pushbuttons in different places
• Labels different to do same tasks
• Menu options inconsistent
• Feedback different

28
User Orientation Mental Models

• System not sufficiently designed to meet user
  goals
• How will system be used?
• Filling out forms from phone calls
• Monitoring status of ongoing activities
• Display needs to facilitate users needs for
  information and reflect information
  prioritization
• Organization and presentation of information
  inadequate inconsistent with needs
• Most important information scattered, imbedded
• Status categories do not map to those relevant to
  maintenance personnel
• Color coding inconsistent with HCI standards

29
Poor Situation Awareness

• Difficult to maintain good awareness of state of
  the system
• Hard to find information needed
• Status of ongoing activities
• Which systems are/are not currently being
  monitored
• Event ticket for an alarmed system and
  disposition of troubleshooting activities
• Easy to get lost in the system
• Excessive hierarchical menuing

30
POCC Goals

• Communicate Coordinate
• Communicate SMO activity
• Determine actions needed
• Respond to aircraft accidents
• Facilities potentially related
• to problem
• Operational status of
• identified facilities
• Status of investigation
• Visibility of accident
• Communicate Coordinate
• Respond to national emergencies
• Coordinate major system restorations
• Reporting
• Daily status briefing
• Current major systems
• problems
• Service interrupts
• National trends

• Monitoring
• Assess national activity
• Facility status
• Weather
• Air traffic
• National events
• Resource base
• Control
• Resolve alarms/incidents
• Status of adjacent facilities
• Integrity of signal available
• to aircraft
• Likely cause of alarm
• Actions available
• Proactive control
• Verify system integrity
• Shutdown facility
• Provide alternate power
• Check and turn on antennae

31
SA Requirements Analysis
32
SA Requirements Analysis (cont)
33
System Status Overview
34
Facility Status
35
Sector Activity Overview
36
Activity Display
37
History Display
38
Advantages of SA Design

• Provides global overview with detailed
  information when needed
• Data organized according to goals
• monitor, diagnose, coordinate, control
• integrated in one source
• transition ease between goals
• Supports diagnosis in one place
• is alarm real?
• what does it relate to?
• will it get worse or go away?
• Less chance of information falling through the
  cracks, missing information, info not
  communicated between personnel
• Better support of sector specific information
• Easy to get needed information
• reduces communication coordination
• minimal wading through screens

39
Designing for SA
SA Requirements Analysis
SA-Oriented Designs
40
Iterative Design Process
41
Situation Awareness GlobalAssessment Technique
(SAGAT)
TIME

• Real-time man-in-the-loop simulation of system
  (rapid prototyping)
• At random times, freeze the simulation, blanking
  all displays
• Administer a rapid battery of queries to
  ascertain the subject's SA at that point in time
• Score the subject's SA on the basis of objective
  data derived from the simulation

42
Direct Measures SAGAT

• Pros
• Overcomes problems of collecting data after the
  fact while minimizing potential biasing of
  subject SA
• Direct measure of SA which is objectively
  collected and evaluated
• Use of random sampling provides unbiased measure
  of SA
• Cons
• Requires the interruption

43
Validation of SAGAT

• Content Validity
• Inclusive of SA elements
• (Endsley, 1990a)
• Construct Validity
• Does not impact on performance
• (Endsley, 1989, 1990)
• Criterion Validity
• Predictive of performance
• (Endsley, 1990b)

44
Example 3-D Display Investigation
45
SA Provided by Display Options
46
Conclusions

• The need to process and understand large volumes
  of data is critical
• Cockpits
• Military Missions
• Power plants
• Automobiles
• Space Stations
• Business
• Designing to meet the situation awareness needs
  of operators in these systems is the key to
  addressing the information gap and creating
  cognitive symbiosis
• The tools and methods needed for enhanced
  operator situation awareness are available for
  designing tomorrows systems

www.satechnologies.com