Improving Cockpit Task Management Performance:

1
Improving Cockpit Task Management Performance

• The AgendaManagerTraining Pilots to Prioritize
  Tasks

2
Observation Cockpit Task Management Errors

• Cockpit (flight deck) is a multitask environment
• aviate
• navigate
• communicate
• manage systems
• Results of distraction, preoccupation
• Everglades L-1011 accident
• many incidents
• Hypotheses
• flightcrew must manage as well as perform tasks
  Cockpit Task Management (CTM)
• CTM is a significant factor in flight safety

3
Preliminary Normative Theory of CTM

• initiate tasks to achieve goals
• assess status of all tasks
• terminate completed and obsolete tasks
• prioritize remaining tasks based on
• importance
• aviate
• navigate
• communicate
• manage systems
• urgency
• other factors (?)
• allocate resources (attend) to tasks in order of
  priority

4
Cockpit Task Management Research

• CTM Errors in Aircraft Accidents (1991)
• 80 CTM errors in 76 (23) of 324 accidents
• CTM Errors in Critical, In-Flight Incidents
  (1993)
• 349 CTM errors in 231 (49) of 470 incident
  reports
• Part-Task Flight Simulator Study (1996)
• CTM error rate increases with workload
• ASRS Study of CTM and Automation (1998)
• Task prioritization error rate higher in advanced
  technology reports
• Findings
• CTM is a significant factor in flight safety
• CTM can potentially be improved

5
Improving CTM Through Technology

• The AgendaManager

6
Statement of Needs and Requirements Definition

• CTM aid shall
• maintain a current model of aircraft state and
  current cockpit tasks,
• monitor task state and status,
• compute task priority,
• remind the flightcrew of all tasks that should be
  in progress, and
• suggest that the flightcrew attend to tasks that
  do not show satisfactory progress.
• leave the pilot in control

7
System Analysis

• Generic, twin-engine transport aircraft
• major subsystems
• power plant
• fuel system
• electrical system
• hydraulic system
• adverse weather system
• autoflight system
• flight management system.
• state variables of importance to pilot
• ? specifications for simulator

8
Basic and Detailed Design of The AgendaManager

• Object-Oriented Design
• things activities from IDEF0 models ? objects
• Multi-Agent Approach
• AMgt functions are complex, cognitive functions ?
  AI
• AMgt is complex interplay of many entities ? DAI
• System Agents
• Actor Agents
• Goal Agents
• Function Agents
• Agenda Agent
• Agenda Manager Interface
• Display Design
• general display design guidelines ? alternative
  display designs
• consistency with EICAS ? final display design

9
AMgr Architecture and Function
10
Simulator(with EICAS)
11
AMgr Display(replaced EICAS)
12
AMgr Operation

• simulator runs
• pilot declares goals via ATC acknowledgements
• System Actor Agents instantiate Goal Agents
• Goal Agents watch for goal conflicts
• Function Agents assess function status
• AgendaManager informs pilot via display

13
AgendaManager Display Design
extinguish L engine fire not OK -gt continuing
descend to 7,000 ft A/F alt goal conflict
maintain 070 deg
slow to 240 kt fast -gt accelerating
correct fuel balance L heavy -gt increasing
14
extremely important, urgent goals (highest
priority)
trend info
aviate goals (high priority)
extinguish L engine fire not OK -gt continuing
descend to 7,000 ft A/F alt goal conflict
system goals (lower priority)
maintain 070 deg
slow to 240 kt fast -gt accelerating
correct fuel balance L heavy -gt increasing
gray OK amber not OK red important/urgent
not OK
15
Initial Conditions altitude 15,000
ft heading 120 deg speed 280 kt all
systems normal

maintain15,000 ft
maintain 120 deg
maintain 280 kt

16
ATC ... descend and maintain 11,000 ft pilot
Roger, ... descend and maintain 11,000
ft sets A/F altitude to 11,000 ft descent
begins

descend to 11,000 ft high -gt descending
maintain 120 deg
maintain 280 kt

17
ATC ... turn left heading 070 pilot Roger,
... turn left heading 070 begins
turn levels off at 11,000 ft

maintain 11,000 ft
turn L to 070 deg right of -gt turning L
maintain 280 kt

18
pilot rolls out on 070 deg AMgr detects fuel
imbalance displays it

maintain 11,000 ft
maintain 070 deg
maintain 280 kt
correct fuel balance L heavy -gt unbalancing
19
pilot begins fuel crossfeed ATC ... descend
and maintain 9,000 ft reduce speed to 240
kt pilot Roger ... descend and maintain 9,000
ft reduce speed to 240 kt sets altitude to
9,000 ft, descent begins reduces throttles,
aircraft slows

descend to 9,000 ft high -gt descending
maintain 070 deg
slow to 240 kt fast -gt slowing
correct fuel balance L heavy -gt balancing
20
AMgr detects left engine fire pilot ... we
have a problem ... ATC ... descend and
maintain 7,000 ft pilot Roger ... descend and
maintain 7,000 ft mis-sets altitude to 6,000
ft speed increases
extinguish L engine fire not OK -gt continuing
descend to 7,000 ft A/F alt goal conflict
maintain 070 deg
slow to 240 kt fast -gt accelerating
correct fuel balance L heavy -gt balancing
21
fire out speed controlled pilot sets A/F to
7,000 ft forgets to secure crossfeed when fuel
balanced
maintain 7,000 ft
maintain 070 deg
maintain 240 kt
correct fuel balance R heavy -gt unbalancing
22
Test and Evaluation (1)

• Objective compare AMgt performance (AMgr vs
  EICAS)
• Apparatus
• flight simulator
• AMgr
• Subjects 8 line pilots
• Scenarios
• EUG to PDX
• PDX to Eugene
• Primary factor monitoring and alerting condition
• AMgr
• EICAS

23
Test and Evaluation (2)

• General Procedure
• subject introduction
• automatic Speech Recognition system training
• flight training (using MCP)
• subsystem training (fault correction)
• EICAS/AMgr training
• Trials
• Scenario 1 (EICAS/AMgr)
• experimenter/ATC controller gives clearances,
  induces faults, induces goal conflicts
• subject acknowledges clearances, flies simulator,
  corrects faults, detects and resolves goal
  conflicts
• Scenario 2 (AMgr/EICAS)

24
Evaluation Results
25
Conclusions

• CTM is a significant factor in flight safety.
• CTM can be facilitated (e.g., AMgr).
• Future success of knowledge-based avionics
  depends on a systematic approach to development
• systematic identification of problems, needs,
  and opportunities
• appropriate application of appropriate technology
• evaluation of systems based on operationally
  relevant performance measures

26
Improving CTM Through Training

• Training Pilots to Prioritize Tasks

27
ResearchMotivation and Objective

• Is task prioritization trainable?
• Evidence suggests that voluntary control of
  attention is a trainable skill
• e.g., Gopher (1992)
• Objective
• Develop and evaluate a CTM training program to
  improve task prioritization performance.

28
Methodology

• Participants
• 12 General Aviation pilots, IFR rated, with at
  least 100 hrs pilot-in-command total time.
• Recruited through flyers and word of mouth
• Oregon State (Corvallis, Albany, Salem, Eugene,
  Portland)
• Apparatus Microsoft Flight Simulator 2000
• 3 monitors, Flight Yoke, Throttles, and Rudder
  Pedals
• IFR conditions
• Two flight scenarios

29
Lab Setup
30
Participant Display(C-182RG)
31
Experimenters Display
32
Experimental Groups

• Control Group No Training
• Descriptive Group CTM lecture
• Multi-tasking
• Attention
• CTM
• Task Prioritization errors
• Accident/Incident examples
• What to be aware of.
• Prescriptive Group
• CTM lecture
• APE procedure

33
APEAssess Prioritize Execute
A P E

• Let the APE help you
• Assess the situation
• aircraft systems, environment, tasks, procedures
• Whats going on? What should I be doing?
• Prioritize your tasks
• Aviate Is my aircraft in control?
• Navigate Do I know where I am and where Im
  going?
• Communicate Have I communicated or received
  important information?
• Manage systems Are my systems okay?
• Execute the high priority tasks Now.
• Invoke the APE frequently.
• Think out loud.

34
Experimental Procedure

1. Initial briefing, informed consent
2. Initial 30-minute simulator training
3. Pre-training flight
4. CTM training (break for control group)
5. Additional 30-minute simulator training
6. Post-training flight (different scenario)
7. Post-experiment questionnaire

35
Dependent Measures

• Task prioritization error rate
• 19 Task prioritization challenges, e.g.
• clearance near end of climb
• bust altitude? (/- 200 ft)
• Prospective memory recall rate
• 5 Memory recall challenges (prospective memory),
  e.g.,
• report crossing SHONE intersection
• remember to report?

36
Data Collection

• Flight Data Recorder
• Videotape
• Observation
• Data reduction to
• task prioritization error rate
• prospective memory recall rate

37
Results ANOVA(task prioritization error rate)
38
Interaction Plot(task prioritization error rate)
39
Results ANOVA(prospective memory recall rate)
40
Interaction Plot(prospective memory recall rate)
Control
Descriptive
Prescriptive
41
Paired t-tests

• Prescriptive training group improved
• Task prioritization error rate
• Prospective memory recall rate
• Descriptive training group improved
• Task prioritization error rate
• Control group did not significantly improve

42
Discussion

• Task Prioritization Error rate
• Reduced, perhaps, due to (Prescriptive) CTM
  training.
• Significant interaction and post-hoc tests
  support that hypothesis.
• Prospective Memory Recall rate
• Increased, perhaps, due to (Descriptive
  Prescriptive) CTM training.
• Significant interaction and post-hoc tests
  support that hypothesis.

43
Possible Interpretations

• Results may have two interpretations
• CTM training did improve task prioritization
  performance.
• CTM training did not improve task prioritization.
• Floor effect
• MSFS experience
• Age
• Research favors first interpretation
• ANOVA results
• t-tests
• Potential for better control group performance
  was there.
• Additional tests

44
Final Comments

• CTM performance significant to flight safety
• Results are encouraging
• Evidence suggests that task prioritization is a
  trainable skill
• Follow-up experiment underway to resolve
  ambiguities
• If successful, would provide evidence that CTM
  training can reduce risk of CTM errors and
  subsequent accidents

45
The Cockpit Task Management Website

• http//flightdeck.ie.orst.edu/CTM/