Automaticity development and decision making in complex, dynamic tasks

1
Automaticity development and decision making in
complex, dynamic tasks

• Dynamic Decision Making Laboratory
• www.cmu.edu/DDMLab
• Social and Decision Sciences Department
• Carnegie Mellon University
• Cleotilde Gonzalez
• Rickey Thomas
• Polina Vanyukov

2
Complex and dynamic tasks

• Executing a battle, driving, air traffic
  controlling, managing of a production plan,
  piloting, managing inventory in a production
  chain, etc.
• Demand real-time decisions (time constraints)
• Demand attentional control
• Require multi-tasking they are composed of
  multiple and interrelated subtasks
• Demand the identification of targets defined by
  multi-attributes
• Demand multiple and possibly changing responses

3
Automaticity in dynamic, complex tasks

• targets and distractors are often inconsistently
  mapped to stimuli and responses
• Often, we bring pre-learned categories and
  mappings to a task
• stimulus - category category -
  response
• L ------------- letter button ---------
  click
• Are decision makers in dynamic situations
  operating in controlled processing continuously?

4
Proposed model of automaticity in DDM
Goals (Relevancy)
Task switching (resource allocation)
5
Experiments

• Automaticity develops with consistently mapped
  stimuli to targets, even when targets move and
  time is limited (Experiment 1)
• The consistency of target to response mapping
  also determines automaticity development
  (Experiment 2)
• Automaticity of a task component frees-up time
  and resources for high level decision-making
  (Experiment 3)
• Automaticity develops differently with different
  degrees of pre-learned categories (Experiment 4)

6
The Radar Task
7
(No Transcript)
8
General method

• Independent variables
• stimulus mapping (CM or VM)
• CM Search for Numbers in Letters
• VM Search for Letters in Letters
• cognitive load
• Memory set size (MSS) Number of possible targets
  to remember (1 or 4)
• frame size (FS) Number of blips present on the
  screen at a given time (1 or 4)
• target present/absent (a target was present 75
  of the trials)
• Dependent variables
• Accuracy proportion of correct detections or
  decision-making responses
• Time mean target detection or decision-making
  time in msec
• From 18 to 30 hours of practice, 3 hours per day
  6 to 10 days

9
Experiment 1 Consistency of stimuli

• Replicate major findings from the dual-process
  theory (Schneider Shiffrin, 1977) in a dynamic
  task
• Automaticity is acquired with practice in
  consistent mapping conditions, and automatic
  performance is unaffected by workload

10
Experiment 1 Method

• CM vs. VM
• Cognitive Load Variables
• Memory Set Size
• Frame Size
• Only one possible response pressing spacebar
  when target is detected

11
Experiment 1 Accuracy
12
Experiment 1 Detect Time
13
Experiment 1 Summary

• Radars manipulations of cognitive load interact
  with stimulus mapping in ways that parallel
  Schneider Shiffrins results
• Automaticity develops with extended practice and
  consistently mapped stimuli even when targets
  move and time is limited
• Radar task can be used to study automaticity in
  dynamic stimulus environments

14
Experiment 2 Response Consistency

• There is some evidence that response mapping is
  not critical for automaticity to develop (Fisk
  Schneider, 1984 Kramer, Strayer, Buckley,
  1991)
• In complex tasks mapping of targets to responses
  can be inconsistent
• Resulting in large processing costs, even when
  stimuli are consistently mapped to targets

15
Experiment 2 Method

• Only consistently mapped stimuli
• Cognitive Load Variables
• Memory Set Size
• Frame Size
• Response consistency varied in four levels

16
(No Transcript)
17
Experiment 2 Accuracy
18
Experiment 2 detect time
19
Experiment 2 Summary

• A consistent response reduces processing
  requirements
• Total task consistency (both, consistency of
  stimuli and consistency of responses) matters
• There are processing costs if responses are not
  consistently mapped, even when stimuli are
• Implications
• Interface design interface influences processing
  of responses
• Response selection using track-up vs. north-up
  displays
• Make response selection intuitive
• Interface design, decision support tools,
  training
• We can now systematically manipulate Radar to
  elucidate the effects of automaticity on
  high-level dynamic decision-making

20
Experiment 3 Automatic detection high-level
decision making

• How would automatic detection of a component help
  decision-making?
• Decision-making component required operators to
  analyze a sensor array of detected aircraft
• Sensor and weapon information changed dynamically

21
Experiment 3 Method

• Sensor Reading Task
• Determine if Target is Hostile
• Scan Sensors
• gt 13 (Hostile)
• lt 13 (Non-Hostile)
• Press Ignore (5-Key)
• Select Response (Weapon Systems)
• Guns vs. Missiles
• gt 10 Missiles (6-Key)
• lt 10 Guns (4-Key)
• Quiet Airspace Report
• No targets detected
• Click submit report with mouse key

22
Experiment 3 Detect Accuracy
23
Experiment 3 Decision-making Accuracy
24
Experiment 3 Detect Time
25
Experiment 3 Decision-making Time
26
Experiment 3 Summary

• Consistent mapping of targets improved he
  accuracy of the decision-making of the task
• Detect time, detect accuracy, and whole-task
  performance are sensitive to workload
  manipulations
• Implications
• Consistent mapping actually improved whole-task
  performance by freeing up time for the controlled
  sensor-reading tasks to run to completion
• Thus, processing speed-up associated with
  automatic detection can have a large impact on
  whole-task performance

27
But?

• Is accuracy of decision-making improved simply
  because there is more time to process?
• Effect of detection on high-level decision-making
  in the presence of a dual-task

28
Experiment 3b Method

• Secondary tone task enter count of number of
  non-standard tones
• Calibrated to standard tone at beginning of
  session for each participant
• Non-standard tones higher/lower pitch than
  standard

29
Experiment 3b results

• In fact the Radar task performance was the same
  with and without the tone task!
• Detect Time
• No Effect of secondary task
• Detect Accuracy
• No Effect of secondary task
• Decision-Making Time
• No Effect of secondary task
• Decision-Making Accuracy
• No Effect of secondary task

30
Experiment 3b Implications

• No effect of dual task on RADAR performance
• Operators are allocating resources away from tone
  task to maintain RADAR performance
• Implications
• Finding supports the hypothesis that consistent
  mapping improves decision-making performance by
  freeing up resources for other tasks
• Thus, processing speed-up and low resource
  requirement associated with consistent mapping
  can have a large impact on performance in complex
  task

31
Experiment 4 Categorization

• Since consistent mapping is the search for
  numbers in letters, it is possible that load-free
  processing is due to categorization (Cheng, 1985)
• Purpose of this experiment is to establish the
  presence of load-free processing without
  categorization

32
Experiment 4 Method

• Incorporate memory ensembles where no possible
  categorization can take place either a priori or
  with learning
• CM vs. VM with tone
• CM C, G, H, M, Q, X, Z, R, S
• VM B, D, F, J, K, N, W, P, L
• Memory ensembles were equated
• Angular H,M,X,Z,F,K,N,W vs. Round
  C,B,D,G,Q,P,R,J
• Beginning B,C,D,F,G,H,J,K vs. End
  M,N,P,Q,R,W,X,Z
• Cognitive Load Variables
• Memory Set Size (1 or 4)
• Frame Size (1 or 4)
• Indicated detection of target by pressing
  spacebar
• Detect Performance
• Detect Response Time

33
Experiment 4 Detect accuracy
34
Experiment 4 Decision-making accuracy
35
Experiment 4 Detect time
36
Experiment 4 Decision-making time
37
Experiment 4 Implications

• Varied mapped performance is more sensitive to
  load than consistently mapped performance
• Individuals performed better in the high-level
  decision-making component of Radar when stimulus
  mapping was consistently mapped
• Implications
• Categorization is NOT a necessary requirement for
  automaticity development
• Consistent stimulus mapping is a necessary
  condition for the development of automatic
  detection

38
Summary of accomplishments

• Developed Radar, a dynamic simulation where it is
  possible to study (i.e., to measure) automaticity
• In Radar it is possible to elucidate the effects
  of automaticity on high-level dynamic
  decision-making
• Established the usefulness and applications of
  the dual-process theory of automaticity
• Deepen our understanding of the implications of
  automaticity development for practical real-world
  tasks
• Brought together two main theories of
  automaticity instance-based theory and
  dual-process theory

39
Future research

• Consistency of mapping and responding is relative
  to the categories (i.e., similarity) that a user
  can form
• Thus, consistent mapping can lead to automatic
  responses for high-level decision-making after
  extended practice

40
Looking towards applications

• Test these hypotheses in airport luggage
  screening
• Decide whether to hand search the luggage
• There is no consistency but rather just
  similarity (relative to a knife category)