Fortuitous Limitations to the Utility of Training Systems

1
Fortuitous Limitations to the Utility of Training
Systems
Cognitive Systems  Human Cognitive Models in
System Design Hosted by Sandia National
Laboratories, University of New Mexico, and
United States Naval Research Laboratory  Santa
Fe, New Mexico / July 6th - July 8th, 2005

• Robert S. Kennedy
• RSK Assessments
• Kay M. Stanney
• University of Central Florida

2
Who will truly be able to benefit from advanced
training technologies?

• Tools for training and evaluation
• Tools for augmenting performance of individuals
  and teams
• Tools for automating human performance
• All of these technologies are intended to enhance
  training and operational performance
• Yet, there may be unforeseen limitations to their
  ubiquitous use

3
Tools for Training and Evaluation
4
Tools for training and evaluation

• Advanced training systems, such as virtual
  environments, allow for unprecedented realism and
  interactivity
• Yet, some individuals may not be able to fully
  benefit from such technology due to
• Emetic response
• Dropouts
• Individual factors (gender vs. susceptibility)
• Sopite syndrome
• Flashbacks
• Long-term aftereffects
• Perceptual illusions
• At-risk individuals
• Negative training

5
Emetic response

• Simulator Exposure 0.1
• Riding in automobiles 1.0
• Virtual Environments 1.5
• Of 1028 participants, 15 (1.46) vomited during
  or after exposure
• 7 M (1.2 of M) 8 F (1.9 F)

Kingdon, K., Stanney, K.M., Kennedy, R.S.
(2001). Extreme responses to virtual environment
exposure. The 45th Annual Human Factors and
Ergonomics Society Meeting (pp. 1906-1910).
Minneapolis/St. Paul MN, October 8-12, 2001
6
Time to emetic response

• 73 of those who had an emetic response did so
  within 30 min of exposure
• Average exposure
• 26.9 min (/- 15.7 min)
• Range 3 60 min

Kingdon, K., Stanney, K.M., Kennedy, R.S.
(2001). Extreme responses to virtual environment
exposure. The 45th Annual Human Factors and
Ergonomics Society Meeting (pp. 1906-1910).
Minneapolis/St. Paul MN, October 8-12, 2001
7
Dropouts

• Of 1097 participants, 133 participants (12)
  requested early termination of session (64
  M12.1 / 69 F14.9)
• Dropout distribution increased as exposure
  duration increased
• 15min, 9 dropouts16.7
• 60 min, 57 dropouts42.8

Stanney, K.M., Kingdon, K., Kennedy, R.S.
(2002). Dropouts and aftereffects  examining
general accessibility to VE technology. The
46th Annual Human Factors and Ergonomics Society
Meeting (pp.2114-2118). Baltimore, MD,
September 29-October 4, 2002.
8
Time to dropout

• 50.7 of dropouts did so
  within first 20 min of
  exposure
• Average dropout time
• was 24 min (SD 13.6)
• Range was
• 3 59 minutes

Stanney, K.M., Kingdon, K., Kennedy, R.S.
(2002). Dropouts and aftereffects  examining
general accessibility to VE technology. The
46th Annual Human Factors and Ergonomics Society
Meeting (pp.2114-2118). Baltimore, MD,
September 29-October 4, 2002.
9
Individual factors

• Individual factors make a difference
• Susceptibility may be more telling than gender

D. Graeber, Dissertation Data
10
Sopite Syndrome

• Of 960 participants, 43.8 experienced drowsiness
  after VE exposure
• Drowsiness positively correlated with VE duration
• 60-min group experienced 54 more severe
  drowsiness as compared to 15-min group
• May be an indication of sopite syndrome,
  characterized by lowered arousal or mood during
  or after VE use
• If sopite syndrome occurs among VE users, likely
  to affect performance without being fully
  detected by afflicted person

Stanney, K.M., Kennedy, R.S. (1998).
Aftereffects from virtual environment exposure
How long do they last? Proceedings of the 42nd
Annual Human Factors and Ergonomics Society
Meeting (pp. 1476-1480). Chicago, IL, October
5-9.
11
Flashbacks

• Flashbacks (i.e., visual illusion of movement or
  false sensations of movement, when not in VE)
  experienced immediately after VE exposure by 144
  of 960 participants (15.0)
• High incidence level for what is thought to be a
  rare outcome from VE exposure
• Could affect post-exposure performance

Stanney, K.M., Kingdon, K., Nahmens, I.,
Kennedy, R.S. (2003). What to expect from
immersive virtual environment exposure
Influences of gender, body mass index, and past
experience. Human Factors, 45(3), 504-522
12
Long-term aftereffects

• Adverse affects post exposure could compromise
  human performance
• At 2-4 hr post-exposure, 73 of participants
  still had symptoms substantially higher than
  pre-VE exposure
• More than 4 hr after VE exposure, 35 of
  participants still reported SSQ symptoms higher
  than pre-VE exposure levels

13
Long-term aftereffects
n366, number that returned take-home SSQ.
Stanney, K.M., Kingdon, K., Nahmens, I.,
Kennedy, R.S. (2003). What to expect from
immersive virtual environment exposure
Influences of gender, body mass index, and past
experience. Human Factors, 45(3), 504-522
14
Total SSQ Severity
32.76
SSQ Mean Score
18.75
1.94
Time Post-Exposure (min)
Stanney, K.M., Kennedy, R.S. (1998).
Aftereffects from virtual environment exposure
How long do they last? Proceedings of the 42nd
Annual Human Factors and Ergonomics Society
Meeting (pp. 1476-1480). Chicago, IL, October
5-9.
15
Its Not Over When Its Over
Objectively measured postural instability
Self-report of symptoms
Severity of Symptoms
Repeated Exposures
16
Perceptual illusions

• When sensorial transpositions are used in
  simulators of VEs (e.g., replace one sense with
  another), there is an opportunity for perceptual
  illusions to occur
• With perceptual illusions, certain perceptual
  qualities perceived by one sensory system are
  influenced by another sensory system
• This can lead to misperceptions upon
  post-exposure that could affect operational
  performance

17
Negative training

• Adopt behaviors in training system that could
  negatively impact real world performance
• For example, minimize head movements to minimize
  pseudo-Coriolis (i.e., experienced when head is
  tilted during illusory self-rotation induced by
  moving visual stimuli) while doing flight
  maneuvers in simulator
• Normally pilots do a lot of head and eye
  movements, thus may adopt adverse habits
  negative habit acquisition to get around
  simulator problems

18
Tools for Augmenting Performance of Individuals
and Teams
19
Tools for augmenting performance of individuals
and teams

• Augmented cognition seeks to substantially extend
  human abilities / performance via computational
  technologies explicitly designed to address human
  information processing (HIP) limitations
• Leverage diagnostic psychophysiological (e.g.,
  EEG, fNRI) sensors to gauge and detect HIP
  bottlenecks and then employ augmentation
  strategies to overcome limitations

20
Realizing augmented cognition

• To realize augmented cognition, first must
  characterize cognitive state to monitor and
  appropriately regulate HIP bottlenecks
• Use neural signatures as diagnostic tool of
  cognitive load, which can be measured in
  real-time while an individual interacts with
  training / operational system
• Yet, some individuals may not be able to fully
  benefit from such technology due to
• Neurological, psychiatric, sleep disorders, drug
  use
• Fatigue, environmental stressors, sleep loss
• Effects of practice

21
Neurological, psychiatric, sleep disorders, drug
use

• May not be able to obtain diagnostic neural
  signatures for those with neurological,
  psychiatric and sleep disorders, and those using
  illegal or prescription drugs that may affect the
  brain (e.g. narcotics, barbiturates or
  anti-psychotics)

22
Subject state

• Subject states can change how brain carries out
  its jobs, thereby altering neural signatures
• Fatigue
• Environmental stressors
• Sleep loss

23
Effects of practice

• Practice can change what neural signature looks
  like
• Parts of brain that are activated may change
  based on how practiced an individual is
• Part of brain used in acquisition of skill may
  change to another part of the brain after skill
  has been acquired
• Neural signatures will have to make allowance for
  human practice issue (i.e., habituation)

24
Tools for Automating Human Performance
25
Tools for automating human performance

• Well-designed automation can enhance human-
  system performance
• Yet, adaptive automation changes nature of work
• Sometimes given function executed by human, at
  other times by automation, and at still others by
  both human and computer
• Vigilance-arousal continuum
• If offload tasks from executive function, then
  ability to monitor those tasks reduced -
  complacency
• Thus, automation indices should be sensitive to
  changes in operator arousal

26
Automation surprises

• 28 of the 58 controllers responding to the
  survey indicated instances in which they had been
  ''surprised" by a reconfiguration of the system
  that had been carried out by a remote operator
  at the time they were not aware of the
  reconfiguration, but only discovered it later,
  when they tried to perform operations that failed
  in the new reconfigured mode. The potential for
  such mode errors is perhaps an inevitable
  downside of the flexible aspects of some
  automation functions.

Wickens, C.D., Mavor, A.S., Parasuraman, R.,
McGee, J.P. (Eds.). (1998). The Future of Air
Traffic Control Human Operators and
Automation. Commission on Behavioral and Social
Sciences and Education. Washington, DC
National Academy Press.
27
Automation and trust

• excessive trust of or excessive mistrust of
  automation on the part of controllers can lead to
  problems. The former can lead to complacency and
  reduced situation awareness, the latter to disuse
  or under-utilization. (Wickens et al., 1998
• Yet, there are individual differences in
  inclination to trust automation
  Complacency-Potential Rating (Singh, 1993)
  measures propensity to display complacent
  behavior and is related to boredom potential and
  cognitive failure (Prinzel et al., 2001)

Prinzel, L.J., DeVries, H., Freeman, F.G .,
Mikulka, P. (2001). Examination of automation
induced complacency and individual differences.
Technical Report NASA/TM-2001-211413. Langley
Research Center,, Hampton, VA. Singh, I.L.,
Molloy, R., Parasuraman, R. (1993).
Automation-induced complacency Development of
a complacency-potential scale.
International Journal of Aviation Psychology, 3,
111-122. Wickens, C.D., Mavor, A.S., Parasuraman,
R., McGee, J.P. (Eds.). (1998). The Future of
Air Traffic Control Human Operators and
Automation. Commission on Behavioral and Social
Sciences and Education. Washington, DC
National Academy Press.
28
Automation and culture

• Automation is viewed very differently by
  different cultures (McClumpha James, 1994) and
  thus may be differentially affective based on a
  users cultural background

McClumpha, A.J., James, M. (1994).
Understanding automated aircraft. In M. Mouloua
R. Parasuraman (Eds.), Human Performance in
Automated Systems. Hillsdale, NJ Erlbaum.
29
Individual factors

• Thus, automation strategy may have to vary by
• Operator arousal
• Complacency potential
• Culture
• and other individual factors

30
Conclusions

• While advanced training technologies are intended
  to enhance training and operational performance
• Unforeseen limitations to their ubiquitous use
  include
• Adverse effects during and after exposure that
  may preclude some individuals from being able to
  use the technology
• Inability to obtain diagnostic neural signatures
  to drive augmentation strategies from some
  individuals
• Inability to effectively modulate automation due
  to individual differences such as operator
  arousal, complacency, and culture