Department of Psychology s Institute for Simulation and Training

1
Measuring the Load of Mental Work Methods and
Techniques
P.A. Hancock
Presentation for the Class of 2008 Human Factors
II EXP 6257 January 24th, 2008
Department of Psychology s Institute for
Simulation and Training University of Central
Florida Orlando, FL 32826
2
Measuring the Load of Mental Work
Structure of the Lecture i) What is Mental
Workload? ii) Why Measure Mental
Workload? iii) Methods to Measure Mental
Work. iv) Current Applications and Future
Directions.
Hancock, P.A., Meshkati, N. (1988). (Eds.).
Human mental workload. North-Holland
Amsterdam Available at www.mit.ucf.edu
3
What Is Mental Workload?

• Mental workload is the portion of operators
  limited mental
• capacities actually required to perform a
  particular task.
• Mental reserves are the difference between
  capacity required
• and capacity available.
• Mental effort is the voluntary matching of mental
  capacities
• with that needed for task success.
• Increase in Mental Workload often precedes
  Performance Failure.

4
The Need To Measure
Mental Workload
Comes From The Changing Nature of Work
5
But Still
Work Overload
6
How Hard Are These People Working?
Apparent Underload
Apparent Overload
7
Measuring the Load of Mental Work
There are FOUR Basic Methodologies i)
Primary Task Performance. ii) Subjective
Response. iii) Physiological Assessment
(Central and Peripheral). iv) Secondary Task
Techniques.
Meshkati, N., Hancock, P.A., Rahimi, M.,
Dawes, S.M. (1995). Techniques of mental workload
assessment. In J. Wilson and E.N. Corlett,
(Eds.). Evaluation of human work A practical
ergonomics methodology. London Taylor
Francis. Available at www.mit.ucf.edu
8
Measuring the Load of Mental Work
Primary Task Performance
Hancock, P.A., Meshkati, N. (1988). (Eds.).
Human mental workload. North-Holland
Amsterdam Available at www.mit.ucf.edu
9
Primary Task Techniques
Measures the Performance Outcome as a function
of Primary Task Demand
How well are you Driving?
How well are you Flying?
Notice that its not so easy to specify well in
complicated performance environments.
10
Primary Task Techniques
As task load increases, the additional demands
on mental capacities result in a degradation in
performance Advantages of This Measure
Workload reflected directly by performance
outcome. Non-invasive and non-interfering. Track
s changes in workload dynamically. (i.e., as
performance proceeds) Uncontaminated by memory
issues Disadvantage of This Measure Only
sensitive to changes in workload at the limits of
mental capacity If operators can compensate for
increased workload by increasing their Effort,
the primary task measure is insensitive Mental
Workload not distinguished from performance
outcome
11
Primary Task Techniques
What About Failure? Most of the time wed like
to know about Mental Workload to know how much is
too much? Primary Task measures do not tell us
this. So, - they fail to be informative just at
the time they are needed most!
12
Multiple Task Demands
Secondary Task Performance
13
Secondary Task Measures
Maximum Capacity (Without Impaired Performance).
S1
S3
S2
S2
Performance Level
S1
P
S3
P
P
E M D
PTD
Easy (E)
Difficult (D)
Moderate (M)
Primary Task Demand (PTD)
14
Measuring the Load of Mental Work
Subjective Measures
Hancock, P.A., Caird, J.K. (1993).
Experimental evaluation of a model of mental
workload. Human Factors, 35, 413-429. Available
at www.mit.ucf.edu
15
Subjective Responses

• If You Want to Know How Hard Someone is Working
  Ask Them
• You Can Formally Ask Them Through Standard
  Techniques.
• Two of the Most Popular are SWAT and NASA-TLX.
• The Advantage It is Easy to Do and Has high Face
  Validity
• The Disadvantage Often Performance and
  Perception Deviate

16
Subjective Workload Assessment Technique (SWAT).

• Developed by the United States Air Force.
• There are THREE Sources of Workload Time,
  Effort, and Stress.
• Each has THREE Levels 1Low, 2Medium, 3High.
• You begin by Putting the 27 Cards
• (3 Sources X 3 Levels X 3 Combinations) into
  Order from 1-27.
• Subjects rate each EVENT by giving a number for
  each,
• (e.g., Time2, Effort1, Stress3).

17
NASA-TLX (Task Load Index).

• Developed by NASA (duh!).
• There are SIX Sources of Workload
• Temporal Demand, Effort, Stress, Own Performance,
  Frustration, Physical Demand
• Each is Compared Pairwise against the Others to
  give a
• Rank Order (0-5).
• Subjects rate each EVENT by giving a 0-100 score
  for each
• Source.

18
Measuring the Load of Mental Work
Physiological Measures
Central vs. Peripheral NS Measures
Hancock, P.A., Meshkati, N., Robertson, M.M.
(1985). Physiological reflections of mental
workload. Aviation, Space, and Environmental
Medicine, 56, 1110-1114. Available at
www.mit.ucf.edu
19
Physiological Reflections of Mental Workload
Electroencephalography (EEG)
Pupillometry/ Eyetracking
Electrocardiography (ECG)
Eyeblink Reflex Modification
20
Pupil Diameter

• Kahneman Beatty (1966 1967) Hakerem Sutton
  (1966) Hess Polt (1964)

21
Visual Scanning and Workload

• Used extensively in aviation research
• Tole, et al. (1982)
• Visual scanning behavior and mental workload in
  aircraft pilots
• gaze characteristics on cockpit instruments
  varied as a function of the level of difficulty
  of a verbal loading task
• mean dwell time of each fixation on the pilot's
  primary instrument increased as a function of
  task load
• scanning behavior was also a function of the
  estimated skill level of the pilots, with novices
  being affected by the loading task much more than
  experts.
• Authors argue that visual scanning of instruments
  in a controlled task may be an indicator of both
  workload and skill

22
Visual Scanning and Workload
Head Mounted Version
Desk Mounted Version
23
Time-Based ECG and Workload

• Bonner Wilson (2001)
• Monitored pilots throughout test and evaluation
  of an aircraft
• Note differences between subjective workload and
  HR

24
Measure HRV Heart Rate Pupil Diameter Eye Tracking EEG ERP
Reliability ? ? ? ? ? ?
Validity ? ? ? ? ? ?
Sensitivity ? ? ? ? ? ?
Diagnosticity ? ? ? ? ? ?
Invasiveness ? ? ? ? ? ?
Cost ? ? ? ? ? ?
Post-Processing ? ? ? ? ? ?
Temporal Sensitivity ? ? ? ? ? ?
25
Measurement Techniques Advantages and
Disadvantages
i) Primary Task Performance. (Data Easily
Available, Future Failure Unpredictable). ii)
Secondary Task Technique. (Diagnostic,
Administration is Intrusive).. iii)
Subjective Measures. (High Face Validity, Often
Dissociate). iv) Physiological
Assessment. (Unobtrusive, Expensive Data but
getting cheaper).
26
Current Applications and Future Directions

• Aviation
• Pilot workload
• Maritime
• Ship navigation
• Ground
• Car and bus drivers workload
• Air traffic control
• Automation cueing modulates cerebral blood flow
  and vigilance in a simulated air traffic control
  task
• Shift-work
• Performance dependent upon shift and workload
• Peacekeeping
• Differential workload of peacekeepers
• Business costs
• employee benefits managers are hoping that
  technology will help them cope with increasing
  workloads

• Employee burnout
• Training effectiveness
• Human computer interaction
• Home-care
• Nursing
• Hospital readmissions
• Parenting
• Consumerism
• Professor productivity
• to improve academic quality
• Student success

To name just a few...
27
Variance in Shear Forces
CURRENT APPLICATIONS AND FUTURE DIRECTIONS
After Marras, W. (2005), Ohio State University
Combined Physical and Mental Workload
If 20 of the Forces calculated in Physical Work
are Personality Factors, what is that number for
Cognitive Work.
After Marras, W. (2005), Ohio State University
28
Neuroergonomics
Neuroergonomics involves the examination of the
neural bases of perceptual, cognitive and
motor functions in relation to real-world
applications as mediated through Machines.
Adaptive Human-Machine Systems
Hancock, P.A. (1997). On the future of work.
Ergonomics in Design, 5 (4), 25-29.
29
RESOLUTION SPACE OF BRAIN IMAGING TECHNIQUES
FOR ERGONOMIC APPLICATIONS
More invasive, Less practical
TCDS
20 cm
EEG, ERPs
10 cm
NIRS
SPATIAL RESOLUTION
PET
Less invasive, More practical
1 cm
MEG
fMRI
5 mm
0.1 mm
1 ms
10 ms
2 s
2 min
TEMPORAL RESOLUTION
30
Workload and Vigilance
Vigilance is a Long-Standing Problem. Sources of
Performance Influence Include Event Rate
Signal Salience Stress/Workload/Fatigue
Glare, Noise, Temperature, Vibration, TOD, Drug
Effects etc Memory Load Successive vs.
Simultaneous Comparisons Feedback Hit
vs. Miss vs. FA KR/KP Individual Differences
Introversion/Extraversion, Age, Sex,
Expertise
Warm, J.S. (Ed.). (1984). Sustained attention in
human performance. New York Wiley.
31
P.A. Hancock, D.Sc., Ph.D. Department of
Psychology, and The Institute for Simulation
and Training University of Central
Florida Orlando, FL 32826 407-823-1492 phancock
_at_pegasus.cc.ucf.edu www.mit.ucf.edu
32
Orlando, Florida, January, 2007. Human Factors II
Personal Biography
Peter A. Hancock Professor of Psychology and
Institute for Simulation and Training,
University of Central Florida
The Load of Mental Work
Peter Hancock is Provost Distinguished Research
Professor in the Department of Psychology, the
Institute for Simulation and Training, and at the
Department of Civil and Environmental Engineering
at the University of Central Florida. He also
holds courtesy appointments at Minnesota,
Michigan, and MIT. He is the author of over five
hundred refereed scientific articles and
publications including Human Performance and
Ergonomics Stress, Workload, and Fatigue and
Essays on the Future of Human-Machine Systems. He
has been continuously funded by extramural
sources for every year of his professional
career, including support from NASA, NIH, NIA,
FAA, FHWA, the US Navy, the US Army and the US
Air Force. In 2000 he was awarded the Sir
Frederic Bartlett Medal of the Ergonomics Society
of Great Britain for lifetime achievement. He was
the Keynote Speaker for the 2000 Meeting of the
International Ergonomics Association. In 2001 he
won the Franklin V. Taylor Award of the American
Psychological Association and in association with
his colleagues Raja Parasuraman and Anthony
Masalonis, he was the winner of the Jerome Hirsch
Ely Award of the Human Factors and Ergonomics
Society for 2001. In 2002, he was awarded the
Jastrzebowski Medal of the Polish Ergonomics
Society for contributions to world ergonomics and
in the same year was named a Fellow of the
Ergonomics Society of Great Britain. He has been
elected to a second, three-year term as a Member
of the National Research Councils Committee on
Human Factors. In 2003 he won the Liberty Mutual
Medal of the International Ergonomics
Association. His current experimental work
concerns the evaluation of time an behavioral
responses to high-stress conditions. His
theoretical works concerns human relations with
technology and the possible futures of this
symbiosis. He is a Fellow of and past President
of the Human Factors and Ergonomics Society.
Further Information can be garnered at
www.mit.ucf.edu
33
(No Transcript)
34
(No Transcript)
35
Workload-Performance Dissociations
Measure of Subjective Workload
Supply of Resources
Underload
Measure of Performance
Overload
Task Demand (difficulty)
36
With the Transition from Physical to Mental Work
Comes the Need to Measure Workload
All Citations Available At www.mit.ucf.edu
37
Workload Applied
Technology is a way of organizing the universe so
that man doesn't have to experience it. -- Max
Frisch

• Aviation - Automation Workload in Aviation
  Systems

• The superiority of Adaptive Automation
• The allocation of a task or a function between
  the operator and the system which is flexible
  and responsive to operators performance and
  level of workload
• Improve situational awareness, regulate
  workload, improve vigilance in high-risk
  environments, and help to maintain manual control
  skills. (Mouloua, Deaton, Hitt in Hancock
  Desmond, 2001 Parasuramanm, Bahri, Deaton,
  Morrison, Barnes, 1992)

• Adaptive Automation
• Adaptive automation (AA) for managing operator
  workload through dynamic control allocations
  between the human and machine over time
• Low-levels of automation - superior performance
  
• Intermediate levels of automation - higher SA
• Not associated with
• Improved performance
• Reduced workload.
• (Kaber Endsley, 2004)

38
Human Performance under Stress

• Information Overload/Complexity
• Physical/Social Environment
• Uncertainty

39
EEG