Implications of Adaptive vs. Adaptable UIs on Decision Making: Why

1
Implications of Adaptive vs. Adaptable UIs on
Decision Making Why Automated Adaptiveness is
Not Always the Right Answer
AAAI-SS on Interaction Challenges for Automated
Assistants 26-28 March, 2007
Originally presented at 1st Annual Conference on
Augmented Cognition Las Vegas, NV July 24-28,
2005

• Christopher A. Miller
• Harry B. Funk
• Robert P. Goldman
• John Meisner
• Peggy Wu

2
Adaptive vs. Adaptable

• Opperman, 1994
• Adaptive Systems are those that automatically
  adapt to user needs
• Adaptable Systems are those in which the user
  adapts system behavior to his/her needs.
• Learning vs. Instructing

3
Adaptive Automation as Meta-Automation
Adaptive Approaches
State, Needs
Adaptive Automation removes a human from an
aspect of the control loop with what results?
4
Benefits of Adaptive Automation

• Greater performance speed
• Greater performance consistency
• Improved safety/survivability
• Reduced training time

5
Disadvantages to Adaptive Automation

• High levels of automation (which remove operators
  from portions of the task/control loop) have been
  shown to
• Increase human workload at critical times if not
  properly designed 1,2
• Decrease human situation awareness 3,4
• Produce human complacency ( unwarranted trust)
  5
• Result in human skill degradation 6
• Decrease human acceptance 7 increase human
  dissatisfaction 8
• Provide inferior human machine system
  performance (relative to lower levels of
  automation) 9

• Parasuraman and Riley, 1997
• Wiener, 1988
• Endsley and Kiris, 1995
• Sarter and Woods, 1995
• Parasuraman, Molloy and Singh, 1993

1. Rose, 1989
2. Miller and Hannen, 1999
3. Vicente, 1999
4. Layton, Smith, McCovy, 1994

6
Characterizing the Tradeoff

• Competency is appropriate behavior in context.

• Competency gets harder with increases in
• Precision of context divisions
• Flexibility of behavior

• In any human automation system, a given level
  of Competency will entail a tradeoff between
• Human Workload
• (Un-)Predictability to User

• Increasing adaptivenss means either

• increasing human workload or

• increasing unpredictability,

• or both.

7

• So maybe taking the human out of the loop, even
  to do the humans bidding, isnt the best idea
• For user acceptance
• For overall human machine system performance

8
How do we want to interact with automation?

• Would you want/trust an assistant who always
  handed you ostensibly right information with no
  input/instruction/clarification from you?
• Wouldnt trust, coordination, SA and maybe
  performance be improved by explicit, meta-level
  intent coordination?

9
Advantages of (more) Adaptable Approaches

• Increased awareness of the situation and of
  system performance
• Better tuning of trust and better automation
  reliance decisions (since operators instruct
  automation how to behave)
• Better skill retention (through allowing the user
  to perform tasks when needed or desired)
• Better balance of mental workload (if humans are
  better assessors of their needs than AugCog
  systems)
• Better human machine performance (if/when users
  make better judgments than automation about what
  is needed to complement their abilities).
• Leaving the user in charge yields greater
  automation acceptance, AND greater sense of being
  primarily responsiblein turn yielding greater
  attention and concern for the situation and all
  aspects of system performance.

10
Its not Either/Or

• Adaptable or Adaptive
• Its a SPECTRUM
• Find the right point on it!

11
A Tale of Two Associates

• Pilots Associate (1985-1991)
• Single Pilot
• Direct pilot interaction with associate meant
  added workload
• Design philosophy minimized direct pilot
  interaction with associate
• Moderate user acceptance

• Rotorcraft Pilots Associate (1994-1999)
• Two Pilots
• 1/3 of human activity is crew coordination
• Design philosophy included some direct pilot
  interaction with associate
• Improved User Acceptance

12
Crew Coordination Task Awareness Display
HOVER MANUAL
AREA
MISSION
PILOT
ASSOCIATE
COPILOT

• Four buttons to convey major, associate-inferred
  task contexts
• Single press overrides No, youre wrong.
  Thats not what were doing
• Associate gets out of the way
• Press and Hold scrolls through tasks at same
  level of hierarchy
• E.g., Area Recon, Zone Recon, Attack in Force,
  Hasty Attack, Delay, Evade, Ingress, Egress, etc.

13
Subjective Workload (TLX) Ratings

• Workload levels consistently higher for Baseline
  (AMEP) than for RPA (CDAS)
• Significant differences for 4 of 6 TLX subscales
  (and close for the 5th)

14
CIM Utility and Overrides
Usefulness Ratings
3.68
Page Selection
Most crews said CIM Behaviors were Of Use or
Of Considerable Use
3.50
Symbol Selection
3.00
Window Location
3.56
Pan Zoom
1 Of no Use
2
3
4
5 Extrem- ely Useful
Pilot-reported Frequency of Overrides/Corrections
Not Very Useful
Of Con- siderable Use
Of Use
3.00
Page Selection
4.50
Symbol Selection
Crews Seldom overrode CIMs symbol selections,
but Now Then overrode other behaviors
2.63
Window Location
3.25
Pan Zoom
1 Always
2
3
4
5 Never
Fre- quently
Now Then
Sel- dom
15
CIM was seen as useful and provided perceived
performance and workload advantages in spite of
Now and Then or Frequently providing the
wrong information. Why?
16
Crew Coordination Task Awareness Display
17
Playbook Interfaces for Plan Delegation

• NUGGET Composable plays derived from Task
  Model shared between human and automation
  software.
• APPLICATION
• Plan specification to any level of specificity
  (calling plays)
• Automatic plan completion within human intent
• Feedback on plan feasibility
• STATUS Prototypes for
• UCAV pre-mission planning
• Tactical Robot Control
• Work on
• Unmanned Vehicle Teams
• Heterogeneous Urban RSTA
• UCAR
• Sim scenario specification

18
Conclusions

• Adaptation is good
• Not everything has to be (or SHOULD be) done by
  automation
• Who does the adapting is important
• He who does the adapting is
• Better aware of whats been adapted
• Better aware of why its been adapted
• Probably better served by the adaptation
• Humans dont like to be managed though there
  are some times when they should be
• Adaptable and Adaptive techniques can be
  complementary, not competing