Interaction Techniques for Common Tasks in Immersive Virtual Environments

1
Interaction Techniques for Common Tasks in
Immersive Virtual Environments

• Design, Evaluation, and Application
• Doug A. Bowman
• April 27, 1998

2
Vision
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Immersive VEs for productivity
• Complex applications for real work
• Example immersive modeling and design

3
Definitions
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Interaction Technique (IT) Method used to
  complete a task via a human-computer interface
  (hardware software)
• Immersive VE A real-time 3D synthetic
  environment that appears to surround the user in
  space
• HMD with head tracking, CAVE
• Fishtank VR, MUDs, Multimedia apps

4
A Brief History of VEs
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• 1968 Sutherlands Ultimate Display
• Hardware advances
• displays trackers 3D graphics
• input devices haptics 3D audio
• Software advances
• view culling level of detail
• VE toolkits collision detection

5
VE Applications
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• In Use
• architectural walkthrough
• phobia treatment
• games (e.g. 1st person shooter)
• Proposed
• information visualization and retrieval
• modeling and design
• constructivist education

6
Interaction the Distinguishing Factor
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Current applications
• may involve movement through VE
• may involve shooting or pointing
• Proposed applications
• require 3D navigation and selection
• require 6 DOF manipulation (object placement)
• require large command spaces

7
How to improve VE Interaction
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• better design of techniques
• systematic evaluation (formative and summative)
• in the context of applications and requirements

8
Universal Tasks Travel
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Viewpoint Motion Control The users interactive
  control of the position and orientation of his
  viewpoint
• Wayfinding Cognitive process of determining a
  route, using landmarks, maps, etc.
• Navigation VMC Wayfinding

9
Universal Tasks Selection Manipulation
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Selection Specification of one or more objects
  from a set
• as the object of a command
• to begin manipulation
• Manipulation Specification of the position,
  orientation, and/or scale of an object

10
Why not natural interaction?
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Term VR implies replication of real world
• Why not use well-developed human skills to
  accomplish tasks in VEs?
• travel walking or driving
• selection manipulation grasp and place
• These mappings are intuitive, but too limited

11
Interaction Techniques and Input Devices
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• input devices are only the hardware component of
  an IT
• input device does not determine IT
• many ITs can be implemented with a single input
  device
• we will not design or evaluate devices
• we will design and evaluate ITs for common VE
  input devices

12
Problem Statement I will...
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• analyze universal tasks and create taxonomies of
  techniques
• design new techniques based on these formal
  frameworks
• design, implement, and conduct formal evaluations
  of IT performance
• apply the results to a complex and useful VE
  application

13
Design and Evaluation Methodology
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Taxonomization and Categorization
• Guided Design
• Performance Measures
• Range of Evaluation Methods
• Testbed Evaluation

14
Taxonomization and Categorization
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Task analysis
• Consider techniques for low-level subtasks
• Promotes deeper understanding of task
• Framework for design
• Framework for evaluation

Task
Subtask
Technique Component
15
Guided Design
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Design new techniques based on taxonomy, not
  simply intuition
• Choose a component for each low-level subtask
• Easy to see holes in design space

1
2
3
4
Task
Subtask
Technique Component
16
Evaluation Methods
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Range of performance metrics (quantitative and
  qualitative productivity and user-centric)
• Range of methods (user studies, usability
  evaluation, formal experiments)
• Consideration of outside factors (characteristics
  of task, environment, user, system that might
  affect performance)

17
Testbed Evaluation
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• testbed representative set of tasks and
  environments
• evaluate techniques for overall performance in a
  wide range of situations
• vary technique components and outside factors
• measure several performance variables
• generalizable and replicable

18
Summary of Methodology
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
Initial Evaluation and Design
Taxonomies
Perf. Metrics
Outside Factors
environment density users reach task
difficulty ...
speed accuracy comfort ...
19
Summary of Methodology
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
Initial Evaluation and Design
Taxonomies
Perf. Metrics
Outside Factors
environment density users reach task
difficulty ...
speed accuracy comfort ...
requirements
Applications
20
Summary of Methodology
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
Initial Evaluation and Design
Taxonomies
Perf. Metrics
Outside Factors
environment density users reach task
difficulty ...
speed accuracy comfort ...
TESTBED EVALUATION
requirements
Applications
21
Summary of Methodology
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
Initial Evaluation and Design
Taxonomies
Perf. Metrics
Outside Factors
environment density users reach task
difficulty ...
speed accuracy comfort ...
TESTBED EVALUATION
requirements
Performance Measurements/ Models
Applications
choice of techniques
22
Informal Evaluation
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• based on observations
• default gaze-directed steering
• lack of published work
• based on our own applications
• Conceptual Design Space
• Virtual GIS

23
Initial Taxonomy
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Task Move from the current location to the
  desired location

gaze-directed pointing physical props
Direction/Target Selection
gesture slow in, slow out physical props
Viewpoint Motion Control
Velocity/Acceleration Selection
start/stop buttons automatic start/stop constant
movement
Conditions of Input
24
Performance Measures
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Quantitative (e.g. speed, accuracy)
• Qualitative (e.g. presence)
• User-Centric (e.g. ease of use, comfort)

Quality Factors -speed -accuracy -cognitive
load -presence -spatial awareness - ...
IT
Apps
25
Simple Experiments(Bowman, Koller, and Hodges,
VRAIS 97)
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Absolute Motion
• no difference between gaze and pointing
• Relative Motion
• pointing superior to gaze
• Spatial Awareness
• teleportation causes disorientation
• any continuous motion does not

26
Expanded Framework
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Absolute vs. Relative Motion
• same techniques, different results
• highlights need to consider outside factors
• Consider task, user, system, and environment
  characteristics

Performance
absolute relative
Task
gaze-directed steering
pointing
27
Complex Experiment
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Does travel IT affect cognitive load?
• Task gather as much info as possible
• Variables
• IT gaze, pointing, torso
• Environment 1-, 2-, or 3-dimensional
• System collision detection

28
Guided Design
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Taxonomy tour technique
• environmental target selection
• gesture-based velocity selection
• explicit or automatic stop inputs
• Intuition travel based on manipulation
• cross-task technique
• still fits in taxonomy

29
Final Framework and Testbed
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Rework taxonomy to be more general
• task analysis 2 basic position-setting methods
  are specifying destination, specifying trajectory
• distinction allows better fitting of techniques
• VMC Testbed
• still in design stage
• based on evaluation framework

30
Initial Taxonomy
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Based on metaphor, not task
• Arm-extension metaphor
• touch and place object with virtual hand
• hand may extend beyond normal range
• Ray-casting metaphor
• point at object to select
• manipulate by attaching to virtual light ray

31
Informal Evaluation(Bowman and Hodges, I3DG 97)
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Studied six techniques (4 AE, 2 RC)
• Simple user study (comments, observations)
• Eleven subjects used techniques to place
  furniture in a room
• Results
• AE excels at manip., RC better at selection
• selection manipulation should be separated

32
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
HOMER Techniques

• Hand-Centered Object Manipulation Extending
  Ray-Casting
• Hybrid technique
• Select ray-casting
• Manipulate v. hand

Time
33
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
HOMER Techniques

• Hand-Centered Object Manipulation Extending
  Ray-Casting
• Hybrid technique
• Select ray-casting
• Manipulate v. hand

Time
34
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
HOMER Techniques

• Hand-Centered Object Manipulation Extending
  Ray-Casting
• Hybrid technique
• Select ray-casting
• Manipulate v. hand

Time
35
HOMER Techniques
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Hand-Centered Object Manipulation Extending
  Ray-Casting
• Hybrid technique
• Select ray-casting
• Manipulate v. hand

Time
36
Formal Taxonomy
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work
touch occlude
Indication of Object
Selection
button gesture
Indication to Select
hand moves to object user scales to touch object
Attachment
Manipulation
1-to-1 hand motion mapping
Positioning
match tracker orientation indirect control
Orientation
button gesture
Indication to Release
Release
Final Object Position/Orientation
remain in place use physics model
37
Evaluation Framework
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Performance measures similar to travel
• Important outside factors
• task characteristics DOFs to manipulate
• user characteristics reach, spatial ability
• system characteristics constraints used

38
Guided Design
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• testbed implemented to allow arbitrary
  combinations of technique components
• 4608 possible combinations - reduced to 667 via
  dependencies and constraints
• Taxonomy gaze-based HOMER with separate
  positioning and orientation
• Intuition manipulation based on travel
  (cross-task technique)

39
Selection/Manipulation Testbed
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Tasks that test all important aspects of a
  select/manip. IT
• Selection variables distance, size, density
• Manip. variables distance, accuracy, DOFs
  required

40
Application Case Study Immersive Design
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Verify evaluation results in a complex VE
  application
• Design system involves all universal tasks
• Choose ITs based on testbed results and specified
  application requirements

41
Interaction Requirements
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Travel
• exploration and goal-based movements
• spatial awareness, info gathering, ease of use
• Selection
• accuracy at a distance, speed, comfort
• Manipulation
• expressibility, accuracy, ease of use

42
Three Levels of Interaction Design
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• Naive design
• taken from CDS application
• (in D. Bertol, Designing Digital Space)
• gaze-directed steering, ray-casting
• Intuitive design iteration
• current implementation
• pen tablet, pointing, Go-Go technique
• Final, systematic design

43
Remaining Work
Introduction Methodology Travel Selection/Manipula
tion Application Remaining Work

• complete design and evaluation framework for
  travel
• design, implement, and run travel testbed
• complete and run selection/manipulation testbed
• modify application interaction design and verify
  with a usability study

44
Contributions

• formal understanding of tasks/techniques
• testbeds for future evaluations
• performance results and models
• new interaction techniques
• useful and usable immersive design application