Title: Effects of Handling Real Objects and Avatar Fidelity on Cognitive Task Performance in Virtual Environments
1Effects of Handling Real Objects and Avatar
Fidelity on Cognitive Task Performance in Virtual
Environments
- Benjamin Lok
- University of North Carolina at Charlotte
- Samir Naik
- Disney VR Studios
- Mary Whitton, Frederick P. Brooks Jr.
- University of North Carolina at Chapel Hill
- March 25th, 2003
2Current ImmersiveVE Approaches
- System has limited shape and motion information
of real objects. - Most objects in the VE are purely virtual
- User
- Tools
- Parts
- Most virtual objects are not registered with a
corresponding real object.
3Ideally
- Would like
- Real objects in the VE, including accurate
virtual representations, or avatars - Virtual objects responding to real objects
- Haptic feedback
- Correct affordances
- Constrained motion
- Example Unscrewing a virtual oil filter from a
car engine model
4Dynamic Real Objects
- By tracking and modeling dynamic objects (change
shape and appearance) would - Improve interactivity
- Enable visually faithful
- virtual representations
5Previous Work Incorporating Real Objects into VEs
- Non-Real Time
- Virtualized Reality (Kanade, et al.)
- Real Time
- Image Based Visual Hulls Matusik00, 01
- 3D Tele-Immersion Daniilidis00
- Augment specific objects for interaction
- Dolls head Hinkley94
- Plate Hoffman98
- How important is to get real objects into a
virtual environment?
6Previous Work Avatars
- Self - Avatars in VEs
- What makes avatars believable? Thalmann98
- What avatars components are necessary? Slater93,
94, Garau01
- VEs currently have
- Choices from a library
- Generic avatars
- No avatars
- Generic avatars gt no avatars Slater93
- Are visually faithful avatars better than
generic avatars?
7Object Reconstruction System
- System takes n live camera images.
- At each frame, generates a view of the visual
hull of the real objects from the user. - It is combined with the virtual environment
- 15-18 fps
- Allows incorporation of dynamic real objects into
a virtual environment
8Spatial Cognitive Task Study
9Study Motivation
- Effects of
- Interacting with real objects
- Visual fidelity of self-avatars
- On
- Task Performance
- For spatial cognitive manual tasks
10Spatial Cognitive Manual Tasks
- Spatial Ability
- Visualizing a manipulation in 3-space
- Cognition
- Psychological processes involved in the
acquisition, organization, and use of knowledge
11Task Performance Hypotheses
- Real Objects Participants will complete a
spatial cognitive manual task faster when
manipulating real objects, as opposed to virtual
objects only. - Avatar Fidelity Participants will complete a
spatial cognitive manual task faster with
visually faithful self-avatars, as opposed to
generic avatars.
12Task
- Manipulated identical painted blocks to match
target patterns - Each block had six distinct patterns.
- Target patterns
- 2x2 blocks (small)
- 3x3 blocks (large)
13Measures
- Task performance
- Time to complete the patterns correctly
- If errors were made
- Participants were notified
- Participants continued until all errors were
corrected - Other factors
- (After experience) Steed-Usoh-Slater Sense of
Presence Questionnaire (SUS) (larger study) - (Before experience) spatial ability
- (Before and after experience) simulator sickness
14Conditions
Purely Virtual
- All participants did the task in a real space
environment. - Each participant did the task in one of three VEs.
Real Space
Hybrid
Vis. Faithful Hybrid
15Conditions
Task performance
Avatar Fidelity Avatar Fidelity
Generic Visually faithful
Interact with Real objects HE VFHE
Interact with Virtual objects PVE
Task performance
16Real Space Environment
- Task was conducted within a draped enclosure
- Participant watched monitor while performing task
- RSE performance was a baseline to compare against
VE performance
17Purely Virtual Environment
- Participant manipulated virtual objects
- Participant was presented with a generic avatar
18Hybrid Environment
- Participant manipulated real objects
- Participant was presented with a generic avatar
19Visually-Faithful Hybrid Env.
- Participant manipulated real objects
- Participant was presented with a visually
faithful avatar
20Task Performance Results
Small Pattern Time (seconds) Small Pattern Time (seconds) Large Pattern Time (seconds) Large Pattern Time (seconds)
Mean S.D. Mean S.D.
Real Space (n41) 16.8 6.3 37.2 9.0
Purely Virtual (n13) 47.2 10.4 117.0 32.3
Hybrid (n13) 31.7 5.7 86.8 26.8
Visually Faithful Hybrid (n14) 28.9 7.6 72.3 16.4
21Task Performance Results
Small Pattern Time Small Pattern Time Large Pattern Time Large Pattern Time
T-test p T-test p
Purely Virtual vs. Vis. Faithful 3.32 0.0026 4.39 0.00016
Purely Virtual vs. Hybrid 2.81 0.0094 2.45 0.021
Hybrid vs. Vis. Faithful Hybrid 1.02 0.32 2.01 0.055
- significant at the ?0.05 level -
?0.01 level - ?0.001 level
22Debriefing Responses
- They felt almost completely immersed while
performing the task. - They felt the virtual objects in the virtual room
(such as the painting, plant, and lamp) improved
their sense of presence, even though they had no
direct interaction with these objects. - VFHE and HE participants felt tactile feedback of
working with real objects improved their
performance and sense of presence. - They felt that seeing an avatar added to their
sense of presence. - PVE and HE participants commented on the fidelity
of motion, whereas VFHE participants commented on
the fidelity of appearance. - VFHE participants reported getting used to
manipulating and interacting in the VE
significantly faster than PVE participants.
23Conclusions
24Study Conclusions
- Interacting with real objects provided a quite
substantial performance improvement over
interacting with virtual objects for cognitive
manual tasks. - Avatar fidelity does not appear to affect
cognitive task performance. - Handling real objects makes task performance and
interaction in the VE more like the actual task.
25Future Work
- Improved Object Reconstruction System
- model fidelity
- lag and latency (could improve results)
- Further studies to illuminate
- Effect of real objects and avatar fidelity on
- Sense of Presence
- Expand on the relationship between avatar
kinematic fidelity and visual fidelity
26Thanks
Collaborators Dr. Larry F. Hodges UNC-CH Effective Virtual Environments Object Reconstruction System I3D2001 and I3D2003 For more information http//www.cs.uncc.edu/bclok/research/vr2003 Funding Agencies The LINK Foundation NIH (Grant P41 RR02170) National Science Foundation Office of Naval Research