Week 1: hardware applications

1
Week 1 hardware / applications
NASA Ames View Workstation, NASA Ames Photo
Archive
2
Nine Lectures

• Introduction
• Human factors
• Immersion / presence
• Three dimensional viewing issues
• Interaction 1
• Interaction 2
• Tracking
• Haptics and Auditory Systems
• Distributed / Collaborative / Telepresence
• Augmented Reality
• Usability

3
Human Factors

• Immersion / Presence
• Three dimensional viewing
• Effect of Stereo
• Effect of Depth cues

4
Immersive Virtual Environment

• Displays in all sensory systems
• Fully encloses participant in the displays
• Tracks the head, limbs, body
• determines the visual, auditory, haptic ...
  sensory data as a function of head tracking

5
Immersion the technology is...

• Inclusive sensory experience from VE only
• Extensive more sensory modalities
• Surrounding from all directions
• Vivid high fidelity
• Egocentric first person point of view
• Plot things are happening
• Proprioceptive match between sensory data and
  proprioception

6
Some Presence Definitions

• The sense of being there (Held Durlach,
  Sheridan, Zeltzer premier issue of PRESENCE,
  1992)
• A mental state in which a user feels physically
  present within the computer-mediated environment
  (Draper Kaber, 1998)
• The subjective experience of being in one place
  or environment, even when one is physically
  situated in another (Witmer Singer, 1998)

7
Presence Operationally

• Successful substitution of real sense data by
  computer generated sense data
• Successful response is similar to expected
  response in everyday reality
• Response
• Low level physiological ? high level cognitive
  and emotional
• Includes verbal responses about being there
• Response includes potential for interaction

8
Example Actors Rehearsing

• When two people interact within a VE how similar
  is their response to when they interact in
  reality?
• Eye movement patterns, heart rate, , emotions,
  thoughts, feelings
• These different responses may be
  self-inconsistent.

9
Examples

• Reading a book
• Playing a computer game
• Watching a movie
• HMD based VE
• Cave based VE
• Large area tracking based VE
• with whole body tracking
• with generation of tactile, haptic sensations .

10
Introduction to 3D Viewing

• Depth Cues
• Primary and Secondary cues
• Presenting Stereo Imagery
• Computing Stereo Pairs
• Ideals in Achieving Depth
• Head-mounted displays
• CAVEs

11
Primary (Physiological) Cues

• Accommodation
• Focal length of the eyes adjust in attempt to
  focus at points in the scene.
• Based on changing thickness of lens caused by
  relaxing and tensing the ciliary muscles.
• Convergence
• Eyes rotate inwards (near objects) become
  parallel (far objects)

12
Accommodation and Convergence

• Usually work in conjunction with each other.
• This correspondence is not physiologically
  determined.
• Learned by experience
• Is broken when looking at eg screen based stereo
  views.

13
Terminology

• Binocular disparity
• The difference between the two images produced by
  left eye and right eye.
• Motion Parallax
• How points move relative to one another with
  respect to head moves.
• Greater apparent movement usually implies smaller
  distance.

14
Secondary (Psychological) Cues

• Linear perspective
• Shading
• Shadows
• Occlusion
• Retinal image size (constancy scaling)
• Texture gradient

15
Presenting Stereo Imagery

• Time Parallel Techniques
• Both images are presented simultaneously
• Dual viewing
• Anaglyph (red/green) glasses
• Polarised glasses
• Main issue is cross-talk
• Separate viewing
• Head-mounted displays

16
Presenting Stereo Imagery

• Field-Sequential Displays
• Images are presented alternately
• Most common techniques is shutter glass stereo
• Takes advantage of persistence of vision
• Frame rate must be very high
• minimum 90Hz 45 Hz per eye

17
Stereo Pairs

• Stereo pairs Two projections, left and right eye
  on flat display.
• Horizontal parallax
• R-L
• R-L gt 0 called positive horizontal parallax
• R-L lt 0 called negative horizontal parallax
• Similar term for vertical parallax
• IPD inter-pupilary distance.

Left Eye
L
IPD
R
Right Eye
Stereo window
18
Effect of Parallax

• Positive parallax points will be virtual points
  behind the stereo window.
• Negative parallax points will be virtual points
  in front of the stereo window
• Note that the projected image points of a single
  point are called homologous points.

Left Eye
R
IPD
L
Right Eye
Stereo window
19
Viewing Stereo Pairs

• Uncrossed/parallel setup when right eye sees
  right image and left eye the left image
• Requires focus beyond the images
• Crossed setup when right eye sees left image and
  left eye sees right image
• Requires crossing eyes.
• Viewing the opposite way around will reverse the
  sense of depth.

20
Stereo Pairs
Keystone View Company - Ruins of the Granite
Temple, the Sphinx and the Great Pyramid (Oliver
Wendell Holmes Stereoscopic Research Library)
21
Ideals in Achieving Depth

• Congruence - left and right images should be the
  same except as caused by the horizontal parallax
• Avoid vertical parallax
• The image plane itself must be mapped to itself.

22
Ideals

• Wide parallax (separation in the views) produces
  good depth, but discomfort.
• Provide maximum depth but lowest parallax.
• Place principal objects so that approx half
  parallax values are positive, half negative.
• Further distance of viewer from display the
  greater the parallax that can be tolerated.

23
Ideals

• Cross talk is when left images reach right eye,
  and right images reach left eye
• For time dependent methods
• afterglow of phosphors
• departures from correct shutter speed
• For anagraphs (red/green filters) colours not
  properly filtered out
• Not same problem in other synchronous methods
  (HMDs)

24
Ideals

• Minimise impact of accommodation and convergence
  breakdown
• Use lowest possible parallax to get required
  depth effect
• The closer homologous points the less the
  disparity between accommodation and convergence.
• Make the parallax less than or equal to IPD
• Avoid screen edge effects

25
Head-Mounted Displays

• Simultaneously projects left-eye and right-eye
  disparate images.

http//www.gel.ulaval.ca/mbernat/rapporta/rapangl
3.htmlHMD helmet
26
Head-Mounted Displays

• Images formed on LCDs or CRTs
• Screens are small, low resolution, too close for
  direct viewing
• Optical system used to magnify and allow focus on
  the displays
• Distortion effects
• pixels magnified
• optics cause image warping and distortions

27
HMD Discussion

• Problems
• Incorrect convergence
• optical axes not parallel
• optical axes do not pass through centre of
  screens
• If so would correctly see far point at infinity.
• Accommodation and convergence not linked
• not much can be done about this

28
HMD Discussion

• FOV incorrect
• physical FOV
• geometric FOV
• they dont match
• Geometric COP doesnt match optical COP
• need off-centre COPs
• easily done in the general camera model

29
HMD Discussion

• Inter-pupillary distance ignored
• could allow mechanical
• optical
• software correction
• Optical distortion
• non-linear optical transformations
• straight lines become curves

30
CAVE-style Displays

• Screens surround the user
• Modelled as a series of cameras, two per wall
• Each camera defined by corners of the wall and
  centre of eye

31
CAVE Projection
32
CAVE Discussion

• Advantages of a CAVE
• Less rotational instability
• Wide field of view
• See yourself
• Higher quality images
• Less optical distortion

33
CAVE Discussion

• Disadvantages
• Expensive and complex to configure
• Need to align several projectors
• Very high refresh rate needed (gt100Hz)
• Projectors (used to) suffer from over-persistent
  green phosphor
• User occludes the screen with their own body
• Other users can occlude screen

34
CAVE Discussion

• Effect of head tracking Latency
• In a HMD
• Latency cause the world to appear to rotate in
  the same direction as head, then slow down
• In a CAVE
• Image is rotational stable, but some incorrect
  parallax effects are seen initially, then
  corrected
• In general CAVE causes less distortion with both
  rotation and displacement movement

35
Summary

• The types of cue that give 3D effect
• Limitations of stereo display
• Convergence/accommodation effects
• Display faults such as cross-talk
• More detailed look at one stereo system
  head-mounted displays