Virtual Reality

1
Virtual Reality

• Dr. Yan Liu
• Department of Biomedical, Industrial and Human
  Factors Engineering
• Wright State University

2
Introduction

• What is Virtual Reality (VR) or Virtual
  Environment (VE)
• A medium composed of interactive computer
  simulations that sense the participants position
  and actions and replace or augment the feedback
  to one or more sense, giving the feeling of being
  mentally immersed or present in the simulation (a
  virtual world) (Sherman Craig, 2003)
• Four Key Elements in Experiencing VR (Sherman
  Craig, 2003)
• Virtual world
• An imaginary space, often (but not necessarily)
  manifested through a medium
• Immersion
• Having a sense of presence within an environment
  this can be purely a mental state, or can be
  accomplished through physical means
• Mental immersion
• A state of being deeply engaged, with a
  suspension of disbelief
• Physical immersion
• Bodily entering a medium

3
Introduction (Cont.)

• Four Key Elements in Experiencing VR (Sherman
  Craig, 2003)
• Sensory feedback
• Visual/aural/haptic feedback to participants,
  based on some aspects of their physical positions
• Interactivity
• In a virtual reality experience, participants are
  able to move around and change their viewpoint,
  generally through movements of their head
• Four Technologies that are Crucial for VR
  (Brooks, 1999)
• The visual (possibly also aural and haptic)
  displays that immerse the user in the virtual
  world and that block out contradictory sensory
  impressions from the real world
• The graphics rendering system that generates, at
  20 - 30 frames/second, the ever-changing images
• The tracking system that continually reports the
  position and orientation of the users head and
  limbs
• The database construction and maintenance system
  for building and maintaining detailed and
  realistic models of the virtual world

4
Introduction (Cont.)

• Four Technologies that are Important for VR
  (Brooks, 1999)
• Synthesized sound, displayed to the ears,
  including directional sound and simulated sound
  fields
• Directional sound is a technology that
  concentrates acoustic energy into a narrow beam
  so that it can be projected to a discrete area,
  much as a spotlight focuses light
• Display of synthesized forces and other haptic
  sensations to the kinesthetic senses
• Devices, such as tracked gloves with pushbuttons,
  by which the user specifies interactions with
  virtual objects
• Interaction techniques that substitute for the
  real interactions possible with the physical
  world
• Examples of VR
• A Desktop VR http//www.youtube.com/watch?vJd3-ei
  id-Uw
• A CAVE VR http//www.youtube.com/watch?v7_nUa4sFH
  Sofeaturerelated

5
Head-Mounted Display (HMD)

• Head-Mounted Display (HMD)
• A video display device mounted in a helmet,
  suspended one in front of each eye (in opaque
  HMDs) or projecting onto half-silvered mirrors in
  front of each eye (in see-through HMDs)

Full immersion HMD http//www.darpa.mil/MTO/Displa
ys/HMD/ Factsheets/immersion.html)
6
CAVETM

• CAVETM
• Provides the illusion of immersion by projecting
  stereo images on the walls and floor of a
  room-sized cube
• Advantages
• A wide surrounding field of view
• The ability to provide a shared experience to a
  small group
• Disadvantages
• The cost of multiple image generation systems
  (although not a serious limitation nowadays)
• Space requirement for rear projection
• 4-8 feet or more, depending on the size of the
  screen
• Brightness limitation due to large screen size
• Result in scenes of approximately full-moon
  brightness and hinder color perception
• Corner and edge effects that intrude on displayed
  scenes
• An alternative is to use Dome systems in which
  imagery is projected onto a hemisphere
  surrounding

7
An illustration of DOME
CAVE at NCSA (National Center for Supercomputing
Applications) at UIUC (http//cave.ncsa.uiuc.edu/a
bout.html)
8
Panoramic Displays

• Panoramic Displays
• One or more screens arranged in a panoramic
  configuration, or a single, curved screen on
  which images from multiple projectors are tiled
  together
• Especially suit groups multidisciplinary design
  reviews commonly use this type of display
• One person drives the viewpoint
• Issues
• Edge blending, viewpoint-dependent distortion
  correction, viewpoint-dependent gain correction

CURV, by Fakespace Lab (http//www.fakespace.com/
curv.htm)
9
Workbenches

• Workbenches
• Flat, rear-projection screens that display images
  in stereo and can be set up in a horizontal or
  tilted position

M1 DeskTM, by Fakespace Lab (http//www.fakespace.
com/M1Desk.htm)
Responsive workbench, by Dr. Krueger at
Stanford (http//graphics.stanford.edu/projects/RW
B/)
10
Boom (Binocular Omni-Orientation Monitor)

• Boom
• A head-coupled stereoscopic display device that
  the screen and optical system are housed in a box
  that is attached to a multi-link arm the user
  looks into the box through two holes, sees the
  virtual world, and can guide the box to any
  position within the operational volume of the
  device

Boom3C, by Fakespace lab (http//www.fakespacelabs
.com/tools.html)
11
Fishtank VR

• Fishtank VR
• A desktop VR system in which images are displayed
  on a desktop monitor, usually in stereo, and
  coupled to the location of the head which is
  tracked, resulting in the illusion of looking
  into a fishtank
• Commonly applied in CAD and design areas where
  immersion is not of much significance

Fishtank VR (http//www.faw.uni-linz.ac.at/save/)
12
Properties of VR Displays

• Spatial Resolution
• The ability of the system to spatially
  discriminate an object in the field of view A
  system with higher resolution can resolve an
  image with smaller size
• Because the smallest unit of an image is pixel,
  the resolution of a display is limited by its
  pixel size
• Temporal Resolution
• The time interval between images, or the number
  of frames captured per second
• Often there is a tradeoff between spatial
  resolution and temporal resolution
• Contrast
• The ratio of the brightest part of an image to
  the darkest part of the image
• Brightness
• The perceived amount of light

13
Properties of VR Displays (Cont.)

• Number of Display Channels
• e.g. RGB channels, luminance channel
• Focal Distance
• Distance from the center of the lens to the point
  that is in focus
• Opacity
• The amount of transparency of the display
• Field of View
• The angular extent of the observable world that
  is seen at any given moment
• Field of Regard
• The amount of space surrounding the user that is
  filled with the virtual world

14
Motion Tracking

• Usage
• In VR, tracking technology is required to monitor
  the real-time position and orientation of the
  users head and limb
• Mechanical Tracker
• A simple mechanical tracker can take the form of
  mechanical arm attached to the tracked object
• Very useful when integrated with a hand-held
  device
• e.g. Boom3C
• High accuracy and low latency due to its
  electromechanical nature
• Restricted active volume (movement)

15
Motion Tracking (Cont.)

• Optical Tracker
• Infrared video cameras that record the movement
  of a person
• Attached to the person is a collection of markers
  in the form of small balls fixed to a critical
  joints
• When the moving person is illuminated with
  infrared light the marker balls are readily
  detected within the video images
• Fast and low latency
• The system depends on the line-of-sight, so the
  orientation of the cameras must ensure that the
  markers are always visible
• Often prone to interference caused by ambient
  lighting conditions

ARTTrack1 and ARTTrack2, by Advanced Realtime
Tracking Inc. (http//www.ar-tracking.de)
16
Motion Tracking (Cont.)

• Ultrasonic Tracker
• Ultrasonic sound waves are used to locate the
  users position and orientation
• Usually used for fishtank VR in which the
  ultrasonic tracker is placed on the top of the
  monitor and records the users head movements
• Simple and low cost
• Slow, restricted active volume, sensitive to
  temperature and depends on the line-of- sight

Logitech Ultrasonic Head Tracker (http//www.i-gla
ssesstore.com/logtractracs.html)
17
Motion Tracking (Cont.)

• Electromagnetic Tracker
• Employ a device called a source that emits an
  electromagnetic field, and a sensor that detects
  the radiated field
• The source, which can be no bigger than a 2-inch
  cube, can be placed on a table or fixed to a
  ceiling
• The sensor is even smaller and is readily
  attached to an HMD or fitted within a 3D mouse
• Fast and very low latency no light-of-sight
  restriction
• Restricted active volume and are prone to
  interference of metallic objects

miniBIRD, by Ascension Technology Corp.
(http//www.ascension-tech.com/
products/minibird.php)
18
Interaction Devices

• Usage
• Allowing users to interact with virtual objects
• SpaceMouse/SpaceBall
• Hand-held device containing a tracker sensor and
  some buttons, used for navigating or picking
  objects within a VE
• 6 DOF operations
• Transitions in X, Y, Z axes and rotations around
  X (pitch), Y (yaw), and Z (roll) axes
• Some allow zooming in/out objects

SpaceMouseTM, SpaceBallTM 5000, by 3DConnexion
Corp. (http//www.vrlogic.com/html/3dconnexion/3d_
connexion.html)
19
Interaction Devices (Cont.)

• Gloves
• Gloves equipped with sensors that track the
  users hand movement
• Enable natural interaction with objects
• Modern VR gloves are used to communicate hand
  gestures (such as pointing and grasping) and in
  some cases return tactile signals to the users
  hand

Pinch Gloves, by Fakespace lab (http//www.fakespa
celabs.com/tools.html)
20
Haptic Devices in VR

• Usage
• A haptic device gives people a sense of touch
  with computer generated environments, so that
  when virtual objects are touched, they seem real
  and tangible
• e.g. A medical training simulator in which a
  doctor can feel a scalpel cut through virtual
  skin, feel a needle push through virtual tissue,
  or feel a drill drilling through virtual bone
• Current Technologies
• Force feedback joystick
• Virtual styluses
• Sensable phantom series, by SensAble Technologies
• Virtual gloves
• Immersion cyber series, by Immersion Corp.

21
Force Feedback Joystick

• Force Feedback Joystick
• A device allowing the users to feel force of
  magnitude and orientation, aside from measurement
  of depression and twist of its stick

Rumble Pak, by Nintendo (In most console video
game systems today)
22
Sensable Phantom Series

• By SensAble Technologies (http//www.sensable.com)
  

• Positional sensing X, Y, Z, pitch, roll, yaw
• Force feedback X, Y, Z
• Range of motion hand movement pivoting at wrist
• Maximum force 1.8 lbs
• Intended for use in haptic research and
  free-form modeling

Phantom desktop

• Positional sensing X, Y, Z, (pitch, roll, yaw
  with an additional separate encoder stylus
  gimbal)
• Force feedback X, Y, Z
• Range of motion hand movement pivoting at wrist
• Maximum force 1.9 lbs

Phantom premium 1.0
23

• Positional sensing X, Y, Z, (pitch, roll, yaw
  with an additional separate encoder stylus
  gimbal)
• Force feedback X, Y, Z
• Range of motion lower arm movement pivoting at
  elbow
• Maximum force 1.9 lbs

Phantom premium 1.5

• Positional sensing X, Y, Z, (pitch, roll, yaw
  with an additional separate encoder stylus
  gimbal)
• Force feedback X, Y, Z
• Range of motion full arm movement pivoting at
  shoulder
• Maximum force 4.9 lbs

Phantom premium 3.0
24
Pros and Cons of Virtual Styluses

• Pros
• Inexpensive
• Easy to set up and operate
• Works on a desktop
• Well suited for remote manipulation
• Cons
• Not immersive
• Haptic response at a single point only

25
Virtual Gloves

• Immersion cyber series, by Immersion Corp.
  (http//www.immersion.com/)

• Senses position of finger no force feedback

CyberGloveTM

• Adds tactile feedback to CyberGlove using
  vibrations on fingertips or palm
• Limited to simple pulses or sustained vibration

CyberTouchTM
26

• Force feedback for fingers and hand

CyberGraspTM

• Force feedback for hand and arm
• Can be used together with CyberGrasp

CyberForceTM
27
Pros and Cons of Virtual Gloves

• Pros
• Multiple points of haptic and tactic responses
• Allows for full immersion with HMDs
• Cons
• Expensive
• Difficult to set up and operate

28
Production-Stage Applications of VR(Brooks, 1999)

• Vehicle Simulation
• Ergonomics Evaluation and Design
• Training and Experience

29
Vehicle Simulation

• Vehicle Simulation
• Was the first application of VR and is still the
  most advanced
• Cases
• 747 simulator at British Airways
• Merchant ship simulation at Warsash Maritime
  Center

Ship bridge simulator at Warsash Maritime Center
30
Vehicle Simulation (Cont.)

• Lessons Learned
• Why are VR vehicle simulators useful?
• They are much cheaper to build than the real
  vehicles
• They make it possible to thoroughly train
  operators in extreme situations and emergency
  procedures where real practice would imperil
  equipment and lives
• Scenarios can be easily run and modified,
  enabling more efficient training
• What aspects of VR make it work so well?
• Immersion is complete (or nearly complete)
• The near-field haptics are perfect
• What aspects of VR are critical to success?
• Realism of the graphics
• Realism of the sound
• Realism of the haptics
• Realism of the motion
• Realism of the interaction (how does the display
  respond to the users actions?)

31
Ergonomics Evaluation and Design

• Cases
• Evaluating ergonomics in cars, at
  Daimler-Chrysler Technology Center
• Submarine design at General Dynamics
• Design review at John Deere

View of virtual wind-shield wiper visibility at
Daimler-Chryslers Technology Center
32
Ergonomics Evaluation and Design (Cont.)

• Lessons Learned
• Why is VR useful for ergonomics evaluation and
  design?
• Facilitate communication of ideas among team
  members
• Save the cost of materials used to develop
  physical prototypes
• Speed up design review and change cycles
• What aspects of VR are critical to success?
• True scales of the modeled objects
• The farther the design gets from its conceiver,
  the better the visuals need to be, in order to
  enable a factory-floor foreman or an operations
  person to get an accurate, internalized
  perception of the design

33
Training and Experience

• Cases
• Astronaut training at NASA-Huston
• Psychiatric treatment at Georgia Tech and Emory
  University Medical School
• Fear of flying
• Fear of heights
• Fear of public speaking
• Post-traumatic stress disorder for Vietnam War
  veterans

34
NASA-Houstons Charlotte (a haptic simulator)
virtual weightless mass lets astronauts practice
handling weightless massy objects

• (a) The psychologist gently leads to the patient
  into a simulated Vietnam battle scene
• (b) Imagery seen by the patient

(a)
(b)
Vietnam War simulation at the Atlanta Veterans
Administration Hospital
35
Training and Experience (Cont.)

• Lessons Learned
• Why is VR useful for training and experience?
• For NASA, it offers the ability to simulate
  unearthly experiences
• e.g. flying about in space using the back-mounted
  flight unit which is designed principally as an
  emergency device for use if an astronauts tether
  breaks moving around on the outside of a space
  vehicle
• For psychiatry, it can save cost (both money and
  time) and offers a safe form of exposure to
  traumatic stimuli
• What aspects of VR are critical to success?
• Immersion
• Haptics
• Sound

36
Open Challenges (Brooks, 1999)

• Technological
• Lowering latency to acceptable levels
• Rendering massive models in real time
• Choosing which display best fits each application
  (HMD, CaveTM, benchmark, or panorama)
• Producing satisfactory haptic augmentation for VR
  illusions
• Systems
• Interacting more effectively with virtual worlds
• Manipulation
• Specifying travel
• Wayfinding
• Making models of worlds efficiently
• Modeling the existing worlds
• Modeling the non-existing worlds
• Measuring the illusion of presence and its
  operational effectiveness

37
References

• Brooks Jr., F.P. (1999). Whats Real About
  Virtual Reality. IEEE Computer Graphics and
  Applications,19(6), 16-27.
• Sherman, W.R., Craig, A.B. (2003).
  Understanding Virtual Reality. San Francisco, CA
  Morgan Kaufmann.