Output Devices: Graphics, 3D Sound, and Haptic Displays

1
Output DevicesGraphics, 3-D Sound, and Haptic
Displays
2

Output Devices

• The human senses need specialized interfaces
• Graphics displays for visual feedback
• 3-D audio hardware for localized sound
• Haptic interfaces for force and touch feedback
• Not interested in smell and taste feedback.

3

Output Devices

• Definition
• A graphics display is a computer interface that
  presents synthetic world images to one or several
  users interacting with the virtual world.

4

Output Devices

• Graphics Displays
• Human stereo viewing
• Personal displays
• Large volume displays
• Active glasses
• Workbenches
• Caves
• Walls

5

Output Devices

• Human Visual System
• Vision is the dominant sensorial channel
• Depth perception in mono images is based
• - on occlusion (one objects blocks another from
  view
• on shadows, textures and motion parallax
• (closer images appear to move more than
• distant ones)

6

• Human Visual System-continued
• Depth perception in stereo is based on
  seteropsis when the brain registers and fuses two
  images
• Image parallax means that the two eyes
• register different images (horizontal shift)
• The amount of shift depends on the
  inter-pupillary distance (IPD) (varies for each
  person 53-73 mm)
• Works in the near field (to a few meters from
  the eye)

7

Output Devices

• Implications for Stereo Viewing devices
• Need to present two images of the same VR
  environment
• The two images can be presented at the same time
  on two displays (HMD)
• The two images can also be presented
  time-sequenced on one display (active glasses)
• The two images can also be presented
  spatially-sequenced on one display
  (auto-stereoscopic displays).

8

Output Devices Active
9

Output Devices Active
The active display operates by very rapidly
alternating between the left-eye and the
right-eye image in the same space.  Usually, this
is done at about twice the frame-rate necessary
for continuity of motion (120Hz).  Special
glasses must be worn in order to view this
method.  These glasses have lenses which turn
from opaque to transparent in perfect synch with
the transformation of the image.  The left image
is only displayed when the left lens is
transparent (and the right one is opaque) and
vice versa
10

Output Devices (Passive)
The passive method of displaying stereoscopic
images is better suited for large groups, since
the expensive technology is primarily in the
display rather than each pair of glasses
(polarized glasses run about 1.20 a pair).  This
is the method that Geowall utilizes. The concept
is simple project both the left eye and the
right eye simultaneously into the same space, and
then use a special set of glasses which shows
each image to its intended eye while blocking out
the unintended image from that eye
11

Output Devices
presented spatially-sequenced on one display
12

Output Devices
Experiment
With this method the images to be shown to the
right and left eyes are either encoded in shades
of blue or red, then combined. Since the colored
lens of the glasses filter out any color but
their own, the left eye, which has a red lens,
only sees the red shades and the right eye, which
has the blue lens, sees the blue shades
13

Output Devices
Experiment
Take a look at the four colored dots in the
middle of the figure at the top of the page
through the blue/red glasses. You should now only
see two dots, one above the other. One of those
dots should appear slightly closer than the other
dot. The reason you see only one dot when you are
really looking at four different colored dots is
because the eye looking through the blue filter
sees only the red dot and the eye looking through
the red filter sees only the blue dot.
14

Output Devices
Experiment
Binocular Disparity Humans use numerous visual
cues to perceive distance. One of the most
accurate of those cues is binocular disparity.
Binocular disparity takes advantage of the fact
that you have two eyes. Because the eyes are
separated by about 2 inches, each eye has a
slightly different view of the world. Binocular
Rivalry Binocular vision requires that the brain
integrate information from both eyes to determine
true stereoscopic depth. As in the binocular
disparity example, the brain matches up images in
the left eye with corresponding images in the
right eye. Binocular rivalry dramatically
demonstrates what would happen if the image in
the left eye was different from the image in the
right eye
15

Output Devices
Personal Displays

• Definition
• A graphics display that outputs a virtual scene
  destined to be viewed by a single user. Such
  image may be monoscopic or stereoscopic,
  monocular (for a single eye) or binocular
  (displayed on both eyes).

16

Output Devices

• Personal Displays
• Head Mounted Displays
• 3-D Binoculars (hand supported)
• Booms (floor supported)
• Virtual windows (floor supported)
• Auto-stereoscopic displays (desk supported).

17

HMD integration in a VR system
Consumer HMD
Professional HMD
18
AMLCD display, Resolution 267x225 FOV 30x23
degrees Equivalent to 62 in at 2 m Weight 100
grams Can be worn over glasses
Olympus Eye Trek Face Mounted Display (FMD 200)
19
Olympus Eye Trek Face Mounted Display Optics
20
Daeyang cy-visor Face Mounted Display
LCOS display, Resolution 800x600 FOV 60x43
degrees Weight 160 grams Can be worn over
glasses
Liquid Crystal on Silicon display (LCOS)
21
5DT Head Mounted Display
800x600 pixels 40o diagonal view Organic
LED Frame sequential stereo 600 grams 4k
22
Samsung Emagin z800 OLED HMD
Weight 8 oz PC connection - USB, RGB input SVGA
resolution (800x600 pixels) stereo Tracking - 360
degrees pan 60 degrees pitch 899 USD
www.3dvisor.com
23
Professional HMDs
Keiser ProView AMLCD display, Resolution
1024x768 FOV 28x21 degrees Weight 992 grams
Active Matrix Liquid Crystal Display
24
Professional HMDs
N-Vision Datavisor CRT display, Resolution
1280x1024 FOV 78x39 degrees Weight 1587 grams
25
LCOS Virtual Binoculars
26
Virtual Binoculars
Liquid Crystal on Silicon
27
Floor-supported displays
forward kinematics, the length of each link and
the angle of each joint is given and we have to
calculate the position of any point in the work
volume
Boom3C (courtesy of Fakespace Labs.)
28
21 LCD display, Resolution SXGA (1600x1200) Weig
ht Counter- Balanced No dead space but High
latencies due to Third-party tracker
Virtual Window 3-D Display (courtesy of Virtual
Research Co.)
29

Output Devices

• Auto-stereoscopic displays
• do not require use of special glasses
• Passive auto-stereoscopic displays do not track
• users head and thus restrict users position
• Active auto-stereoscopic displays track the head
  
• motion and give more freedom of motion..

30
18.1 LCD display, Resolution 1280x1024
(mono) 640 x 1024 (stereo) Weight 11.25 kg
Auto-stereoscopic 3-D Display(courtesy of
Dimension Technologies Co.)
31
40 LCD display, Resolution 1280x768 pixels 70o
horizontal viewing (7 to 15 feet) Weight 33.2 kg
SynthaGram 404 (courtesy of StereoGraphics Co. -
12,000)
32
20 LCD display, Resolution 1600x1200
(mono) 100o horizontal viewing (1.5 to 6
feet) Weight 8.4 kg
SynthaGram 204 (courtesy of StereoGraphics Co. -
3,000)
33
18 LCD display, Resolution 1280x1024
(mono) 640 x 1024 (stereo) Weight 17 kg
Active auto-stereoscopic 3-D Display (courtesy of
Dresden 3D Co.)
34
(No Transcript)
35

• Sharp autostereoscopic laptop
• Pentium 4, 15 diagonal display, 1024x768
  resolution, 2D and 3 D mode, uses parallax
  barrier.

http//www.inition.co.uk/inition/product_stereovis
_sharp_actius_rd3d.php
36
Output Devices

• Large Volume Displays
• Allow several co-located users to view a
  monoscopic or stereoscopic view of the virtual
  world
• Can be classified as monitor-based large
• volume displays or projector-based large
• volume displays.
• Allow more freedom of motion vs. personal
  displays.

37

Output Devices

• Monitor-based Large Volume Displays
• Use active or passive glasses
• Several users can look at a monitor
• Can have a single monitor, or multiple
  side-by-side monitors
• If side-by-side, image continuity becomes an
  issue.

38
Untracked and wireless
Tracked and wireless
Active glasses
39
Output Devices
Active glasses vs. FMDs (Face Mounted Display)
as if you were watching a 52" screen from 2m
distance.

• Some advantages
• no cables if head position is not tracked
• light and ergonomic (can be used over vision
  glasses)
• work well with large volume displays.
• allows full screen resolution 1280x1024
• Some disadvantages
• lose 2/3 of image light intensity through LCD
  filtering
• require special CRT stereo ready that has
  twice
• the hardware refresh rate (Hz) 120 Hz or
  more
• require direct line of sight for IR controller

40
Active glasses
Wired to the synchronizing jack of the graphics
card
wireless
41
Passive glasses vs. active glasses
42
Passive glasses vs. active glasses
43
Passive glasses vs. active glasses
44
Passive glasses vs. active glasses
45
Active glasses system
46

Tiled monitors-based display
VC 3.1 on book CD
Resolution is 3840 x 1024 and dimensions are 1,11
x 0.29 m2
47

Non-synchronized tiled image
discontinuity
Synchronized tiled image
48

Output Devices

• Projector-based Large-Volume Displays
• Old technology is CRT-based
• (analog) three projector tubes (R, G, B)
• Requires special fast green coating to avoid
  the
• fogging due to fast switching (at 120 Hz)
• Suffer from low luminosity problems
• (200-300 lumens)

49
Output Devices

• Projector-based Large-Volume Displays
• Technology makes transition from CRT-based
• (analog) to Digital Micro-mirror Device (DMD)
• (digital) projectors
• Workbench-type displays (Fakespace Responsive
• Workbench, Barco Baron, V-desk, etc.)
• Cave-type display (CAVE, RAVE)
• Wall-type displays
• Domes

50
Tilted surface
Viewing Cone
Reflector mirror Floor CRT projector (not shown)
The old Fakespace ImmersaDesk workbench
51
Baron workbench (courtesy of BARCO Co.)
IR Controllers
CRT Projector
Mirrors
Tilting mechanism
52

Baron
Workbench-type display geometries
V-desk
53
CRT Projector
Screen
Mirror
CAVE 3-D large volume display (courtesy of
Fakespace Co.)
54
CAVE 3-D large volume display (courtesy of
Fakespace Co.)
55
(No Transcript)
56
Output Devices
Wall-type displays

• Accommodate more users
• Using a single projector on a large wall means
  small image resolution
• Thus tiled displays place smaller images
  side-by-side so they need multiple projectors
• Images need to have overlap, to assure
  continuity
• However overlap from two projectors means
  intensity discontinuity (brighter images in the
  overlap areas)
• Projectors need to modulate intensities to dim
  their light for overlap pixels.

57
Pano-Wall display
Three projectors Approx. 7 x 2 m2
58
PanoWall display
59
VizTek Portable Wall

• 1 active DLP projector (3500 lumens)
• 1 portable screen 20 ft x 15 ft
• 1 portable workstation
• 25,000 includes 3 active stereo glasses

60
Output Devices
61
Tiled composite image from four projectors
62
Tiled composite image from four projectors after
adjustment
63
Wall and Dome-type displays

• Advantages
• Accommodate more users (tens to hundreds)
• Give users more freedom of motion
• Disadvantages
• Large cost (up to millions of dollars)
• Even with multiple projectors, resolution is
  much lower than for CRTs (because the area is
  large).
• Example PanoWall has 200,000 pixels/m2 while
  a monitor has 18,200,000 pixels/m2
• To have equal numbers of pixels/unit are more
  projectors (military)

64
(No Transcript)
65

Output Devices
3-D Audio Displays

• Definition
• Sound displays are computer interfaces that
  provide synthetic sound feedback to the user
  interacting with the virtual world. The sound can
  be monoaural (both ears hear the same sound) or
  binaural (each ear hears a different sound).

66

Output Devices

• 3-D Audio Displays
• Human hearing model
• HRTF-based 3-D sound
• The common belief is that the shape of the head,
  torso, and especially outer ear (pinna) effect
  the spectral contents of a sound as it propagates
  from free space to the inner ear. Additionally,
  because of the irregular shape of the upper body
  and pinna it is theorized that these filtering
  effects would be spatially dependent, varying
  with the location of the sound source. This
  filtering effect is commonly referred to as a
  "Head-Related Transfer Function" (HRTF). Because
  the HRTF would vary with elevation and azimuth,
  this would create a unique HRTF for each sound
  source location.

67

Output Devices

• 3-D Audio Displays
• Convolvotron
• The world's first multi-source, real-time,
  digital localization system, built by Crystal
  River Engineering for NASA in 1987. Can produces
  four independent virtual sound sources. The
  acoustic reflection provides transmission-loss
  modeling, several reflective surface materials,
  up to six programmable reflection paths.

68

Output Devices

• Human Hearing Model
• Polar coordinate system azimuth, elevation,
  distance (range)
• azimuth cues
• Azimuth is measured as an angle between a
  projection of the vector onto the horizontal
  plane and a second vector extending in front of
  the listener
• elevation cues
• Elevation is measured as the angle between a
  third vector pointing forward in the horizontal
  plane (0º) and the height of the sound source

69

Output Devices

• Human Hearing Model
• azimuth cues
• Azimuth is measured as an angle between a
  projection of the vector onto the horizontal
  plane and a second vector extending in front of
  the listener
• elevation cues
• Elevation is measured as the angle between a
  third vector pointing forward in the horizontal
  plane (0º) and the height of the sound source

70

Output Devices
3-D Audio Displays
71

Output Devices
Haptic Interfaces

• Haptics
• Comes from Greek Hapthai meaning the
• sense of touch
• Groups touch feedback and force feedback

72

Output Devices

• Touch Feedback
• Relies on sensors in and close to the skin
• Conveys information on contact surface
• geometry, roughness, slippage, temperature
• Does not actively resist user contact motion
• Easier to implement than force feedback.

73

Output Devices

• Force Feedback
• Relies on sensors on muscle tendons and
• bones.
• Conveys information on contact surface
• compliance, object weight, inertia
• Actively resist user contact motion
• More difficult to implement than touch feedback
• (no commercial products until mid 90s).

74

Haptic Interfaces

• Human touch sensing mechanism
• Most touch sensors are on the hand (much less
• density on other parts of the body)

75

Haptic Interfaces

• Sensorial adaptation
• Measure the decrease in electrical signals from
• The skin sensor over time, for a constant
  stimulus
• If the sensor produces a constant electrical
• Discharge for a constant mechanical stimulus
• Called Slow Adapting (SA)
• If the rate of electrical discharge drops
  rapidly
• Over time for a constant stimulus
• Rapidly Adapting (RA)

76

Haptic Interfaces
Human grasping configurations
77

Haptic Interfaces

• Maximum and sustained force exertion
• Maximum force exerted during power grasp
• Averages 400 N (male) and 225 N (female)
• Looking at body location, force output
• Grows from 50 N at PIP finger joint, to 100 N
• at shoulder
• Sustained force feedback is much smaller than
• maximum, owing to fatigue and pain

78

Haptic Interfaces

• Haptic feedback actuators
• Need to maximize power/weight ratio
• Need to have high power/volume ratio
• Need to have high dynamic range (fidelity)
• Need to be safe for the user
• - None of the current actuator technology
• satisfies all these requirements

79

Output Devices

• Touch Feedback Interfaces
• Can be desk-top or wearable (gloves)
• touch feedback mouse
• CyberTouch glove
• Temperature feedback actuators

80

Haptic Interfaces
The iFeel Mouse (0-125 Hz).
81
Haptic Interfaces

6 individually Controlled Vibrotactile actuators
0-125 Hz frequency 1.2 N amplitude at 125 Hz
CyberTouch Glove (Virtex)
82

Output Devices

• Temperature feedback
• Added simulation realism by simulating
• surface thermal feel
• No moving parts
• Uses thermoelectric pumps made of solid-state
• materials sandwiched between heat source and
• heat sink
• Single pump can produce 65C differentials

83

Temperature feedback actuator
84

Output Devices

• Force Feedback Interfaces
• Need mechanical grounding to resist user
• motion
• Can be grounded on desk, wall, or on user body
• More difficult to construct and more expensive

85

Haptic Interfaces
86
Uses potentiometers to sense position in
spherical coordinates Uses electrical actuators
to apply resistive torques
Logitech Force feedback joystick
87
The PHANToM used for 3D sculpting (courtesy of
SensAble Technology Co.)
88

89
PHANToM Omni
90
PHANToM Comparison
91

PHANToM 1.6/3.0
92
Omega Haptic Device 3 DOF Force Feedback
Delta Haptic Device 3 DOF Force Feedback
Delta Haptic Device 6 DOF Force Feedback

• 3 DOF
• End-effector replaceable
• Resolution 0.009 mm
• Max continuous force 12 N
• Stiffness 14.5 N/mm
• Connectivity USB 2.0/PCI
• full gravity compensation
• Real-time safety (velocity monitoring and
  electromagn. brakes)

• 3 DOF (36 cm diam ? 30 cm)
• Resolution .03 mm
• Max continuous force 20 N
• Stiffness 14.5 N/mm
• Connectivity PCI
• Real-time safety (velocity monitoring and
  electromagnetic brakes)

• 6 DOF (36 cm diam ? 30 cm)
• ?20o /axis
• Resolution .03 mm, 0.04o
• Max continuous force 20 N
• torques continuous 0.2 Nm
• Stiffness 14.5 N/mm
• Connectivity PCI
• Real-time safety (velocity monitoring and
  electromagnetic brakes)

93
NOVINT FALCON 3 DOF Force Feedback

• 3 DOF (right-left, forward/backward, up/down) 75
  ?75 ? 75 mm
• rumble, vibrations
• 3D exploration and textures
• Dynamic effects (inertia, weight, momentum)
• Cost less than 100
• Resolution gt0.06 mm
• Max continuous force 10 N
• Stiffness 5 N/mm
• Connectivity USB 2.0
• 1 kHz control bandwidth

94
Exoskeleton over CyberGlove)
Cables and pulleys
16 N/finger (continuous?) Weight 539 grams
remote electrical actuators in a control box.
The CyberGrasp force feedback glove
95
The CyberGrasp force feedback glove
VC 3.4 on book CD
96
CyberGrasp glove
Electronic interface box
Tether
Wrist Tracker
The CyberPack (courtesy of Virtex Co.)
97

6 DOF mechanical arm Wrist position and Force
feedback No need for a tracker
Allows simulation of weight and inertia, not
possible with glove-only interfaces
CyberForce interface (introduced recently)
98

Haptic Interfaces
VC 3.5 on book CD
CyberForce interface
99
(No Transcript)