Reynold Bailey

1
Visual Perception and Computer Graphics
Reynold Bailey
2
The Human Eye
http//webvision.med.utah.edu/
3
The Human Eye
4
Visible Light
5
Retina
Light
6
Photopigment Sensitivity
7
Photoreceptor Density
8
Opponent process theory
9
Opponent process theory
10
Beyond the Retina
11
Beyond the Retina

• Visual cortex and beyond
• Edges
• Motion
• Specific color processing
• Shape
• Orientation
• Depth
• Stereo information processing
• Understanding visual information

12
Luminance Range in Natural Environments
How does our visual system adapt to such a wide
range ?
13
Visual Adaptation
14
Visual Adaptation

• How does our visual system adapt to such a wide
  range ?
• Pupil changes size 7mm down to 2mm
• Results in about 1 log unit change
• Rods and Cones
• Rods operate best at scotopic levels
• Cones operate best at photopic levels
• Both rods and cones operate in mesopic levels
• Bleaching and regeneration of photopigments
• Neural processes adjust the base and gain of
  system

15
Visual Adaptation
Threshold studies
16
Visual Adaptation
Sensitivity of rods and cones
scotopic
mesopic
photopic
17
Visual Adaptation
Sensitivity of rods and cones
18
Visual Adaptation
Visual Acuity measure of visual systems
ability to resolve spatial detail.
Snellen Acuity Chart
19
Visual Adaptation
Visual Acuity
20
Visual Adaptation
Time Course for Light Adaptation - Rods
21
Visual Adaptation
Time Course for Light Adaptation - Cones
22
Visual Adaptation
Time Course for Dark Adaptation
23
Visual Adaptation
All this info is used to create a mathematical
model of adaptation.
A Model of Visual Adaptation for Realistic Image
Synthesis James A. Ferwerda Sumanta N. Pattanaik
Peter Shirley Donald P. Greenberg Program of
Computer Graphics, Cornell University
24
Visual function across range of light
Time course for light and dark adaptation
25
Interesting Effects
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
Interesting Effects
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
(No Transcript)
34
Interesting Effects
35
Perceptually Adaptive Graphics
EUROGRAPHICS/SIGGRAPH Campfire May 2001
Researchers from various fields
Computer Graphics and visualization
Psychology
Neuroscience
Psychophysics
Medicine
This paper outlines the state of the art as
discussed at that event and progress made since.
36
Perceptually Adaptive Graphics
Perceptually Adaptive Graphics
For photorealistic images, how do we know that we
are not simply producing pretty pictures and
actually representing reality in a faithful
manner?
For real time rendering and simulation, how do we
make speed-accuracy trade-offs while minimizing
the perceptibility of any resulting anomalies?
What types of anomalies are most noticeable?
How can we quantify these factors and use them in
a methodical way to adapt our graphics to the
perception of the viewer?
37
Outline
Interactive Graphics
Image Fidelity
Animation
Virtual Environments
Visualization
Non-Photorealistic Rendering
38
Interactive Graphics
Ideal render fully detailed and photorealistic
scenes in real time.
Not yet a feasible option.
Aim produce the best perceived image in the time
available.
Gaze-contingent approaches.
Perceptually guided polygonal simplification.
Interruptible rendering.
39
Interactive Graphics Gaze-contingent approaches
Basic idea
Degrade the resolution in the peripheral image
regions.
The high resolution area moves with the users
focus, so area under scrutiny is always rendered
in high resolution.
40
Interactive Graphics Gaze-contingent approaches
Gaze-contingent screens Focus Plus Context
Screens are one result of new research.
Usually display hardware has the same resolution
for all areas of the screen even though the
peripheral content is rendered at low resolution.
In FC screens, there is a difference in
resolution between the focus and the context area.
Wall-sized low-resolution display with an
embedded high-resolution screen.
When user moves the mouse, the display content
pans and can be brought into high-resolution
focus as required.
41
Interactive Graphics Gaze-contingent approaches
Focus Plus Context Screens
42
Interactive Graphics Gaze-contingent approaches
Focus Plus Context Screens
43
Interactive Graphics Gaze-contingent approaches
Focus Plus Context Screens
44
Interactive Graphics Gaze-contingent approaches
Focus Plus Context Screens
45
Interactive Graphics Gaze-contingent approaches
Focus Plus Context Screens
46
Interactive Graphics Gaze-contingent approaches
Attentive 3D-rendering engines have developed.
Uses the viewers gaze position to vary the LOD
at which an object is drawn.
Similar to gaze-contingent displays but with one
main difference
Objects are simplified at the object geometry
level instead of the image level.
47
Interactive Graphics Gaze-contingent approaches
Display systems that guide the viewers
attention Easily Perceived Displays
Users attention is directed as opposed to
followed.
Use the gaze information from one user to decide
which parts of an image are important and which
parts should be removed.
Subsequent viewers are guided to what the
original viewer found important.
48
Interactive Graphics Gaze-contingent approaches
New research suggests that visual attention is
largely controlled by the task being performed.
Identify task related objects in advance.
Render other objects at lower resolution.
49
Interactive Graphics Perceptually guided polygon
simplification
Reducing model complexity based on perceptual
criteria.
The circumstances under which simplification will
be perceptible are determined.
Those deemed perceptible are not carried out.
Some techniques allow the user to identify
important features of the models either by
eye-tracking or use of the mouse.
How can we tell if one simplification is actually
better than another?
Several techniques that experimentally and
automatically measure and predict the visual
fidelity of the simplified models have emerged.
50
Interactive Graphics Interruptible Rendering
Trade off between fidelity and performance.
Basic idea Progressive rendering framework.
A coarse image is drawn on the back buffer and is
continuously refined.
An error function is defined that combines the
spatial error due to the coarseness of the image
and the temporal error due to the time delay.
At each step the error is calculated and the
image is rendered when the error threshold is met.
51
Image Fidelity Metrics
Develop perceptual metrics and heuristics to
measure or predict the fidelity of images.
At the Campfire, one researcher described three
standards of realism
Physical realism the image provides the same
visual stimulation as the scene depicted.
Photorealism the image produces the same visual
response as the scene depicted.
Functional realism the image provides the same
visual information as the scene depicted.
Metrics that evaluate visual appeal, structural
distortion of scene objects, and animation
quality have been proposed.
52
Image Fidelity Rendering
In 2004, a technique for high quality global
illumination rendering using perceptual
illumination components was presented
The illumination of a surface can be split into
components that are separately computable
Direct illumination.
Indirect glossy illumination.
Indirect diffuse illumination.
Indirect specular illumination.
Goal was to produce a perceptual metric
functioning on these terms and use it to drive
the rendering process.
Perceptual experiments were conducted and a
mathematical model was fitted to the experimental
data to formulate the metric.
53
Image Fidelity High Dynamic Range image
reproduction
The dynamic range of a scene is the contrast
ratio between its brightest and darkest parts.
A high dynamic range image is one that has a
greater dynamic range than can be shown on
standard CRTs or LCDs.
High dynamic range display devices have developed.
54
Animation Physical simulation
Evaluate and/or improve animations by applying
perceptual principles.
In real-time animation, fully accurate processing
of dynamic events such as collisions can lead to
long delays.
The role of various factors on human perception
of anomalous collisions has been investigated.
Psychophysical experiments were conducted to
determine the thresholds for human sensitivity to
dynamic anomalies.
55
Animation Human Motion
Plausible human motion in animation is difficult
to achieve.
Motion capture data is used extensively.
Techniques have been developed to generate
transitions between two sets of motion capture
data and also to combine two sets of motion
capture data.
Several perceptual metrics for character
animation have also been developed.
56
Virtual Environments
At the campfire, the problem of effectively
generating images of objects and environments
that convey an accurate sense of distance and
size was discussed.
The is interaction between visual perception and
locomotion.
Displays that combine computer-generated visual
information with biomechanical information on
locomotion.
Does image quality affect the perception of
distance in VE?
The image quality played no role in the
under-estimation of distances compared with the
real world scenario (2003).
Distance perception is affected by cues such as
shadows and reflections (2002).
57
Virtual Environments
Research was also done to determine the impact of
field of view and binocular viewing restrictions
on peoples ability to perceive distance in the
real world.
The under-estimation of distances is not due to
not being able to see ones own body (same
conclusion reached in a separate study).
Having a restricted field of view does not impact
distance perception as long a head movement is
allowed.
Monocular viewing did not produce poorer
performance than binocular viewing.
These restrictions do not explain the poor
performance in distance estimation for tasks in
immersive environments displayed using a HMD.
58
Virtual Environments
Research on the simulation fidelity of immersive
virtual environments
The use of memory tasks as a measurement if the
fidelity of VE was proposed.
Memory recall and memory awareness states.
Measurement of presence (the feeling of being
there)
Previously done using questionnaires and
interviews.
Technique based on physiological measurements is
proposed.
Preliminary results are promising.
59
Visualization
Designing and implementing algorithms for
displaying large, complicated data sets.
Goal Facilitate the rapid, intuitive
appreciation of the essential features.
Perceiving shape from texture.
The texture used impacts the shape perceived.
Especially true for unfamiliar structures.
Several different textures were synthesized and
applied to various surfaces.
Users were asked to orient probes so that matched
the surface normal as closely as possible.
60
Visualization
61
Non-Photorealistic Rendering
NPR techniques are well suited to achieving
functional realism.
Studies have been done to evaluate how space is
perceived in NPR immersive environments.
Participants wore HMD and were asked to walk
towards a target.
In NPR environment, perceived distance were 66
of intended distances.
62
Non-Photorealistic Rendering
Functional realism in the context of facial
illustrations.
New technique to automatically generate
caricatures from photographs of people.
63
Non-Photorealistic Rendering
Manipulating rendering methods in order to evoke
certain responses
Strength.
Weakness.
Danger.
Safety.
Goal related judgement.
Rendering style can convey meaning and influence
judgement in a controllable fashion.
64
Human Visual Perception Seminar
http//www.cs.wustl.edu/rjb1/perceptionseminar.ht
m