Visual Perception in Humans and Machines

1
Visual Perception in Humans and Machines

• Kostas Daniilidis
• Assistant Professor
• GRASP Lab
• University of Pennsylvania

2
Examples

• How do we (humans) recognize faces ? Make a
  machine find President Clintons face in the web

3
(No Transcript)
4
An interdisciplinary definition

• Computer Vision is devoted to the discovery of
  algorithms, representations, and architectures
  that embody the principles of visual
  capabilities.
• What are visual capabilities?
• Recognizing objects and faces
• Estimating shapes and distances
• Moving, grasping

5
Relation to other fields

• Computer Vision is inspired from Biological
  Vision (Phenomenology and Models in Psychophysics
  and in Neurobiology) but does not try to imitate
  the nature's architecture or algorithms.
• Biological Vision and Psychophysics may find
  computational models discovered in Computer
  Vision useful for explaining nature.

6
Target problem in computer vision

• Compute properties of the 3D world from one or
  more digital images
• These properties may be
• dynamic (observer and object motion)
• geometric (distances, object shape)
• enabling recognition
• The result may be an action (grasp an object,
  avoid an obstacle)

7
What is an image ?

• A gray-value image is just a set of numbers
  (usually from 0 to 255)

8
An image is a set of numbers

• 175 189 190 188 199 197 196 193
• 181 189 191 194 198 196 191 179
• 189 191 197 198 200 195 173 129
• 192 194 198 200 194 161 116 116
• 198 200 200 190 152 113 116 119
• 201 202 185 135 105 103 114 119
• 205 180 121 89 104 101 109 114
• 177 105 88 90 100 103 101 105

9
An image is a surface I(x,y)

10
Basic image processing operations

• Smoothing and Noise Removal

11
Blur removal
12
Edge detection

• X derivative

• Gradient magnitude

• Y derivative

• After thresholding

13
(Sub) sampling

• Shannon Theorem Sampling frequency must be
  greater than the maximal frequency in the image
  (therefore smooth before subsample)

• first smooth

14
Brightness perception
15
How do we perceive distances?

• Perspective distortion in texture, contour,
  shading, and a-priori knowledge
• Stereopsis (what most people believe)
• Motion

16
The Eye as a Pinhole CameraPerspective
Projection
Z
X
u X/Z
17
Ames Illusion
18
Perspective Illusions

• A-priori-knowledge bias

19
Quiz

• From which points in space is a rectangle viewed
  as a square (more difficult an ellipse viewed as
  a circle ?

• Be careful The center of
• the ellipse in the image
• is not the projection of the
• center of the ellipse on the floor!

20
The power of vanishing points

• Perspective projection preserves
• cross-ratio AC/AD BC/BD is the same on the
  street and in the image. If A is a vanishing
  point AC/AD 1.

We measure A,B,C,D in pixels in the image and
form cross ratio for image and for the street.
BC is computed from the equality of the two
ratios.
A
B
C
D
21
Stereopsis
Infer depth from the disparity between the
positions of the same feature in left and right
image
22
Stereo Reconstruction
23
Stereo-disparity estimation

• Search at every pixel for candidates of maximum
  correlation between left and right
• Estimate 3D-coordinates of point

24
The power of visual motion

• Most of the animals have monocular vision (left
  and right visual fields do not overlap)
• 8 of the population can not see stereo
• Stereopsis is limited to a very short depth of
  field (10m).

25
(No Transcript)
26
Kinetic depth effect(moving dots)
27
Self- and object-motion

28
Motion artifacts
29
Structure from Motion

• Given a sequence of images
• find
• 1. Ego-motion
• 2. 3D-structure
• 3. Independent motions
• applying only the assumption of
• rigidity.

30
Motion Field and Heading Direction

31
Depth map

32
Temporal aliasing

• Wagon Wheel Illusion A wheel with a periodic
  radial pattern is perceived to move backwards
  depending on the relation between the speed, the
  radius of the wheel, and the period of the
  pattern (www.cstr.ed.ac.uk/rjc/wagonWheel)

33
Aperture problem

• Inside a small aperture displaying a small line
  we can estimate only the motion direction
  perpendicular to the line.

34
My wife and my mother-in-law

• The role of the
• focus of attention