Computer Vision

1
Computer Vision

• Stereo Vision

2
Pinhole Camera
3
Perspective Projection
4
Stereo Vision

• Two cameras.
• Known camera positions.
• Recover depth.

scene point
p
p
image plane
optical center
5
Correspondences
p
p
6
Matrix form of cross product
aaxiayjazk
ababsin(?)u
bbxibyjbzk
7
The Essential Matrix
Essential matrix
8
Stereo Constraints
M
Image plane
Y1
p
O1
Z1
X1
Focal plane
9
A Simple Stereo System
LEFT CAMERA
RIGHT CAMERA
baseline
Right image target
Left image reference
Zw0
10
Stereo View
Right View
Left View
Disparity
11
Stereo Disparity

• The separation between two matching objects is
  called the stereo disparity.

12
Parallel Cameras
P
Z
xl
xr
pl
f
pr
Ol
Or
Disparity
T
T is the stereo baseline
13
Finding Correspondences
14
Correlation Approach
LEFT IMAGE

• For Each point (xl, yl) in the left image, define
  a window centered at the point

15
Correlation Approach
RIGHT IMAGE
(xl, yl)

• search its corresponding point within a search
  region in the right image

16
Correlation Approach
RIGHT IMAGE
(xl, yl)
dx
(xr, yr)

• the disparity (dx, dy) is the displacement when
  the correlation is maximum

17
Comparing Windows
18
Comparing Windows
Minimize
Sum of Squared Differences
Maximize
Cross correlation
19
Correspondence Difficulties

• Why is the correspondence problem difficult?
• Some points in each image will have no
  corresponding points in the other image.
• (1) the cameras might have different fields of
  view.
• (2) due to occlusion.
• A stereo system must be able to determine the
  image parts that should not be matched.

20
Structured Light

• Structured lighting
• Feature-based methods are not applicable when the
  objects have smooth surfaces (i.e., sparse
  disparity maps make surface reconstruction
  difficult).
• Patterns of light are projected onto the surface
  of objects, creating interesting points even in
  regions which would be otherwise smooth.

• Finding and matching such points is simplified by
  knowing the geometry of the projected patterns.

21
Stereo results

• Data from University of Tsukuba

Ground truth
Scene
(Seitz)
22
Results with window correlation
Estimated depth of field (a fixed-size window)
Ground truth
(Seitz)
23
Results with better method
A state of the art method Boykov et al., Fast
Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision,
September 1999.
Ground truth
(Seitz)