Computer Vision

1
Chapter 9.3.2, 10.5.2, 13.11
Feature Extraction and Motion Estimation
2
Motion

• Rather than using two cameras, we can extract
  information about the environment by moving a
  single camera.
• Main problems are similar to stereo
• Correspondence
• Reconstruction
• Sometimes a new problem is also present
• Segmentation Which image regions correspond to
  rigidly moving objects.

3
Structure from Motion
Textbook treats motion largely from the
perspective of small camera motions. We will not
be so limited!
Reprinted with permission from Affine Structure
from Motion, by J.J. (Koenderink and A.J.Van
Doorn, Journal of the Optical Society of America
A, 8377-385 (1990). ? 1990 Optical Society of
America.

• Given m pictures of n points, can we recover
• the three-dimensional configuration of these
  points?
• the camera configurations?

(structure) (motion)
Computer Vision - A Modern Approach Set Feature
Matching and Stereo Slide by J. Ponce
4
Structure From Motion

• Several questions must be answered
• What image points should be matched?
• feature selection
• What are the correct matches between the images?
• feature tracking (unlike stereo, no epipolar
  constraint)
• Given the matches, what is camera motion?
• Given the matches, where are the points?
• Simplifying assumption scene is static.
• objects dont move relative to each other

5
Feature Selection

• We could track all image pixels, but this
  requires excessive computation.
• We want to select features that are easy to find
  in other images.
• Edges are easy to find in one direction, but not
  the other aperture problem!
• Corner points (with gradients in multiple
  directions) can be precisely located.

6
Feature Selection

• Basic idea for corner detection
• Find image patches with gradients in multiple
  directions.

Input
Corners selected
7
Feature Tracking

• Determining the corresponding features is similar
  to stereo vision.
• Problem no epipolar line
• Matching point could be anywhere in the image.
• If small motion between images, can search only
  in small neighborhood.
• Otherwise, large search space necessary.
• Coarse-to-fine search used to reduce computation
  time.

8
Feature Matching

• Example
• The set of vectors from each image location to
  the corresponding location in the subsequent
  image is called a motion field.

9
Feature Matching

• Example
• If the camera motion is purely translation, the
  motion vectors all converge at the
  focus-of-expansion.

10
Ambiguity

• The relative position between the cameras has six
  degrees of freedom (six parameters)
• Translation in x, y, z
• Rotation about x, y, z
• Problem images looks exactly the same if
  everything is scaled by a constant factor. For
  example
• Cameras twice as far apart
• Scene twice as big and twice as far away
• Can only recover 5 parameters.
• Scale cant be determined, unless known in advance

11
Solving for Structure and Motion

• Total number of unknown values
• 5 camera motion parameters
• n point depths (where n is the number of points
  matched)
• Total number of equations
• 2n (each point match has a constraint on the row
  and column)
• Can (in principle) solve for unknowns if 2n 5
  n (n 5)
• Usually, many more matches than necessary are
  used.
• Improves performance with respect to noise

12
Solving for Structure and Motion
Once the motion is known, dense matching is
possible using the epipolar constraint.
13
Multiple Images

• If there are more than two images, similar ideas
  apply
• Perform matching between all images
• Use constraints given by matches to estimate
  structure and motion
• For m images and n points, we have
• 6(m-1)-1n unknowns 6m-7n
• 2(m-1)n constraints 2mn-2n
• Can (in principle) solve when n is at least
  (6m-7)/(2m-3).

14
Stereo Ego-motion

• One application of structure from motion is to
  determine the path of a robot by examining the
  images that it takes.
• The use of stereo provides several advantages
• The scale is known, since we can compute scene
  depths
• There is more information for matching points
  (depth)

15
Stereo Ego-motion

• Stereo ego-motion loop
• Feature selection in first stereo pair.
• Stereo matching in first stereo pair.
• Feature tracking into second stereo pair.
• Stereo matching in second stereo pair.
• Motion estimation using 3D feature positions.
• Repeat with new images until done.

16
Ego-motion steps
Features selected
Features matched in right image
Features tracked in left image
Features tracked in right image
17
Stereo Ego-motion
Odometry track Actual track (GPS) Estimated track
Urbie
18
Advanced Feature Matching
Right image
Left image
Left image after affine optimization