Step-by-Step%20model%20building

1
Step-by-Step model building
Jana Kosecka Department of Computer Science
George Mason University http//www.cs.gmu.edu/kos
ecka
2
Review
Feature selection
Feature selection
Feature correspondence
Camera Calibration
Landing
Augmented Reality
Euclidean Reconstruction
Vision Based Control
Sparse Structure and camera motion
3
Review
Feature selection
Feature selection
Feature correspondence
Camera Calibration
Epipolar Rectification
Dense Correspondence
Texture mapping
Euclidean Reconstruction
Sparse Structure and motion
3-D Model
4
Review
Feature selection
Feature selection
Feature correspondence
Projective Reconstruction
Epipolar Rectification
Camera Self-Calibration
Dense Correspondence
Texture mapping
Euclidean Reconstruction
3-D Model
5
Examples
6
Feature Selection

• Compute Image Gradient
• Compute Feature Quality measure for each
  pixel
• Search for local maxima

Feature Quality Function
Local maxima of feature quality function
7
Feature Tracking

• Translational motion model
• Closed form solution
• Build an image pyramid
• Start from coarsest level
• Estimate the displacement at the coarsest level
• Iterate until finest level

8
Coarse to fine feature tracking
0
1
2

1. compute
2. warp the window in the second image by
3. update the displacement
4. go to finer level
5. At the finest level repeat for several
   iterations

9
Optical Flow

• Integrate around over image patch

• Solve

10
Affine feature tracking

Intensity offset
Contrast change
11
Tracked Features
12
Wide baseline matching
Point features detected by Harris Corner detector
13
Difficulty in motion estimation using
wide-baseline matching
14
Least square estimator cant tolerate any outlier

• Robust techniques is needed to solve the problem.

15
Robust estimators for dealing with outliers

• Use robust objective functions
• The M-estimator and Least Median of Squares
  (LMedS) Estimator
• Neither of them can tolerate more than 50
  outliers
• The RANSAC (RANdom SAmple Consensus) algorithm
• Proposed by Fischler and Bolles
• The most popular technique used in Computer
  Vision community
• It can tolerate more than 50 outliers

16
The RANSAC algorithm

• Generate M (a predetermined number) model
  hypotheses, each of them is computed using a
  minimal subset of points
• Evaluate each hypothesis
• Compute its residuals with respect to all data
  points.
• Points with residuals less than some threshold
  are classified as its inliers
• The hypothesis with the maximal number of inliers
  is chosen. Then re-estimate the model parameter
  using its identified inliers.

17
RANSAC Practice

• The theoretical number of samples needed to
  ensure 95 confidence that at least one outlier
  free sample could be obtained.

• It has been noticed that the theoretical
  estimates are wildly optimistic
• Usually the actual number of required samples is
  almost an magnitude more than the theoretical
  estimate.

18
The difficulty in applying RANSAC

• Drawbacks of the standard RANSAC algorithm
• Requires a large number of samples for data with
  many outliers (exactly the data that we are
  dealing with)
• Needs to know the outlier ratio to estimate the
  number of samples
• Requires a threshold for determining whether
  points are inliers
• Various improvements to standard approaches
  Torr99, Murray02, Nister04, Matas05,
  Sutter05 and many others
• Still rely on finding outlier-free samples.

19
Robust technique result
20
More correspondences and Robust matching

• Select set of putative correspondences
• Repeat
• 1. Select at random a set of 8 successful
  matches
• 2. Compute fundamental matrix
• 3. Determine the subset of inliers, compute
  distance to epipolar line
• 4. Count the number of points in the
  consensus set

21
RANSAC in action
Inliers
Outliers
22
Epipolar Geometry

• Epipolar geometry in two views
• Refined epipolar geometry using nonlinear
  estimation of F

23
Two view initialization

• Recover epipolar geometry (essential/fundamental
  matrix)
• Compute (Euclidean) projection matrices and 3-D
  struct.
• Compute (Projective) projection matrices and
  3-D struct.

calibrated
24
Multiple-view structure and motion recovery
Given images of points Knowing all the
motions, estimate the depth of a point using all
frames
Estimate motion between any two frames using the
points and their depths visible in those frames
25
Multi-view reconstruction

• Two view - initialized motion and structure
  estimates (scales)
• Multi-view factorization - recover the remaining
  camera
• positions and refine the 3-D structure by
  iteratively computing

1. Compute i-th motion given the known structure
iteration
26
Example of multi-view reconstruction
Euclidean reconstruction
27
Nonlinear Refinement

• Euclidean Bundle adjustment
• Initial estimates of are
  available
• Final refinement, nonlinear minimization with
  respect
• to all unknowns

28
Example - Euclidean multi-view reconstruction
29
Example
Original sequence
Tracked Features
30
Recovered model
31
Euclidean Reconstruction
32
Epipolar rectification

• Make the epipolar lines parallel
• Dense correspondences along image scanlines
• Computation of warping homographies

1. Map the epipole to infinity
Translate the image center to the origin
Rotate around z-axis for the epipole lie on the
x-axis such that Transform the
epipole from x-axis to infinity
2. Find a matching transformation
is compatible with the epipolar geometry
is chosen to minimize overall disparity
33
Epipolar rectification
Rectified Image Pair
34
Epipolar rectification
Rectified Image Pair
35
Dense Matching

• Establish dense correspondences along scan-lines
• Standard stereo configuration
• Constraints to guide the search
• 1. ordering constraint
• 2. disparity constraint limit on disparity
• 3. uniqueness constraint each point has a
  unique
• match in the second view

36
Dense Matching
37
Dense Reconstruction
38
Texture mapping, hole filling
39
Texture mapping