Structure from Motion

1
Structure from Motion

• A moving camera/computer computes the 3D
  structure of the scene and its own motion

2
Sensing 3D scene structure via a moving camera
We now have two views over time/space compared to
stereo which has multiple views at the same time.
3
Assumptions for now

• The scene is rigid.
• The scene may move or the camera may move giving
  a sequence of 2 or more 2D images
• Corresponding 2D image points (Pi, Pj) are
  available across the images

4
What can be computed

• The 3D coordinates of the scene points
• The motion of the camera

Camera sees many frames of 2D points
Rigid scene with many 3D interest points
From Jabara, Azarbayejani, Pentland
5
From 2D point correspondences, compute 3D points
and TR
6
applications

• We can compute a 3D model of a landmark from a
  video
• We can compute the trajectory of the sensor
  relative to the 3D object points

7
Use only 2D correspondences, SfM can compute 3D
jig pts
up to one scale factor.
8
http//www1.cs.columbia.edu/jebara/htmlpapers/SFM
/sfm.html Jabara, Azarbayejani, Pentland

1. Two video frames with corresponding 2D interest
   points. 3D points can be computed from SfM
   method.
2. Some edges detected from 2D gradients.
3. Texture mapping from 2D frames onto 3D polyhedral
   model.
4. 3D model can be viewed arbitrarily!

9
Virtual museums

• Much work, and software, from about 10 years ago.
• 3D models, including shape and texture can be
  made of famous places (Notre Dame, Taj Mahal,
  Titanic, etc.) and made available to those who
  cannot travel to see the real landmark.
• Theoretically, only quality video is required.
• Usually, some handwork is needed.

10
SfM methods

• Typically require careful mathematics
• EX from 5 matched points, get 10 equations to
  estimate 10 unknowns also a more popular 8 pt
  linear method
• Methods must consider effects of noise
• See Faugeras et al and
• Methods can run in real time

11
Special mathematics

• Epipolar geometry
• Fundamental matrix computed from a pair of
  cameras
• Essential matrix specialization of fundamental
  matrix when calibration is available

12
Finishing the course 2 options

• HW7 on stereo computation (Sat 6 Dec) (60/7)
• Final Exam (25) Monday 8 Dec 1245-245

• SfM report (6 Dec) and demonstration. Find
  existing code and get it to work on CSE 803
  images
• (60/7 10)
• Final Exam (15) limited problems