Ismail Oner Sebe

1
Rapid Part-Based 3D Modeling

• Ismail Oner Sebe
• Suya You
• Ulrich Neumann

2
Motivation

• Rapidly model 3D objects one class at a time from
  single uncalibrated images
• Key points
• High Quality
• Rapid
• Single Image
• Uncalibrated

3
Outline

• System Overview
• Generic Model Generation
• Camera Calibration
• Texturing/Overlay
• User interface/inputs
• Part-based Modifications
• Results and Discussion

Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
4
Generic Model Generation

• Models are created by modifying a part-based
  generic model of that class.
• Part-based 3D Model can be created by
• Pre-existing 3D model
• Create/connect parts
• Progression of movement if parts connected
• Windows connected to doors
• Tires do not need to be

Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
5
Camera Calibration

• Uncalibrated camera
• No access to the camera needed
• Vanishing points approach
• 13 image clicks
• Internal camera parameters
• External camera parameters
• Scaling

Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
6
Texturing/Overlay

• Once the camera is calibrated
• 3D Model over image
• or
• 3D Model with image (texture)
• Symmetry of the model is used for texture copying
  
• Extracted color information fills the holes

Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
7
User Interface

• Interface
• Select a part
• Choose a direction
• Magnitude by mouse drag
• This user input is converted to a 3D movement by
  back-projecting the 2D mouse movement to the 3D
  vector.

Back-Projection
2D Input
3D Input
Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
8
Part-Based Modification

• Each part can be moved by one of the following
  modes
• MoveSame Part moves in the same direction as
  the input movement
• MoveOpposite Part moves to the mirror
  direction (mirror is alligned with the symmetry)
• DoNotMove Part is kept still
• Interpolate Scattered Data Interpolation (SDI)
  triangle-based SDI, thin-plate, closest-point,
  Shepards interpolator
• The order of modification MoveSame,
  MoveOpposite, DoNotMove, and Interpolate.

Camera Calibration
Y
Texture 2D/3D
Model Update
Save 3D model
Input 2D/3D
Done
Generic Model
Image
N
9
Results Ford Explorer 2D
10
Results - Ford Explorer 3D
3D model with estimated color
3D model with plain color
3D model with Texture
11
Results Toyota Matrix 2D
Input Image
12
Results Toyota Matrix 3D
3D model with plain color
3D model with Texture
3D model with estimated color
13
Discussion

• Pros
• Fast (1-2) minutes
• High-quality 3D models
• Single uncalibrated image
• Intuitive user interface
• Cons
• Create generic model for every class
• Such a model may not exist
• Need at least one photo

14
Conclusion and Future Work

• We present a rapid part-based 3D modeling
  technique from single uncalibrated image
• One class of objects, vehicles, are shown in this
  paper
• Current work
• Modeling of insects (Joint work with SFSU, Ilmi
  Yoon et.al.)
• Model-based tracking and free-viewpoint video
• Future work
• Automatization of the modeler
• Modeling of humans, animals, etc.

15
Thank you!
Thank you ! Questions?
16
Video
17
Results Mercury Cougar 3D
3D model with plain color
3D model with estimated color
3D model with Texture
18
Results Toyota Scion 3D
3D model with plain color
3D model with estimated color
3D model with Texture
19
Results Citroen 3D
3D model with plain color
3D model with estimated color
3D model with Texture
20
System Overview