Automatic 3D modelling of Architecture

1
Automatic 3D modelling of Architecture

• Anthony Dick1 Phil Torr2 Roberto Cipolla1
• 1Department of Engineering 2Microsoft Research,
• University of Cambridge Cambridge

2
The goal

• Generate 3D models of architectural scenes from
  several images automatically
• Including accurate geometry, texture

Interactively built using Photobuilder!
Available at http//svr-www.eng.cam.ac.uk/photobu
ilder/download.html
3
Our approach

• Previous structure from motion algorithms use
  only image data
• We integrate image data with prior knowledge of
  architecture
• The scene will be piecewise planar
• Walls are likely to intersect at right angles
• Walls are likely to be perpendicular to a common
  ground plane
• Walls are likely contain doors and windows which
  have a highly constrained shape

4
Model representation

• Scene is modelled as a collection of wall
  planes
• Each wall plane has a plane equation and a
  boundary
• Each wall plane may contain offset layers such as
  doors, windows

c

• Each offset layer is one of a collection of
  parameterised shapes

b
(x,y)
d
a
r
a
Front view
Overhead view
5
Model estimation

• Structure estimation has 2 parts
• How many walls are in the scene and what are
  their parameters?
• How many offset layers does each wall contain,
  what shape are they, and what are their
  parameters? ECCV2000
• Model selection between different shapes

6
Previous work

• Manually defined homography
• Initialise offset layer estimates using dense
  correspondence
• Fit 4 different shape models to each region
• Use Bayesian model selection criterion to select
  best shape model

Initial
After model fitting selection
7
Whats new

• Extension to scenes with multiple wall planes
• Automatic segmentation of walls

8
Initialisation

• Feature-based structure from motion
• Track points
• Estimate pairwise epipolar geometry
• Camera self-calibration Mendonca CVPR99

9
Plane segmentation

• Recursive RANSAC plane extraction
• Assume all planes perpendicular to common ground
  plane
• Project onto ground plane
• Derive plane boundaries perpendicular and
  parallel to ground plane

Reconstruction projected onto ground plane
10
Optimising the planar model

• Gradient descent search
• Cost function SSE of model projected into each
  image
• Parameters to vary
• Ground plane orientation
• Boundary and intersection points of each plane

Before fitting
After fitting
11
Evaluating the cost function

• Search requires many evaluations of cost function
• This is expensive
• Greens Theorem
• Sum vector field A around region boundary
• Cache results for best efficiency

e1
e1
e2
e2
e3
e4
R1
R2
Cost of R2, L(R2) L(R1) L(e1) L(e2) L(e3)
L(e4)
12
Results

• Courtyard corner

13
The castle sequence

• Images from http//www.esat.kuleuven.ac.be/pollef
  ey/demos/castle.html

14
Future work

• Use of lines to initialise offset layers
• Join nearby lines into rectangles
• Use knowledge of window height/width ratios
• More extensive and structured set of shapes
• Rather than simply testing each possibility
• Possible use of architectural shape grammars
• And in conclusion
• General framework of combining prior knowledge
  and image data is a useful one
• Challenge is to formulate prior knowledge usefully

15
The Bayesian framework
model prior
evidence
Bayes Rule
constant
Evidence
prior
likelihood
Model parameters q Wall planes plane
boundary, plane equations Offset layers
Height, width, x, y position
16
Planar parallax

• Having optimised for main walls, want to fit
  doors, windows etc.
• This is the same problem tackled earlier, but
  initialisation is now more difficult
• Each plane covers less of the image
• There may be some fitting errors
• Manually set number of primitives on each plane
• Assumes evenly spaced, vertically centred
• Fits each model from this initialisation