MIT Computer Graphics Group Laboratory for Computer Science

1
MIT Computer Graphics GroupLaboratory for
Computer Science

• Professors Julie Dorsey, Leonard McMillan, Seth
  Teller
• graphics.lcs.mit.edu

2
The Capture Problem

• City Scanning
• Modeling Foliage

How can we import 3D scene data quickly and
automatically? Starting point for
visualization, design, simulation, teaching.
3
From 2D Images to 3D Site Models
3) Who establish camera control

• How is it done today?

4) Indicate/verify scene structure
5) Control, structure then optimized generalized
triangulation ensues.
2) Processed by human image analysts
Note human(s) in the loop !
1) Images acquired
Implications for scaling, throughput
4
Fundamental limitations of semi-automated
approaches

• Every image is handled by a human
• Semi-automatic algorithms assume
• Small number (tens) of input images
• Images taken from outside the scene
• All pairs of images are correlated O(n2)
• Implications
• Quadratic processing scaling limitation
• Human in loop throughput limitation
• System does not improve w/ technology!

5
Fully Automated Site Modeling

• How it can be done in the future

3) Geometry, reflectance estimated
combinatorially, and with robust statistics
1) Many geo-located images acquired
2) Images are spatially indexed
6
Implications of Novel Approach

• Computation to process a spatial region depends
  only on acquisition density and the size of the
  region (but not on the total number of images)
• System throughput increases as storage,
  processing technology advances
• Quantity/complexity of reconstructed scenes will
  grow steadily with time
• Fusion of spatially distributed models eased
• Quality may be less than that possible with a
  human image analyst

7
Novel problem domain

• Urban exteriors (built structure)
• Tens of thousands of digital still images
• Acquisition near-ground, inside scene
• Absolute, a priori camera pose estimates
• No human in the loop (break scale barrier)

8
Challenges (Research/Engineering/Systems)

• Robust instrumentation for absolute geolocation
• Sparse/dense correspondence algorithms
• Incremental/multiresolution reconstruction
• Scaleable in of images, output features
• Estimation of surface reflectance (texture)
• System assessment (speed, error, cost)

9
Geo-located digital camera
Cheap digital cameras, GPS, MEMS inertial
chipsets soon available device will someday be
hand-held
10
Image acquisition First dataset
Early prototype of pose camera deployed in and
around Tech Square (4 structures) Collected 81
nodes, 4,000 geo-located images
11
Mosaic generation
Each node is 50-70 images tiling a sphere All
nodes correlated to form spherical mosaics Camera
internal parameters auto-calibrated
Computation is automated (no human in loop) Per
node, about 20 CPU-minutes _at_ 200 MHz
12
Mosaics A Closer Look
Each is about 75 Mega-Pixels with
improved cameras, each will be about 300
Mega-pixels
13
Imagery Control
Each node is controlled, or co-situated, in a
common, global (Earth) coordinate system
Current instrumentation requires human assist-
ance (1 hr total, or about 1 second per
image) Mosaicing is significant engineering
advantage Goal full automation of geo-location
instruments
14
Building detection
Sweep-plane technique identifies vertical
façade orientations, locations, boundaries
15
Texture estimation challenge
16
Reflectance (Texture) estimation
Robust, weighted median - statistics algorithm
estimates texture/BRDF for each building facade
weighted median
Algorithm removes structural occlusion foliage
blur (obliquity) color and lighting variations!
17
Texture estimation results
Input Raw imagery
Output Synthetic texture

• Made possible by many observations
• A sensor and system that effectively see
  through complex foliage!

18
Preliminary results
Model represents about 1 CPU-Day at 200 MHz
Next acquire MIT campus compare to reference
model captured via traditional surveying
19
From the East
From the South
20
Contributions
Instrumentation, scaleable end-to-end system
design Address scaling with geo-location, spatial
data structures Novel mosaicing, reconstruction,
texturing algorithms Significant step toward
fully automated reconstruction
Goal for 1998 capture MIT Campus (200
structures) from 1 Tb of ground, 1 Tb of
aerial imagery
21
Further information

• graphics.lcs.mit.edu
• graphics.lcs.mit.edu/city/city.html
• graphics.lcs.mit.edu/publications.html \
• LCS Technical Memo 561 (w/ Coorg)
• AI Technical Memo 1593 (w/ Mellor, L.-Perez)
• Proc. 1997 Image Understanding Workshop
• LCS Technical Memo 568 (w/ Coorg, Master)
• LCS Technical Report 729 (w/ Coorg)