Design Considerations for the City Scanning Project

1
Design Considerations for the City Scanning
Project

• Seth Teller
• MIT Computer Graphics Group
• graphics.lcs.mit.edu
• 6.033 Lecture, May 1999

2
Motivation Modeling process

• Representing object/scene in form suitablefor
  manipulation by a computer program
• But how do you get things into the computer ?

3
Idea scan urban scenes

• Much like photography
• Fix a sensor on object of interest
• Develop its observations computationally
• Produce an artifact a representation of scene
• Sensor a geo-referenced camera
• Computational development
• Register observations to single coordinate system
  
• Extract structural elements
• Infer appearance (color, etc.) for each element
• Aggregating local inferences into coherent whole
• Artifact
• Textured, geodetic CAD model
• Suitable for visualization, simulation, etc.

4
Design Goals

• Resolve structure, color to 5 centimeters
• Necessary for human-scale simulation
• Capable of acquiring extended urban areas
• 100s/1000s of structures over several km2
• Close-range (ground-based) observations
• Typically, 5-25 meter standoff from buildings
• Sensor operable by one person (a UROP)
• Small, rolling platform (sidewalks, access ramps)
• End-to-end time (overlapped) of days
• Acquisition time, (wall-clock) processing
• No manual data processing
• No film, scanning, feature indication, etc.

5
Geo-located digital camera
Cheap digital cameras, GPS, MEMS inertial
chipsets soon available also MAVs
6
Acquiring observations

• Each with a tag that records date,
  time,(estimated) camera position, orientation

7
Image acquisition Test dataset
Early prototype of pose camera deployed in and
around Tech Square (4 structures) Collected 81
nodes 4,000 geo-located images
8
Register images
9
Structure extraction
Sweep-plane algorithm identifies locations and
extents of significant vertical façades
10
Preliminary results (with overlay of aerial image)
Model represents about 1 CPU-Day at 200 MHz
Next acquire full MIT campus compare to
refer-ence model captured via traditional
surveying
11
Principles

• Study previous successes failures
• Avoid black holes
• Architect an end-to-end system
• Use stand-ins for parallel development
• Design virtual sensors
• Make them good enough to do the job
• Design with scaling in mind
• Bound growth with input, output size
• Make data dispensable
• Use ensemble data for robustness
• Self-check until validated (and afterward!)
• Tap, inspect, and validate

12
Study successes failures

• Previous systems foundered because of
• Insufficient data (too few images)
• Couldnt resolve detail
• Lack of initial estimates for camera registration
• Required human in the loop for initialization
• Reliance on correspondence (30-year black hole)
• Matching features across different images
• Combinatorial blowup
• Quadratic, cubic time algorithms clearly dont
  scale well
• So
• Gather lots of (approximately) registered imagery
• Eliminate feature correspondence entirely
• Use spatial data structures for linear growth

13
Architect end-to-end system

• Simple modules first elaborate them later
• Use stand-ins for parallel development

14
Design Virtual Sensors

• Make them good enough to do the job
• Mechanical pointing accuracy 1 degreeCCD camera
  dynamic range 7 bits
• After optimization 1 milliradian (1/20
  degree)High dynamic range 15 bits

15
High dynamic range

• Simulate technology trends in software

16
Mosaics A Closer Look
Each mosaic has 100s of Mega-pixels
17
Trading space for time
18
Design with scaling in mind

• Acquire meta-data for spatial indexing
• Use spatial data structures to organize data
• Strive for linear storage, computation timeas
  function of input and output complexity

19
Make data dispensable

• Use ensemble data for robustness

20
Texture estimation challenge
21
Texture estimation results
Input Raw imagery
Output Synthetic texture

• Made possible by many observations
• A sensor and system that effectively see
  through complex foliage!

22
Self-checking algorithms

• Tap, inspect, and validate

23
Further information

• graphics.lcs.mit.edu
• graphics.lcs.mit.edu/seth
• graphics.lcs.mit.edu/city/city.html
• graphics.lcs.mit.edu/publications.html

Students wanted!

• UROPs, AUPs, MEngs, PhDs
• 6-1s, 6-2s for Argus, Rover instrumentation
• 6-2s, 6-3s for System architecting, appns