3DDI Visualization MURI 20002001

1
3DDI Visualization MURI2000-2001

• UC Berkeley and MIT
• February 2000 MURI Review

2
Recap of 3DDI system organization

• Project pipeline

3D capture
Modeling, simulation
Rendering
3D Display
Application Exterior and Interior Urban
Environments
3
Previous review

• Increased emphasis on urban modeling
• Scale, complexity, throughput
• Attention to principled integration
• Particularly of indoor, outdoor dataand various
  simulation engines

4
Overview Urban model acquisition

• Automation
• Automatic exterior calibration of imagery
• Generalization Aggregation
• 3D reconstruction, merging
• Texture, occlusion, relief estimation
• Collaboration with Fua (EPFL) and LeClerc (SRI)
• Symbolic window extraction
• Collaboration with Wang and Hansen (UMass)
• Scale and Throughput
• Data acquisition, distributed processing
• Input and Output Validation
• Sensor improvement surveying efforts

5
Overview Integration efforts

• Weekly UCB/MIT teleconferences in 1999
• Shared message archive, PC/Irix codebase
• Reciprocal visits Summer, Fall 1999
• Generalized database (Rick Bukowski)
• SYLIF automated furniture placement (Kari
  Anne Kjolaas Laura Downs)
• Princeton Radiosity (Mike Wittman)
• Indoor-outdoor visibility (Eric Brittain) 
• Floorplan annotation and extrusion
• With MIT physical plant City of Cambridge
• Autostereoscopic display (Steve Benton)

6
Alignment is automation bottleneck

• Overview of end-to-end pipeline
• Recovering rotation and translationfor acquired
  hemispherical images
• Short-baseline techniques not applicable
• Exploit navigation informationlarge number
  (1000s) of images
• Tack decouple rotation, translationsolve
  independently (w/ Matt Antone)

7
Generalization and Aggregation

• Several 3D reconstruction techniques
• Large planar surfaces (Coorg)
• Small surfels (Mellor)
• Bottom-up surface inferences (Chou)
• Aggregation phase (Cutler)
• Principled merging of algorithm outputsto
  produce single consistent CAD model

8
Texture, Occlusion, Relief Estimation

• Fua and LeClercs mesh optimization
• Students Eric Amram, Stefano Totaro
• Added iterative occlusion masking
• Improved self-occlusion checking
• Significantly improved results

9
Symbolic Window Extraction

• Based on algorithm of Wang et al. (Proc. SPIE
  97)
• Oriented region-growing technique
• Applied to composite façade images
• After removal of occlusion, shadows
• Planned application to
• Mesh regularization (quantized depth)
• Model color from multiple distributions

10
Increasing Scale, Throughput

• Scale sensor, spatial infrastructure
• Sensor node time reduced to 1 minute from 5
  minutes in 1998, including HDR
• Input second MIT dataset, severalhundred nodes
  across East Campus
• Throughput map algorithms to parallel,
  distributed Linux cluster
• 1-32 CPUs with near-linear speedups
• Currently limited by I/O bandwidth

11
Campus datasets
East Campus
12
Validation

• Input (Mike Bosse)
• Survey waypoints to characterizeprecision,
  accuracy of navigation sensor
• Suppress sub-systems (GPS, inertial, odometry) to
  gauge contribution of each
• Output (Qixiang Sun)
• Synthetic inputs, idealized results
• Real inputs, optimization residuals
• Compare reconstructed models tosurveyed,
  hand-solved models

13
Evaluation Criteria

• Throughput
• Complexity
• Fidelity (Geometric, Photometric)
• Adoption of tools and models by users
• Assessment of results by community

14
Connections to other communities

• MIT Physical Plant
• Well-maintained 2D CAD
• City of Cambridge Planning Dept.
• Surveying, GIS expertise/expectations
• MIT Depts of Architecture, Urban Planning
• GIS software, demographic data

15
Conclusions

• Emphasis on infrastructure, solid engineering,
  scale, validation
• Significant progress toward rapid, end-to-end
  acquisition capability
• Significant infrastructure and integration
  efforts for simulation, use

16
Plans for coming year

• Improvements to City Scanning
• Larger scenes
• Faster end-to-end processing
• More robust, accurate results
• Continue integration efforts
• MURI pipeline (collaborators)
• MIT physical plant (customers)

17
City Scanning module improvements

• 10x area implies 10x datasize
• Data scaling, naming conventions
• Speed
• Pose-camera (Argus) improvements
• Parallel distributed processing pipeline
• Accuracy, validation
• 6-DOF raw navigation data
• Imagery control (exterior calibration)
• Derived feature points, edges, faces
• Relief extraction
• Symbolic windows

18
Integration efforts (MURI)

• Elaboration of interior spaces
• Increased symbolic content in DB
• Semi-automated furnishing, etc.
• Use of acquired phototextures
• Breadth of simulation connections
• Lighting, NOW back end
• Enhanced dynamics
• Multi-user operation
• Adoption of segmentation algorithms
• Extended indoor-outdoor visibility
• Visible, occlusion volumes algebra

19
Integration efforts (MIT Phys. Plant)

• Goal compile entire campus into 3D
• Determine required metadata
• Robust large-scale conversion process
• Registration with scanned exteriors
• User-based model personalization
• Deliverable artifacts and tools
• Compilation of revised floorplans
• Shortest path planning (abled, disabled)
• Virtual campus exploration, rendering