Image Based Routing for Image Based Rendering

1
Image Based Routing forImage Based Rendering

• Daniel Crispell daniel_crispell_at_brown.edu
• Gabriel Taubin taubin_at_brown.edu
• John Jannotti jj_at_cs.brown.edu

OMNIVIS 2005 Beijing, China
2
Outline

• Introduction / Previous Work
• High Level Algorithm
• The Binmesh
• Detailed Algorithm
• Results

3
Introduction

• Large amounts of research focused on multi-camera
  algorithms
• Most algorithms assume a centralized topology
• We wish to operate in a distributed manner to
  improve scalability

4
Distributed IBR

• Image Based Rendering
• Arbitrary number of networked cameras
• Arbitrary locations and pose
• Arbitrary network topology

5
Selected Previous Work

• Buehler et. al. (2001)
• Unstructured Lumigraph Rendering
• Yang, Everett, Buehler, McMillan (2002)
• A Real-time distributed Light field Camera
• Wilburn et. al. (2005)
• High Performance Imaging using
• Large Camera Arrays

6
Outline

• Introduction / Previous Work
• High Level Algorithm
• The Binmesh
• Detailed Algorithm
• Results

7
Algorithm Overview

• Each node is responsible for
• Summary of available views from its children and
  their descendants

8
Algorithm Overview

• Each node is responsible for
• Aggregation of views for presentation to parent

9
Algorithm Overview

• Each node is responsible for
• Distribution of requests from parent node for a
  virtual view

10
Algorithm Overview

• Each node is responsible for
• Combination of responses to requests from
  children for presentation to parent

11
Outline

• Introduction / Previous Work
• High Level Algorithm
• The Binmesh
• Detailed Algorithm
• Results

12
Binmesh

• Based on Geometric Proxy

13
Binmesh

• Stores directions of viewing cameras at each face
  of proxy
• One set of vector bins for every face of proxy
  mesh
• Easily aggregated

14
Binmesh

• Vector Quantization
• Local coordinate system
• z axis face normal
• Determine intersection with unit pyramid
• Project onto XY plane
• Change of variable

15
Binmesh

• Aggregation
• Binmesh structures are easily aggregated
• Element by element addition
• Uncompressed structure is constant size with
  respect to number of views represented
• Typically sparse, easily compressible
• Example
• 338 face proxy, 6 subdivision levels (46 bins),
  1024 cameras
• 1.32 MB uncompressed, 21kB compressed (gzip)

16
Outline

• Introduction / Previous Work
• High Level Algorithm
• The Binmesh
• Detailed Algorithm
• Results

17
Rendering Algorithm

• Spanning Tree Construction
• Virtual camera node injects request into system
• Request contains geometric proxy mesh

18
Rendering Algorithm

• Spanning Tree Construction
• First node to receive request becomes root of the
  spanning tree

19
Rendering Algorithm

• Spanning Tree Construction
• Request is propagated until spanning tree is
  constructed

20
Rendering Algorithm

• Spanning Tree Construction
• Binmesh Structures filled in and propagated back
  up tree

21
Rendering Algorithm

• Novel View Generation
• Virtual camera node sends
• Virtual camera parameters
• Set of visible faces to render
• Number of views k to blend for each face

22
Rendering Algorithm

• Novel View Generation
• Nodes determine which children to send requests
  for faces to

23
Rendering Algorithm

• Novel View Generation
• Once replies are received from all children,
  results are blended and returned to parent

24
Rendering Algorithm

• Novel View Generation
• Root node passes virtual camera completed view

25
Outline

• Introduction / Previous Work
• High Level Algorithm
• The Binmesh
• Detailed Algorithm
• Results

26
Large Network Simulation

27
Large Network Simulation

• Organized Network Connections

Random Network Connections
28
Large Network Simulation
29
Large Network Simulation
30
Prototype Implementation 1

• 4x4 camera array
• Routers simulated
• Novel views generated in real-time
• Virtual camera has very limited range

31
Prototype Implementation 2

• 28 cameras arranged in 4x1 arrays
• 8 Linux nodes performing routing and image
  processing

32
Future Work

• Dynamic, Distributed Calibration
• Geometric Proxy Refinement
• Improve Synchronization
• More cameras

33
Thank You

• Questions?