Dynamic Refraction Stereo

1
Dynamic Refraction Stereo
Kiriakos N. Kutulakoskyros _at_ cs.toronto.edu
Nigel J. W. Morrisnmorris _at_ dgp.toronto.edu
7. Contributions
2. Related Work

• Refractive disparity optimization gives stable
  reconstructions regardless of surface shape
• Require no geometric assumptions other than
  single light refraction
• Produces a full resolution height map and a
  separate, full-resolution normal map
• Highly-detailed reconstructions for a variety of
  complex, deforming liquid surfaces

• Single-view methods (require extra
  assumptions/optics)
• Shape from Distortion (Murase, PAMI 1992) ?
  assumes constant mean distance
• Shape from Refraction (Jähne et al., JOSA 1994)
  ? uses collimating lens lighting gradient
• Multi-view methods
• Sanderson et al. (PAMI 1998), Bonfort Sturm
  (ICCV 2003) ? static mirror scenes (known
  refractive index), ? optimization degrades
  for shallow liquid heights
• Multi-media photogrammetry (Flach Maas, IAPRS
  2000) ? known parametric shape model

8. Experimental results
Simulations vs. Ground-truth Experiments
Refractive index estimation
q
Total reconstruction error as function of
refractive index (liquid was water)
Observed results (solid red) vs simulated results
(dotted blue) for 0.08 pixel localization error
for set of planar refractive surfaces at various
heights
Experiments with dynamic water surfaces
C(q)
Instantaneous 3D reconstructions
t1.32s
t0.6s
t0.92s
t0.92s
4. Refractive Stereo Algorithm
Height map

• Discretize interval of possible refractive index
  values.For each value, do Steps 1-4
• Step 1 Initialize correspondence function C(.)
  for time 0
• Step 2 For each time t and each pixel q in
  Camera 1
• 2a (Refractive disparity optimization)
  1D optimization along ray of q,
  searching for height hypothesis
  consistent with both viewpoints
• 2b (Bundle adjustment) 5D optimization
  of p and n using reprojection error
• Step 3 Fuse depth normal map to obtain 3D
  surface
• Step 4 Update correspondence function for next
  frame
• Choose refractive index value minimizing total
  reconstruction error (across all frames and
  pixels)

Tilt map
q
q
n
p
t0.4s
t0.3s
t0.35s
t0.3s
C(q)
C(q)
Height map
Tilt map
t0.91s
t0.75s
t0.83s
t0.91s
Height map
Tilt map