Occlusion and smoothness probabilities in 3D cluttered scenes

1
Occlusion and smoothness probabilities in 3D
cluttered scenes

• Michael Langer
• School of Computer Science
• McGill University

2
3D cluttered scene
3
Motivation

• INPUT binocular image pair
• OUTPUT depth map (or disparity map)
• that minimizes an energy
• E data E smoothness

4
Motivation

• INPUT binocular image pair,
• OUTPUT depth map (or disparity map)
• that minimizes an energy
• E data E smoothness

likelihood
priors
5
Motivation

• INPUT binocular image pair,
• OUTPUT depth map (or disparity map)
• that minimizes an energy
• E data E smoothness

likelihood
priors
TODAY 3D cluttered scenes
6
Overview of Talk

• Cluttered scenes model
• Visibility probabilities
• monocular
• depth, occlusions, smoothness
• binocular
• disparity, half occlusions

7
Cluttered scene model
8
Cluttered scene model
9
Visibility and depth
observer
visible
occluded
10
Visibility and depth
observer
visible
occluded
11
Visibility and depth
observer
visible
occluded
12
Visibility and depth
observer
visible
occluded
13
Poisson model
density h (sphere centers)
14
Poisson model
V
15
Poisson model
V
16
Cluttered Scene Model
density h , radius R
17
Overview of Talk

• Cluttered scenes model
• Visibility probabilities
• monocular
• depth, occlusions, smoothness
• binocular
• disparity, half occlusions

18
Monocular Visibility
What is the probability that the surface seen at
a pixel is at depth z ?
19
Monocular Visibility
A point at depth Z is visible if no sphere center
lies within a distance R from the line of sight.
20
Poisson model
V
21
Poisson model
22
Depth
p(Z)
p( Z )
depth Z
depth Z
23
Occlusions
Given a pixel is at depth z, what is the
probability that its right neighbor is closer (on
a more nearby i.e. occluding surface) ?
24
Occlusions
25
Discontinuities
near sphere
far sphere
Given a pixel is at depth z, what is the
probability that its right neighbor is further
away (on a more distant surface) ?
26
Discontinuities
27
Binocular Occlusions
28
Half Occlusions
monocular left monocular right
29
Half Occlusions
visible to both eyes
visible to right eye only
occluded
30
Half Occlusions
Given a pixel in the right eyes image has depth
z, what is the probability that the same surface
point is visible to the left eye?
31
Binocular half-occlusions
32
Binocular half-occlusions
monocular
binocular
33
Summary of main results
occlusions
depth
half-occlusions
discontinuities
34
Future Work

• INPUT binocular image pair,
• OUTPUT depth map that minimizes
• the energy
• E data E smoothness occlusions

priors based on 3D cluttered scene models
35
Questions ?
36

Disparity (inverse depth)
37
Disparity (inverse depth)
Disparity (inverse depth)
38

Image Velocity