Statistics in the Image Domain for Mobile Robot Environment Modeling

1
Statistics in the Image Domain forMobile Robot
Environment Modeling

• L. Abril Torres-Méndez and Gregory Dudek
• Centre for Intelligent Machines
• School of Computer Science
• McGill University

2
Our Application

• Automatic generation of 3D maps.
• Robot navigation, localization
• - Ex. For rescue and inspection tasks.
• Robots are commonly equipped with camera(s) and
  laser rangefinder.
• Would like a full range map of the
• the environment.
• Simple acquisition of data

3
Problem Context

• Pure vision-based methods
• Shape-from-X remains challenging, especially in
  unconstrained environments.
• Laser line scanners are commonplace, but
• Volume scanners remain exotic, costly, slow.
• Incomplete range maps are far easier to obtain
  that complete ones.
• Proposed solution Combine visual and partial
  depth Shape-from-(partial) Shape

4
Problem Statement
From incomplete range data combined with
intensity, perform scene recovery.
5
Overview of the Method

• Approximate the composite of intensity and
  range data at each point as a Markov process.
• Infer complete range maps by estimating joint
  statistics of observed range and intensity.

6
What knowledge does Intensity provide about
Surfaces?

• Two examples of kind of inferences

Intensity image Range image
7
What about Edges?

• Edges often detect depth discontinuities
• Very useful in the reconstruction process!

Intensity Range
edges
8
Isophotes in Range Data

• Linear structures from initial range data
• All normals forming same angle with direction to
  eye

Intensity Range
9
Range synthesis basis

• Range and intensity images are correlated, in
• complicated ways, exhibiting useful
  structure.
• - Basis of shape from shading shape from
  darkness, but they are based
• on strong assumptions.
• The variations of pixels in the intensity and
  range images are related to the values
  elsewhere in the image(s).

Markov Random Fields
10
Related Work

• Probabilistic updating has been used for
• image restoration e.g. Geman Geman, TPAMI
  1984 as well as
• texture synthesis e.g. Efros Leung, ICCV
  1999.
• Problems Pure extrapolation/interpolation
• is suitable only for textures with a stationary
  distribution
• can converge to inappropriate dynamic equilibria

11
MRFs for Range Synthesis

• States are described as augmented voxels
  V(I,R,E).
• Zm(x,y)1x,ym mxm lattice over which the
  image are described.
• I Ix,y, (x,y)? Zm intensity (gray or color)
  of the input image
• E is a binary matrix (1 if an edge exists and 0
  otherwise).
• RRx,y, (x,y)? Zm incomplete depth values
  
• We model V as an MRF. I and R are random
  variables.

R
vx,y
I
12
Markov Random Field Model

• Definition A stochastic process for which a
  voxel value is predicted by its neighborhood in
  range and intensity.

Nx,y is a square neighborhood of size nxn
centered at voxel Vx,y.
13
Computing the Markov Model

• From observed data, we can explicitly compute

Nx,y
Vx,y

• This can be represented parametrically or via a
  table.
• To make it efficient, we use the sample data
  itself as a table.

14
Estimation using the Markov Model

• Fromwhat should an unknown range value be?
• For an unknown range value with a known
• neighborhood, we can select the
  maximum
• likelihood estimate for Vx,y.

15
Interpolate PDF

• In general, we cannot uniquely solve the desired
  neighborhood configuration, instead assume

The values in Nu,v are similar to the values in
Nx,y, (x,y) ? (u,v). Similarity measure
Gaussian-weighted SSD (sum of squared
differences). Update schedule is purely
causal and deterministic.
16
Order of Reconstruction

• Dramatically reflects the quality of result
• Based on priority values of voxels to be
  synthesize
• EdgesIsophotes indicate which voxels are
  synthesized first

• ? Region to be synthesized (target region)
• ?? The contour of target region
• ? The source region ? ?i ?r

17
Priority value computation
Confidence value
Data term value
18
Experimental Evaluation
Input data given to our algorithm
Scharstein Szeliskis Data Set Middlebury
College
19
Isophotes vs. no Isophotes Constraint
Results without isophotes
Results using isophotes
Synthesized range images
20
More examples
Synthesized result. MAR error 5.94 cms.
21
More examples
Synthesized result. MAR error 5.44 cms.
22
More examples
Synthesized result. MAR error 7.54 cms.
23
Adding Surface Normals

• We compute the normals by fitting a plane
• (smooth surface) in windows of mxm pixels.
• Normal vector Eigenvector with the smallest
  eigenvalue of the covariance matrix.
• Similarity is now computed between surface
• normals instead of range values.

24
Adding Surface Normals
25
More Experimental Results
Initial range scans
Synthesized range image
26
More Experimental Results
Synthesized range image
27
Conclusions

• Works very well -- is this consistent?
• Can be more robust than standard methods (e.g.
  shape from shading) due to limited dependence on
  a priori reflectance assumptions.
• Depends on adequate amount of reliable range as
  input.
• Depends on statistical consistency of region to
  be constructed and region that has been measured.

28
Discussion Ongoing Work

• Surface normals are needed when the input range
  data do not capture the underlying structure
• Data from real robot
• Issues non-uniform scale, registration,
  correlation on different type of data
• Integration of data from different viewpoints

29
Questions ?