Motion from image and inertial measurements additional slides

1
Motion from image and inertial measurements
(additional slides)

• Dennis Strelow
• Carnegie Mellon University

2
Outline
Robust image feature tracking (in detail)
Lucas-Kanade and real sequences The smalls
tracker Motion from omnidirectional images
3
Robust image feature tracking Lucas-Kanade and
real sequences (1)

• Combining image and inertial measurements
  improves our situation, but
• we still need accurate feature tracking tracking
• some sequences do not come with inertial
  measurements

4
Robust image feature tracking Lucas-Kanade and
real sequences (2)

• better feature tracking for improved 6 DOF motion
  estimation
• remaining results will be image-only

5
Robust image feature tracking Lucas-Kanade and
real sequences (3)

• Lucas-Kanade has been the go-to feature tracker
  for shape-from-motion
• minimizes a correlation-like matching error
• using general minimization
• evaluates the matching error at only a few
  locations
• subpixel resolution

6
Robust image feature tracking Lucas-Kanade and
real sequences (4)
Additional heuristics used to apply Lucas-Kanade
to shape-from-motion
7
Robust image feature tracking Lucas-Kanade and
real sequences (5)
But Lucas-Kanade performs poorly on many real
sequences
8
Robust image feature tracking the smalls
tracker (1)

• smalls is a new feature tracker targeted at 6 DOF
  motion estimation
• exploits the rigid scene assumption
• eliminates the heuristics normally used with
  Lucas-Kanade
• SIFT is an enabling technology here

9
Robust image feature tracking the smalls
tracker (2)

• First step epipolar geometry estimation
• use SIFT to establish matches between the two
  images
• get the 6 DOF camera motion between the two
  images
• get the epipolar geometry relating the two images

10
Robust image feature tracking the smalls
tracker (3)
11
Robust image feature tracking the smalls
tracker (4)

12
Robust image feature tracking the smalls
tracker (5)

• Second step track along epipolar lines
• use nearby SIFT matches to get initial position
  on epipolar line
• exploits the rigid scene assumption
• eliminates heuristic pyramid

13
Robust image feature tracking the smalls
tracker (6)

• Third step prune features
• geometrically inconsistent features are marked as
  mistracked and removed
• clumped features are pruned
• eliminates heuristic detecting mistracked
  features based on convergence, error

14
Robust image feature tracking the smalls
tracker (7)

• Fourth step extract new features
• spatial image coverage is the main criterion
• required texture is minimal when tracking is
  restricted to the epipolar lines
• eliminates heuristic extracting only textured
  features

15
Robust image feature tracking the smalls
tracker (8)
16
Robust image feature tracking the smalls
tracker (9)
left odometry only
right images only

• average error 1.74 m
• maximum error 5.14 m

• total distance 230 m

17
Robust image feature tracking the smalls
tracker (10)

• Recap
• exploits the rigid scene and eliminates
  heuristics
• allows hands-free tracking for real sequences
• can still be defeated by textureless areas or
  repetitive texture

18
Outline
Robust image feature tracking (in detail) Motion
from omnidirectional images
19
Motion from omnidirectional images (1)
20
Motion from omnidirectional images (2)
21
Motion from omnidirectional images (3)
22
Motion from omnidirectional images (4)
23
Motion from omnidirectional images (5)
left non-rigid camera
right rigid camera
squares ground truth points solid
image-only estimates dash-dotted
image-and-inertial estimates
24
Motion from omnidirectional images (6)

• In this experiment
• omni images
• conventional images inertial
• have roughly the same advantages

• But in general
• inertial has some advantages that omni images
  alone cant produce
• omni images can be harder to use