Landmark Selection

1
Landmark Selection for Vision-Based Navigation
Pablo L. Sala Joint work with Robert Sim, Ali
Shokoufandeh and Sven Dickinson To be presented
in IROS 2004 September 17th, 2004
2
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I?
• Where am I going?
• How do I get there?

3
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going?
• How do I get there?

4
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there?

5
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there? Path-planning

6
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there? Path-planning

7
Landmark-Based Navigation

• What makes a good landmark?

8
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)

9
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility

10
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?

11
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually

12
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically

13
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically but how?

14
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically
• Store every landmark visible at each location
  (costly!)

15
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically
• Store every landmark visible at each location
  (costly!)
• Find smallest subset of landmarks that supports
  reliable navigation (optimal!)

16
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
17
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

18
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

19
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

20
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

21
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

22
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
23
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
Four features are needed in this set.
24
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
Four features are needed in this set.
Only two features needed. Our goal is to find
this decomposition.
25
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization

• Current pose is estimated using the locations of
  a small number of features in the current image,
  matched against their locations in two model
  views.

26
View-Based Robot Navigation
27
View-Based Robot Navigation
28
View-Based Robot Navigation
29
View-Based Robot Navigation
30
View-Based Robot Navigation
31
View-Based Robot Navigation
32
View-Based Robot Navigation
33
View-Based Robot Navigation
34
View-Based Robot Navigation
35
View-Based Robot Navigation
36
View-Based Robot Navigation
37
Intuitive Problem Formulation
38
Outline

• Problem Formulation
• Complexity
• Heuristic Methods
• Results on Synthetic and Real Images
• Conclusions

39
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MOVRDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that
40
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MOVRDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that   Theorem 1 A
?-MOVRDP can be reduced to an equivalent
0-MOVRDP, and the solution to this latter problem
can be extended to a solution for the original
problem.
41
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MOVRDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that   Theorem 1 A
?-MOVRDP can be reduced to an equivalent
0-MOVRDP, and the solution to this latter problem
can be extended to a solution for the original
problem. Theorem 2 The decision problem
lt0-MOVRDP, dgt is NP-complete. (Proof by
reduction from the Minimum Set Cover Problem.)
42
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.

43
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region
44
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 25
45
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 25
46
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 19
47
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 19
48
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 19
49
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 19
50
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 17
51
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 17
52
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 14
53
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 14
54
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 11
55
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 11
56
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 9
57
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 8
58
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 8
59
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 6
60
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 4
61
Heuristic Methods for 0-MOVRDP

• 0-MOVRDP is intractable.
• Optimal decomposition not needed in practice.
• We developed and tested six greedy approximation
  algorithms.
• Algorithm A.x O(V2F)

k 4
Features commonly visible in region 4
62
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region
63
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
64
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
65
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
66
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
67
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
68
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
69
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
70
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
71
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
72
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
73
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
74
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 3
75
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 4
76
Heuristic Methods for 0-MOVRDP
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 5
77
Results
Simulated Data
78
Results
Simulated Data (cont.)
79
Results

• Simulated Data (cont.)
• World Settings
• Two types of Worlds Irregular (Irreg) and
  Rectangular (Rect).
• average diameter 40m.
• pose space sampled at 50 cm intervals.
• average number of sides 6.
• average number of obstacles 7.

80
Results

• Simulated Data (cont.)
• World Settings
• Two types of Worlds Irregular (Irreg) and
  Rectangular (Rect).
• average diameter 40m.
• pose space sampled at 50 cm intervals.
• average number of sides 6.
• average number of obstacles 7.
• Two types of Features Short-Range and
  Long-Range.
• visibility range N(0.65, 0.2) to N(12.5, 1) m,
• and angular range N(25, 3) degrees.
• Visibility range N(0.65, 0.2) to N(17.5, 2) m,
• and angular range N(45, 4) degrees.

81
Results (cont.)

• Simulated Data (cont.)
• 300 randomly generated worlds
• Runtime of few seconds
• Avg. size of regions depends on stability of
  features in pose space.
• Number of regions increases as avg. size of
  regions decreases.
• Alg. B.2 achieved the best results.

Algorithms Algorithms
? A.1
? A.2
? A.3
? B.1
? B.2
? C
Setting World Feature
1 Rect Short-Range
2 Rect Long-Range
3 Irreg Short-Range
4 Irreg Long-Range
82
Results (cont.)
Real Data We applied the best-performing
algorithm (B.2) to real feature visibility data.
0?
90?
180?
270?
83
Results (cont.)

• Real Data
• Experiment 1
• Data collected in 2m ? 2m area.
• Sampled at 20 cm intervals.
• Total of 46 visible features.
• Camera at a fixed orientation (looking forward).
• Features were extracted using the
  Kanade-Lucas-Tomasi operator.
• Parameters used ? 0, k 4.

84
Results (cont.)

• Real Data
• Experiment 1
• Data collected in 2m ? 2m area.
• Sampled at 20 cm intervals.
• Total of 46 visible features.
• Camera at a fixed orientation (looking forward).
• Features were extracted using the
  Kanade-Lucas-Tomasi operator.
• Parameters used ? 0, k 4.

85
Results (cont.)

• Experiment 2
• Data collected in 6m ? 3m area.
• Sampled at 25 cm intervals.
• Total of 897 visible features.
• Camera at 0, 90, 180, and 270
• degree orientations.
• Lowes SIFT features.

86
Results (cont.)

• Experiment 2
• Data collected in 6m ? 3m area.
• Sampled at 25 cm intervals.
• Total of 897 visible features.
• Camera at 0, 90, 180, and 270
• degree orientations.
• Lowes SIFT features.

Typical Feature Visibility Regions
87
Results (cont.)

• Experiment 2 (cont.)
• k4, ?0

88
Results (cont.)

• Experiment 2 (cont.)
• k4, ?1

89
Results (cont.)

• Experiment 2 (cont.)
• k10, ?0

90
Results (cont.)

• Experiment 2 (cont.)
• k10, ?1

91
Conclusions

• We have introduced a novel graph theoretic
  formulation of the landmark acquisition problem,
  and have established its intractability.

• We have explored a number of greedy approximation
  algorithms, systematically testing them on
  synthetic worlds and demonstrating them on two
  real worlds.

• The resulting decompositions find large regions
  in the world in which a small number of features
  can be tracked to support efficient on-line
  localization.

• The formulation and solution are general, and can
  accommodate other classes of image features.

92
Future Work

• Integrate the image collection phase with the
  region decomposition stage to yield an on-line
  process for simultaneous exploration and
  localization (SLAMB).

• Path planning through decomposition space,
  minimizing the number of region transitions in a
  path.

• Detect and cope with environmental change.

• Compute the performance guarantee of our
  heuristic methods and provide tight upper bounds
  on the quality of our solution compared to the
  optimal.

• Use feature tracking during the image collection
  stage to achieve larger areas of visibility for
  each feature. (Maintain equivalence classes of
  features in the DB.)