Landmark Selection

1
Landmark Selection for Vision-Based Navigation
IEEE/RSJ International Conference on Intelligent
Robots and Systems October 2nd, 2004 Pablo
Sala1, Robert Sim1,2, Ali Shokoufandeh3, Sven
Dickinson1 1 University of Toronto 2
University of British Columbia 3 Drexel
University
2
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I?
• Where am I going?
• How do I get there?

3
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going?
• How do I get there?

4
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there?

5
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there? Path-planning

6
Robot Navigation

• Leonard and Durrant-Whyte
• Where am I? Localization
• Where am I going? Goal Identification
• How do I get there? Path-planning

7
Landmark-Based Navigation

• What makes a good landmark?

8
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)

9
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility

10
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?

11
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually

12
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically

13
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically but how?

14
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically
• Store every landmark visible at each location
  (costly!)

15
Landmark-Based Navigation

• What makes a good landmark?
• Distinctiveness (does it tell me where I am?)
• Wide Visibility
• How do we select good landmarks?
• Manually
• Automatically
• Store every landmark visible at each location
  (costly!)
• Find smallest subset of landmarks that supports
  reliable navigation (optimal!)

16
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
17
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

18
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

19
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

20
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

21
View-Based Robot Navigation
Landmark Database Construction
On-line Localization
Off-line Exploration

• Collection of images acquired at known discrete
  points in pose space. Pose recorded and image
  features extracted and stored in database.

22
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
23
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
Four features are needed in this set.
24
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization
Four features are needed in this set.
Only two features needed. Our goal is to find
this decomposition.
25
View-Based Robot Navigation
Off-line Exploration
Landmark Database Construction
On-line Localization

• Current pose is estimated using the locations of
  a small number of features in the current image,
  matched against their locations in two model
  views.

26
View-Based Robot Navigation
27
View-Based Robot Navigation
28
View-Based Robot Navigation
29
View-Based Robot Navigation
30
View-Based Robot Navigation
31
View-Based Robot Navigation
32
View-Based Robot Navigation
33
View-Based Robot Navigation
34
View-Based Robot Navigation
35
View-Based Robot Navigation
36
View-Based Robot Navigation
37
Robot Navigation

• Visual Landmarks

• View-Based

38
Intuitive Problem Formulation
39
Intuitive Problem Formulation
40
Intuitive Problem Formulation
41
Intuitive Problem Formulation
42
Intuitive Problem Formulation
43
Intuitive Problem Formulation
44
Intuitive Problem Formulation
45
Intuitive Problem Formulation
46
Intuitive Problem Formulation
47
Outline

• Problem Formulation
• Complexity
• Heuristic Methods
• Results on Synthetic and Real Images
• Conclusions

48
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MORDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that
49
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MORDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that   Theorem 1 A
?-MORDP can be reduced to an equivalent 0-MOVRDP,
and the solution to this latter problem can be
extended to a solution for the original problem.
50
A Graph Theoretic Formulation
Problem Definition The ?-Minimum Overlapping
Region Decomposition Problem (?-MORDP) for a
world instance ltG(V,E), F, ?v v?Vgt consists of
finding a minimum size ?-overlapping
decomposition D R1, , Rd
of V into regions such that   Theorem 1 A
?-MORDP can be reduced to an equivalent 0-MOVRDP,
and the solution to this latter problem can be
extended to a solution for the original
problem. Theorem 2 The decision problem
lt0-MORDP, dgt is NP-complete. (Proof by reduction
from the Minimum Set Cover Problem.)
51
Heuristic Methods for 0-MORDP

• 0-MORDP is intractable.

• Can we efficiently find an effective
  approximation?

• We developed and tested six greedy approximation
  algorithms.

52
Algorithm A.x O(V2F)
k 4
Features commonly visible in region
53
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 25
54
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 25
55
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 19
56
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 19
57
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 19
58
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 19
59
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 17
60
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 17
61
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 14
62
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 14
63
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 11
64
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 11
65
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 9
66
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 8
67
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 8
68
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 6
69
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 4
70
Algorithm A.x O(V2F)
k 4
Features commonly visible in region 4
71
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region
72
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
73
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
74
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
75
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
76
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
77
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 1
78
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
79
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
80
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
81
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
82
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 2
83
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 3
84
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 4
85
Algorithms B.x and C O(kV2F)
k 5
Features commonly visible in region 5
86
Results
Simulated Data
87
Simulated Data (cont.)

• Two types of Worlds Irregular (Irreg) and
  Rectangular (Rect).
• average diameter 40m.
• pose space sampled at 50 cm intervals.
• average number of sides 6.
• average number of obstacles 7.

• Two types of Features Short-Range and
  Long-Range.
• visibility range N (0.65, 0.2) to N (12.5, 1) m,
• and angular range N (25, 3) degrees.
• Visibility range N (0.65, 0.2) to N (17.5, 2) m,
• and angular range N (45, 4) degrees.

88
Simulated Data (cont.)
89
Real Data
We applied the best-performing algorithm (B.2)
to real feature visibility data.
0?
90?
180?
270?
90
Results (cont.)

• Real Data
• Experiment 1
• Data collected in 2m ? 2m area.
• Sampled at 20 cm intervals.
• Total of 46 visible features.
• Camera at a fixed orientation (looking forward).
• Features were extracted using the
  Kanade-Lucas-Tomasi operator.
• Parameters used ? 0, k 4.

91
Results (cont.)

• Real Data
• Experiment 1
• Data collected in 2m ? 2m area.
• Sampled at 20 cm intervals.
• Total of 46 visible features.
• Camera at a fixed orientation (looking forward).
• Features were extracted using the
  Kanade-Lucas-Tomasi operator.
• Parameters used ? 0, k 4.

92
Real Data (cont.)

• Data collected in 6m ? 3m area.
• Sampled at 25 cm intervals.
• Total of 897 visible features.
• Camera at 0, 90, 180, and 270
• degree orientations.
• SIFT features.

93
Typical Feature Visibility Regions
94
Real Data Decompositions
k 4, ? 0
95
Real Data Decompositions (cont.)
k 4, ? 1
96
Real Data Decompositions (cont.)
k 10, ? 0
97
Real Data Decompositions (cont.)
k 10, ? 1
98
Conclusions

• We have introduced a novel graph theoretic
  formulation of the landmark acquisition problem,
  and have established its intractability.

• We have explored a number of greedy approximation
  algorithms, systematically testing them on
  synthetic worlds and demonstrating them on real
  worlds.

• The resulting decompositions find large regions
  in the world in which a small number of features
  can be tracked to support efficient on-line
  localization.

• The formulation and solution are general, and can
  accommodate other classes of image features.

99
Future Work

• Integrate the image collection phase with the
  region decomposition stage to yield an on-line
  process for simultaneous exploration and
  localization (SLAM).

• Path planning through decomposition space,
  minimizing the number of region transitions in a
  path.

• Detect and cope with environmental change.

• Compute the performance guarantee of our
  heuristic methods and provide tight upper bounds
  on the quality of our solution compared to the
  optimal.

• Use feature tracking during the image collection
  stage to achieve larger areas of visibility for
  each feature. (Maintain equivalence classes of
  features in the DB.)