Techniques for Extracting Contours and Merging Maps

1
Techniques for Extracting Contours and Merging
Maps

• Nagesh Adluru
• Dept. of Computer and Information Sciences
• Temple University

November 26, 2008
2
Acknowledgements

• Longin Jan Latecki (Phd Adviser)
• Committee
• Collaborators
• Discussions

Rolf Lakaemper, Slobodan Vucetic, Marc Sobel
Rolf Lakaemper, ChengEn Lu, Haibin Ling, Marc
Sobel, Xingwei Yang, Thomas Young
Moo Chung, Andrew Alexander, Alexander Yates,
Ganesh Adluru
3

• Journal
• N. Adluru and L. J. Latecki. Contour grouping
  based on contour-skeleton duality, IJCV 2008,
  submitted.
• R. Lakaemper, N. Adluru, L. J. Latecki and R.
  Madhavan. Multi robot mapping using force field
  simulation, JFR 2007.
• Conference Proceedings
• C. Lu, N. Adluru, H. Ling and L. J. Latecki.
  Contour Based Detection in Cluttered
  Environments, CVPR 2009, submitted.
• X. Yang, N. Adluru, L.J. Latecki, X. Bai and A.
  Gross. Symmetry of shapes via self-similarity,
  ISVC 2008.
• R. Lakaemper and N. Adluru. Using virtual scans
  to improve alignment performance in robot
  mapping, PerMIS 2008.
• N. Adluru, L. J. Latecki, M. Sobel and R.
  Lakaemper. Merging maps of multiple robots,
  ICPR 2008.
• R. Lakaemper and N. Adluru. Improving sparse
  laser scan alignment with virtual scans, IROS
  2008.
• N. Adluru, L. J. Latecki, R. Lakaemper, T. Young,
  X. Bai and A. Gross. Contour grouping based on
  local symmetry, ICCV 2007.
• R. Lakaemper, A. Nuechter, N. Adluru and L. J.
  Latecki. Performance of 6D LuM and FFS SLAM An
  Example for comparison using grid and pose based
  evaluation methods, PerMIS 2007.
• R. Lakaemper, N. Adluru and L. J. Latecki. Force
  field based n-scan alignment, ECMR 2007.
• N. Adluru, L. J. Latecki, R. Lakaemper and R.
  Madhavan. Robot mapping for rescue robots, SSRR
  2006.

4
Background
Robotics
Machine Learning
Computer Vision
Robot Mapping
Contour Grouping
Medical Imaging
Psychology
Detecting Symmetry
5
Recognition and Navigation

• Object recognition and autonomous navigation are
  central components for intelligent robots.
• Image based retrieval, video surveillance, rescue
  operations, medical robotics
• My dissertation focused on two important
  components of recognition and navigation
• Contour Grouping
• Robot Mapping

Images taken from Internet.
6
Is recognition easy for humans?

• Can humans can find the swans from these images?

• Welllearned humans can.

7
Outline of the Talk
Contour Grouping
Robot Mapping
8
What are contours?

• Sequence of pixels in an image that are useful in
  understanding the image.
• Contours are useful for encoding shapes of
  objects.
• Shapes are very useful for object recognition.

9
How to find contours?
Extract edge pixels
Group relevant edge pixels
Original image
Edge image
Sample contour in blue
10
Grouping constraints

• Broadly speaking the grouping constraints are
  classified into
• Low level based on perceptual psychology
  (Gestalt principles Wertheimer 1923, closure
  and smoothness Mohan and Nevatia 1992, minimal
  models Feldman 1999).
• Mid level based on geometry of curves,
  orientations Tamrakar and Kimia 2007, motion
  Stein et. al. 2007, symmetry global, Stahl
  and Wang 2006, local Liu et. al. 1998).
• High level Explicit models of interest based on
  application. Currently very fertile area!

11
Traditional Object Recognition Process
Low level segmentation
Parameterize
Original image
Segmented image
Parameterized image
Matching
Rabbit
Database of parameterized images
12
Modern Object Recognition Process
Low level segmentation
Parameterize
Parametric search
Original image
Segmented image
Parameterized image
Matching
Rabbit
Database of parameterized images
13
Our grouping approaches

• Symmetry based models as grouping constraints.
  Grouping tokens small line segments.
• Hierarchy based models as grouping constraints.
  Grouping tokens smooth contour pieces.

14
Symmetry Based Models(Grouping tokens edgels or
small line segments)
15
How to capture symmetry?

Generalize the problem Group both contours and
skeletons
Images taken from Bai et. al. 2007.
16
Symmetry Based Models
Longest paths in symmetry sets
17
Algorithmic Flow
Low level edge linking
Edge extraction
Scale-adaptive edge segmenting
Symmetry based models
Particle filter based search
18
Formal Problem Statement

• Recall that we want to regularize both contour
  and skeleton growth using model and grouping
  constraints.
• Maximum A Posteriori (MAP) estimate of

19
Rao-Blackwellized particle filter using
Contour-Skeleton duality

1. Sampling (S)
2. Importance weighting (I)
3. Resampling (R)
4. Updating the contour

20
Sample Results
21
(No Transcript)
22
(No Transcript)
23
Quantitative Evaluation
24
Recognition with Missing Parts

• For efficiency we can not capture every change in
  an image.
• Objects can be recognized even with missing
  contour fragments!

25
Hierarchy Based Models(Grouping tokens smooth
contour pieces)
26
Hierarchy Based Models
Part-bundle based hierarchy for models
27
Algorithmic Flow
Edge extraction
Low level linking
Hierarchy based models
Efficient search using local and global shape
28
Formal Problem Statement

• Recall that we want to assign the part-bundles to
  edge fragments.

29
Combinatorial search using shape based centroid
voting
30
Formal Problem Statement

• Recall that we want to assign the part-bundles to
  edge fragments.

31
Appearance Helps
Shape only
Shape and Appearance
32
Precision / Recall
33
Robot Mapping
34
Importance of Localization
Robots deployed after the 9/11 attack. Interviews
with the rescue teams revealed heavy limitations
in the usability.

• Robot needs to know where it is (Localization).
• Usually maps are not available.
• GPS fails in indoor and rescue type scenarios.
• Hence simultaneously build maps and localize
  (SLAM).

35
Sensor data
Images from the Internet.
36
SLAM
Standard indoor SLAM
Images courtesy Grisetti et. al.
37
Multiple robots and disaster areas

• Cooperative SLAM using teams of robots is still
  an open problem with good progress being made.
• Disaster areas provide a strong challenge to the
  existing SLAM algorithms.
• FFS is a step towards solving some issues (order
  independence, limited overlap) in the above two
  challenges.

38
Sample scans
39
Sample configuration with 60 scans
40
Force field based merging

• Scans are treated as rigid bodies moving in force
  fields.

41
Force Between Two Points
Visual significance
Spatial significance
42
Force field of a sample configuration with 60
scans
43
Naïve ICP vs. FFS
Iterative Closest Point
FFS
44
Merging in action
45
Occupancy Grid Maps
Image taken from a lecture notes I found using
Google.
46
Initial Occupancy Grid
47
Final Occupancy Grids
FFS after 50 iterations
LuM after 500 iterations
48
Grid Based Evaluation
49
Grid Based Evaluation (Regions)
50
Pose Based Evaluation
51
Thank you for your attention!
52
Details of Symmetry Based Grouping

• Rao-Blackwellized particle filtering
• Key recursions
• Proposal
• Importance weights
• Global localization
• Scale-adaptive edge segmenting

53
Key recursions
Proposal
Importance weights
54
Proposal / Sampling (1)

• State space of symmetric points.

Quality of a SP
55
Proposal / Sampling (2)

• Contour-skeleton smoothness prior.

Contour smoothness
Skeleton smoothness
56
Importance weights (1)

• Symmetry based model.

Longest paths in the symmetry sets
Details of a sample path
57
Importance weights (2)

• Model fitness.

Localizing onto the model using SP vector
compatibility
Contour deformability threshold
Skeleton deformability threshold
58
Global localization
59
Global localization
http//www.cs.washington.edu/ai/Mobile_Robotics/mc
l/animations/global-floor.gif
60
Scale-adaptive edge segmenting (1)
61
Scale-adaptive edge segmenting (2)