Minsu Cho

1
Day and Night, 1938, M.C. Escher
Bilateral Symmetry Detection via Symmetry-Growing
Minsu Cho Kyoung Mu Lee Department of
EECS Seoul National University, Korea
2
Symmetry Everywhere
Symmetry is a complexity-reducing concept ...
seek it everywhere. - Alan J. Perlis
Symmetry creates a spontaneous impression of
balance, harmony and order. - Gombrich 1984
Kreitler and Kreitler 1972
Symmetry provides humans with pre-attentive cues
that enhance object recognition. - R. W. Conners
and C. T. Ng 1989
3
Symmetry
invariance of a configuration of elements under
a group of automorphic transformations.
- Hermann Weyl, Symmetry 1952

• Four basic symmetry operations in 2D
• This work is on bilateral (reflectional) symmetry

http//mathforum.org/sum95/suzanne/symsusan.html
4
Symmetry Detection

• Given

An Image
5
Related Work

• Global methods the entire image as a signal
• Not robust to background clutter
• Local methods Grouping symmetric sets of local
  features
• efficiently detect local symmetries against
  background clutters
• But, largely influenced by initial feature
  detection step

C. Sun and D. Si. RTI1999
Y. Keller and Y. Shkolnisky ICPR2004
G. Loy and J.O. Eklundh ECCV2006
S. Lazebnik et al. BMVC2004
H. Cornelius et al. SCIA2007
6
Our Contributions

• An efficient and powerful method via
  symmetry-growing
• Direct feature grouping in the growing without
  voting
• Robust to low inlier features and deformation
• Produces dense symmetric features
• State-of-the art performance in detection accuracy

Global methods Exploit global information, but
vulnerable to clutter
Local methods Robust to clutter, but restricted
to detected features
Our method Explore image regions to find further
symmetry beyond detected symmetric features
7
Overview
A given image
8
Overview
Local feature detection
9
Overview
Local feature detection
Symmetry seed extraction
10
Overview
Symmetry-growing
11
Overview
Local feature detection
Symmetry Verification
12
Step1 Symmetry Seed Extraction
Goal Extract seed matches for symmetric patterns
from the given image
Seed extraction
Symmetry-growing
Verification
13
Local feature detection

• Any of scale- or affine-invariant feature
  detectors

• SIFT (Lowe 99)
• MSER (Matas et al 02)
• Harris Hessian-affine (Mikolajczyk and Schmid
  04)

Rb
Ra
14
Symmetric feature pairs

• Mirror matching with normalized feature regions

Region Orientation Normalization
Rb
Mirror Matching
ka
kb
Ra
kb
G. Loy and J. O. Eklundh. Detecting symmetry and
symmetric constellations of features. ECCV2006.
15
Symmetry score

• Reisfeld's phase weighting function
• Symmetry seeds feature pairs with positive
  weights

Reisfeld D., Wolfson H., and Yeshurun Y. Context
free attentional operators the generalized
symmetry transform. Int. J. of Computer Vision,
Special Issue on Qualitative Vision, 1994.
16
Step2 Symmetry-Growing
Goal Grow the obtained symmetry seeds by
multi-layer symmetry-growing
Seed extraction
Symmetry-growing
Verification
17
Symmetry Cluster Initilization

• Initially, each seed constitutes a singleton
  cluster

Cluster 2
Cluster 3
Cluster 1
Cluster 4
18
Supporter List Initilization

• Initialize supporter list as the set of seed
  matches

Cluster 2
Cluster 3
Cluster 1
Cluster 4
19
Iterative Growing Process

• Pick out the best supporter, and grow its cluster

The best supporter
Expand its cluster
Add expanded matches
Cluster 2
Cluster 3
Cluster 1
Cluster 4
20
Iterative Growing Process

• Pick out the best supporter, and grow its cluster

The best supporter
Expand its cluster
Add expanded matches
Cluster 2
Cluster 3
Cluster 1
Cluster 4
21
Expansion via symmetry propagation

• Propagate a neighbor region via a supporter

Rb
Ti
Ti
Rd
Ra
Rc
22
Expansion via symmetry propagation

• Refine the propagated region

Rb
Ti
Ti
Rd
Ra
Rc
Vittorio Ferrari, Tinne Tuytelaars, and Luc Gool.
Simultaneous object recognition and segmentation
from single or multiple model views. IJCV, 2006
23
Expansion Layer

• Which neighboring regions the cluster propagates?
• Expansion layer - the set of regions to be
  propagated
• Each cluster has it own expansion layer!

24
Expansion

• A supporter Mi propagates its neighboring
  regions on its clusters expansion layer around
  the larger feature of Mi

25
Expansion

• Update expansion layer after expansion
• Expanded regions are eliminated from its own
  expansion layer

26
Merge

• Merge when two clusters are connected by
  expansion
• Only if a cluster propagate an equivalent match
  in another cluster
• equivalent matches overlaps over 50 in both
  local regions

27
Merge

• Update their expansion layers by intersection

Merge

n
28
Symmetry-Growing

• Our multi-layer growing framework enables
  overlapping symmetries robust feature grouping

29
Symmetry-Growing

• Locally symmetric parts are inferred by the
  feature distribution

30
Step3 Symmetry Verification
Goal Eliminate the unreliable clusters from the
grown symmetry clusters
Seed extraction
Symmetry-growing
Verification
31
Symmetry Cluster Verification

• Eliminate unreliable symmetry clusters
• The more reliable ones grow larger
• Both reflective areas of the cluster are larger
  than daI (I the area of the given
  image)

32
Experimental Results

• Settings
• MSER Hessian affine detector, SIFT descriptor
• Parameters
• Radius of latent regions 1/25 the shorter image
  axis
• Similarity threshold ds 0.7, phase weight
  threshold dF 0.99
• Reliable cluster threshold da 0.02
• Test dataset
• The PSU Ref. symmetry dataset
• Comparison with the evaluation results in M. Park
  et al.s, CVPR2008

( http//vision.cse.psu.edu/evaluation.html )
G. Loy and J. O. Eklundh. Detecting symmetry and
symmetric constellations of features. ECCV2006.
Ying-Qing Xu, Yanxi Liu, James H. Hays and
Heung-Yeung Shum. Digital papercutting.,
SIGGRAPH2005.
33
Comparative examples
Input Our result
LE06 LHS05
34
Comparative examples
Input Our result
LE06 LHS05
35
Experimental Results

• Our results on images with single symmetry
  patterns

36
Experimental Results

• Our results on images with multiple symmetry
  patterns

37
Experimental Results

• Quantitative results
• Measure sensitivity false positive rate
• On all the 83 images of PSU Ref. symmetry dataset
• Ground truth other results from M. Park et
  al.s CVPR2008
• Overall S0 84 (20 than LE06), RFP 38
  (-4 than LE06)

38
Detected Symmetries beyond Ground Truth

• Examples with a single symmetry pattern

Input Our
result Ground
Truth
39
Detected Symmetries beyond Ground Truth

• Examples with multiple symmetry patterns

Input Our
result Ground Truth
40
Conclusion Future Work

• Symmetry-Growing
• overcomes the locality of local feature based
  methods
• detects detailed partial symmetries
• especially robust detection on real-world complex
  images
• Future Work
• Other types of symmetry
• Large deformation
• Application to object recognition

41
Thanks for your attention! http//cv.snu.ac.kr