A NOVEL LOCAL FEATURE DESCRIPTOR FOR IMAGE MATCHING

1
A NOVEL LOCAL FEATURE DESCRIPTOR FOR IMAGE
MATCHING

• Heng Yang, Qing Wang
• ICME 2008

2
Outline

• Introduction
• Local feature descriptor
• Feature matching
• Experimental result and discussions
• Image matching experiments
• Image retrieval experiments
• Conclusion

3
Local feature descriptor

• Local invariant features have been widely used in
  image matching and other computer vision
  applications
• Invariant to Image rotation, scale, illumination
  changes and even affine distortion
• Distinctive, robust to partial occlusion,
  resistant to nearby clutter and noise
• Two concerns for extracting the local features
• Detect keypoints
• Assigning the localization, scale and dominant
  orientation for each keypoint
• Compute a descriptor for the detected regions
• To be highly distinctive
• As invariant as possible over transformations

4
Local feature descriptor

• At present, the SIFT descriptor is generally
  considered as the most appealing descriptor for
  practical uses
• Based on the image gradients in each interest
  points local region
• Drawback of SIFT in matching step
• High dimensionality (128-D)
• SIFT extensions
• PCA-SIFT descriptor
• Only change SIFT descriptor step
• Pre-compute an eigen-space for local gradient
  patches of size 41x41
• 2x39x393042 elements
• Only keep 20 components
• A more compact descriptor

5
Local feature descriptor

• GLOH (Gradient location-orientation histogram)
• Divides local circular region into 17 location
  bins
• gradient orientations are quantized in 16 bins
• Analyze the 17x16272-d
• Eigen-space (PCA) keep 128 components
• Computationally more expensive and need extra
  offline computation of patch eigen space
• This paper presents a local feature descriptor
  (GDOH)
• Based on the gradient distance and orientation
  histogram
• Reduce the dimensional size of the descriptor
• Maintain distinctness and robustness as much as
  SIFT

6
Local feature descriptor

• First
• Image gradient magnitudes and orientations are
  sampled around the keypoint location
• Assign a weight to the magnitude of each point
• Gaussian weighting function with equal to half
  the width of the sample region is employed
• Reduce the emphasis on the points that are far
  from the center
• Second
• The gradient orientations are rotated relative to
  the keypoint dominant direction
• Achieve rotation invariance
• The distance of each gradient point to the
  descriptor center is calculated
• Final
• Build the histogram based on the gradient
  distance and orientation
• 8(distance bins) 8(orientation bins) 64 bins

7
Feature matching

• Given keypoint descriptors extracted from a pair
  of two images
• Find a set of candidate feature matches
• Using Best-Bin-First (BBF) algorithm
• Approximate nearest-neighbor searching method in
  high dimensional spaces
• Only consider the matches in which the distance
  ratio of nearest neighbor to the second-nearest
  neighbor is less than a threshold
• Correct matches should have the closest neighbor
  significantly closer than the closest incorrect
  match

8
Feature matching

• Find Nearest neighbor feature points
• A variant of the k-d tree search algorithm makes
  indexing in higher dimensional spaces practical.
• Best Bin First
• Approximate algorithm
• Finds the nearest neighbor for a large fraction
  of the queries
• A very close neighbor in the remaining cases
• Standard version of the K-D tree
• Beginning with a complete set of N points in Rk
• Data space is split on the dimension i
• which the data exhibits the greatest variance
• A cut is made at the median value m of the data
  in that dimension
• equal number of points fall to one side or the
  other
• An internal node is created to store i and m
• Process iterates with both halves of the data
• This creates a balanced binary tree with depth d
  log2 N

9
NN search using K-D tree
Nearest ? dist-sqd ? NN(c, x)
a
b
c
nearer
further
e
f
b
c
f
d
g
a
e
g
d
Nearer e Further b NN (e, x)
10
NN search using K-D tree
Nearest ? dist-sqd ? NN(e, x)
a
b
c
e
f
b
c
further
nearer
f
d
g
a
e
g
d
Nearer g Further d NN (g, x)
11
NN search using K-D tree
Nearest ? dist-sqd ? NN(g, x)
a
b
c
e
f
b
c
f
d
g
a
r
e
g
d
Nearest g dist-sqd r
12
NN search using K-D tree
Nearest g dist-sqd r NN(e, x)
a
b
c
e
f
b
c
f
d
a
g
r
e
g
d
Check d2(e,x) gt r No need to update
13
NN search using K-D tree
Nearest g dist-sqd r NN(e, x)
a
b
c
e
f
b
c
f
d
g
a
r
e
g
d
Check further of e find p d (p,x) gt r No need
to update
14
NN search using K-D tree
Nearest g dist-sqd r NN(c, x)
a
b
c
e
f
b
c
f
d
g
a
r
e
g
d
Check d2(c,x) gt r No need to update
15
NN search using K-D tree
Nearest g dist-sqd r NN(c, x)
a
b
c
e
f
b
c
f
d
g
a
r
e
g
Check further of c find p d(p,x) lt r !! NN
(b,x)
d
16
NN search using K-D tree
Nearest g dist-sqd r NN(b, x)
a
b
c
e
f
b
c
f
d
g
a
r
e
g
d
Nearer f Further g NN (f,x)
17
NN search using K-D tree
Nearest g dist-sqd r NN(f, x)
a
b
c
e
f
b
r
c
f
d
g
a
e
g
d
r d2 (f,x) lt r dist-sqd ? r nearest ?f
18
NN search using K-D tree
Nearest f dist-sqd r NN(b, x)
a
b
c
e
f
b
r
c
f
d
g
a
e
g
Check d(b,x) lt r No need to update
d
19
NN search using K-D tree
Nearest f dist-sqd r NN(b, x)
a
b
c
e
f
b
r
c
f
d
g
a
e
g
Check further of b find p d(p,x) gt r No need
to update
d
20
NN search using K-D tree
Nearest f dist-sqd r NN(c, x)
a
b
c
e
f
b
r
c
f
d
g
a
e
g
d
21
Search Process BBF Algorithm

• Set
• v query vector
• Q priority queue ordered by distance to v
  (initially void)
• r initially is the root of T
• vFIRST initially not defined and with an
  infinite distance to v
• ncomp number of comparisons, initially zero.
• While (!finish)
• Make a search for v in T from r gt arrive to a
  leaf c
• Add all the directions not taken during the
  search to Q in an ordered way (each division node
  in the path gives one not-taken direction)
• If c is more near to v than vFIRST, then vFIRSTc
• Make r the first node in Q (the more near to
  v), ncomp
• If distance(r,v) gt distance(vFIRST,v), finish1
• If ncomp gt ncompMAX, finish1

22
BBF search example
Requested vector
a1gt2
20,8
a2gt3
a2gt7
a1gt6
1,3
2,7
20,7
5,1500
9,1000
Queue
23
BBF search example
Requested vector
a1gt2
20,8
a2gt3
a2gt7
CMIN
a1gt6
1,3
2,7
20,7
Distance from best-in-queue is NOT lesser than
distance from cMIN Finish
We arrived to a leaf Store nearest leaf in
CMIN
5,1500
9,1000
Queue
14
24
Experimental result

• Compare the performance of SIFT and GDOH by image
  matching experiments and an image retrieval
  application
• Dataset for image matching experiments
• contains test images of various transformation
  types
• Dataset for image retrieval experiment
• includes 30 images of 10 household items

25
Image matching experiments

• Target images are rotated by 55 degree and scaled
  by 1.6
• Target images are rotated by 65 degree and scaled
  by 4

26
Image matching experiments

• Target images are distorted to simulate a 12
  degree viewpoint change
• Intensity of target images is reduced 20

27
Image matching experiments

• GDOH outperforms SIFT slightly
• GDOH can performs comparatively with SIFT over
  various transformation types of images
• Table lists the comparison result of average
  matching time of SIFT and GDOH, respectively
• GDOH is significantly faster than SIFT in the
  image matching stage
• GDOH requires about 63 of the time of SIFT to do
  65 pairs of image matching

28
Image retrieval experiments

• We first extract the descriptors of each image in
  the image dataset
• Then we find matches between every pair of images
• Matches if the distance ratio of the nearest
  neighbor to the second-nearest neighbor is less
  than a threshold
• Similarity measure
• Number of matched feature vector as a similarity
  measure between images
• For each image, the top 2 images with most
  matched number are returned

29
Conslusion

• GDOH
• Is created based on the gradient distance and
  orientation histogram
• Can be invariant to image rotation, scale,
  illumination and partial viewpoint changes
• Distinctive and robust as SIFT descriptor.
• The dimensionality of GDOH is much lower than
  that of SIFT, which can result in high efficiency
  in image matching and image retrieval
  application.