Evaluation of Interest Point Detectors

1
Evaluation of Interest Point Detectors
C. Schmid, R. Mohr, and C. Bauckhage, IJCV00
Presenter Qi Li
Supplemental paper Indexing based on scale
invariant interest points. K. Mikolajczyk and C.
Schmid, ICCV01
2
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

3
Introduction

• Detecting interest points has wide applications,
  such as image matching, object recognition, 3D
  reconstruction, etc.
• Existing evaluation criteria include ground-truth
  verification, localization accuracy, etc.

4
Introduction

• A new criterion in evaluating the performance of
  interest point detector Repeatability

• The repeatability involves the certain degree of
  localization accuracy
• The repeatability criterion proposed in the paper
  is valid for planar scenes

5
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

6
Interest point detectors

• Harris
• Standard version
• Improved version
• Cottier
• Forstner
• Horaud
• Heitger

7
Interest point detectors

• Harris
• Standard version
• Improved version
• Cottier
• Forstner
• Horaud
• Heitger

Based on auto-correlation matrix
8
Three basic steps for auto-correlation based
detectors

• Step 1. Constructing the auto-correlation matrix
• Step 2. Strength assignment
• Step 3. Non-maximum suppression

9
Auto-correlation matrix

• Auto-correlation matrix, also called (weighted)
  gradient covariance matrix

10
Auto-correlation matrix Derivation

• Auto-correlation function
• The auto-correlation function at an interest
  point has high value for all shift directions
• A basic tool
• First-order Taylor expansion

11
Auto-correlation matrix Derivation

• Auto-correlation function
• The auto-correlation function at an interest
  point has high value for all shift directions
• A basic tool
• First-order Taylor expansion

12
Auto-correlation matrix Derivation

• Auto-correlation function
• The auto-correlation function at an interest
  point has high value for all shift directions
• A basic tool
• First-order Taylor expansion

13
Auto-correlation matrix Derivation
14
Strength assignment

• Strength is determined by eigen-structure of
  auto-correlation matrix
• For example,
• Or the min of eigenvalues

15
Non-maximum suppression

• Only the image points that have the local maximal
  strength become candidates of interest points
• Local maxima is defined by
  

16
Harris detectors Basic parameters

• In construction of auto-correlation matrix
• In strength assignment
• In non-maximum suppression
• Threshold in selecting the first 1 from the set
  of candidates

17
Harris detectors Two versions

• The first derivative is computed using different
  filters
• Standard version
• The filter is
• Improved version
• The filter is

18
Cottier detector

• Basic idea CannyHarris
• First, apply Canny edge detector to extract
  contour points
• An edge detector usually involves a non-maximum
  suppression
• Second, apply Harris to the contour points

19
Forstner detector

• Basic idea It is based on auto-correlation
  matrix, but its interest assignment scheme is
  different from the ones in Harris
• First, compute auto-correlation matrix
• Derivative using
• Weight using
• Second, classify pixels into region or non-region
  using the trace of the matrix
• Third, classify non-region pixels into contour or
  interest points using the ratio of the
  eigenvalues and a fixed threshold, 0.3.

20
Horaud detector

• Basic idea Cannyintersection
• First, extract contour chains, and fit lines
  using the contour points
• Interest points are the intersections between
  neighboring lines

21
Heitger detector

• Basic idea Gabor-like filter
• First, the image is convolved with even and odd
  symmetrical orientation-select filters, which are
  computed for 6 orientations
• Second, for each orientation, an energy map is
  computed by combining even and odd filter outputs
• Third, each energy map is differentiated to
  compute interest strength
• Finally, non-maximum suppression is applied

22
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

23
Repeatability criterion

• Homography/perspective transformation

24
Repeatability criterion

• Definition of repeatability rate

25
Repeatability criterion

• Definition of repeatability rate

26
Repeatability criterion

• Definition of repeatability rate

27
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

28
Evaluation

• Homography estimation
• Unconstrained conditions
• Rotation
• Scale
• Illumination
• View
• Image noise

29
Homography estimation

• Accurate estimation of homography
• Project black dots onto the scene
• Dots are extracted precisely by fitting a
  template, and centers are used to compute the
  homography

30
Homography estimation

• Projection mechanism

31
Harris standard vs. improved
32
Repeatability across rotation angles

• Experiment setup
• Rotating the camera around its optical axis

38-degree
116-degree
0-degree
33
Repeatability across rotation angles

• Experiment setup
• Rotating the camera around its optical axis

38-degree
116-degree
0-degree
34
Repeatability across rotation angles
35
Repeatability across rotation angles
36
Repeatability across scales

• Experiment setup
• Varying the focal length of the camera

Scale factor1.5
Scale factor 4.1
Scale factor1
37
Repeatability across scales

• Experiment setup
• Varying the focal length of the camera

Scale factor1.5
Scale factor 4.1
Scale factor1
38
Repeatability across scales
39
Repeatability across scales
40
Repeatability across illuminations

• Uniform illuminations
• Changing the camera aperture, which is quantified
  by relative greyvalue, i.e., the ratio of mean
  greyvalue of an image to that of the reference
  image which has medium greyvalue

Relative greyvalue 1
Relative greyvalue 1.7
Relative greyvalue 0.6
41
Repeatability across illuminations

• Uniform illuminations

42
Repeatability across illuminations

• Complex variation of illumination
• Moving the light source in an arc from -45-degree
  to 45-degree

Frontal light source
Leftmost light source
Rightmost light source
43
Repeatability across illuminations

• Complex variation of illumination

44
Repeatability across views

• Experiment setup
• Moving the camera in an arc around the scene,
  from -50-degree to 50-degree, and the different
  viewpoints are approximately regularly spaced

Rightmost camera
Frontal camera
Leftmost camera
45
Repeatability across views
46
Repeatability across camera noise

• Experiment setup
• A static scene is recorded several times

47
Summary
scale
View
rotation
Uniform illum
Complex illum
Noise
48
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

49
Conclusion

• Harris detector (improved version) shows its
  superiority to other detectors, under the
  evaluation criterion of repeatability rate
• Harris detector is
• very robust to image rotation and image noise
• fairly robust to certain illumination change
• not very robust to view changes
• very poor to scaling

50
Additional comments

• Auto-correlation matrix is a good tool for
  interest point detections and the calculation of
  derivatives is important
• 0.5-repeatability rate is around 90 (under
  camera noise) may be not very satisfactory in
  certain applications, e.g., computation of
  epipolar geometry that are sensitive to 1-pixel
  localization error
• Higher complexity of a detection algorithm tends
  to degrade its performance
• Evaluation of repeatability on planar scene is a
  well-defined (ground-truth is reliable and easy
  to get), but the evaluation of repeatability of
  3D scene is highly desirable

51
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

52
Motivation of Harris-Laplacian

• We have observed that Harris is poor in attaining
  maxima in scale space
• There are two more observations
• First. Normalized Harris rarely attains maxima in
  scale space (Mikolajczyk and Schmid ICCV01)
• Second. Normalized Laplacian finds the highest
  percentage of correct maxima, under significant
  change of image resolution (Mikolajczyk and
  Schmid ICCV01)

53
Harris-Laplacian

• Notations
• Laplacian
• Normalized scale Laplacian

54
Harris-Laplacian

• Notations
• Laplacian
• Normalized scale Laplacian
• Discrete scales
• Given , and

55
Harris-Laplacian

• Basic idea
• First, Harris is used to detect local maxima in
  each scale. (These local maxima are the input of
  the next step.)

Scale n-1
Scale n
Scale n1
56
Harris-Laplacian

• Basic idea
• First, Harris is used to detect local maxima in
  each scale. (These local maxima are the input of
  the next step.)
• Second, normalized Laplacian is used to detect
  the maxima across different scales

57
Harris-Laplacian

• Implementation
• Step 1. for each n, detect local maxima using
  Harris
• Step 2. across consecutive scales, detect scale
  maxima using normalized Laplacian

58
Outline

• Introduction
• Interest point detectors
• Repeatability criterion
• Evaluation
• Conclusion
• Extended work Harris-Laplacian
• Potential directions

59
Potential directions

• Two doable directions
• Improving localization accuracy using interest
  point detector without involvement of non-maximum
  suppression
• Higher-order auto-correlation matrix via
  higher-order Taylor expansion
• Three challenging directions
• Improving the repeatability across uniform
  illumination
• Improving the repeatability across views
• Repeatability on 3D scene using three-view
  epipolar geometry