A Methodology for Comparing Edge Detectors

1
A Methodology for Comparing Edge Detectors

• Speaker Junwen WU
• Course ECE285A
• Date 01/16/2002

2
Achievement
Motivation

• A lot of edge detectors have been proposed, yet
  no solid basis to evaluate the performance.

All evaluating metricsleave something to be
desired

• Describe a new experimental framework for making
  quantitative comparisons using subjective ratings
  made by people

3
Four Edge Detectors to Be Compared

• Canny Edge Detector Find the first derivative of
  the Gaussian smoothed image
• Sobel Edge Detector(Supplemented with
  double-thresholding) Using 3 X 3 neighbor-hood
  mask operator to calculate the gradient
• Nalwa-Binford Edge Detector Surface-fitting
  approach to do edge detection
• Sarkar-Boyer Edge Detector Zero-crossing
  detection on a smoothed image to find the edge

4
Experiment Design

• Experiment 1 Edge Detector Parameter Settings
• Purpose

Sufficient to use a fixed set of parameter values
for a given edge detector across all test images
YES
NO
5
Experiment Design(Contd.)

• Experiment 2 Compare Rating of Edge Detectors
• Purpose Make a direct comparison between edge
  detectors
• Complexity There is a statistically significant
  interaction of image and parameter set
• Method Using two scenarios for each edge
  detector
• 1. Current Scenario exploit the best
  fixed
• parameter set for each edge detector across all
  test images
• 2. Ideal Scenario for every edge
  detector, it exploit the best parameter set for
  each individual image.

6
Analysis Method

• ANOVA ANalysis Of Variance
• It is used to uncover the main and interaction
  effects of factors on an interval dependent
  variables.
• How to interpret an ANOVA table
• Influence factors Influence sources(image,
  detector, interaction etc.)
• Degree of freedom Number of free available
  observations (DF)
• Sum of square Each sources variation from
  the ground mean (SS)
• values reflect the relative magnitude of
  the different influence sources
• F-values Significance test statistics
• p-value Estimated probability to indicate if
  the variation is caused by chance

7
Examples of Original Test Images and
Questionnaires
8
Experiment 1 Details

• Experiment Implement
• For each parameter of every edge detector,
  24 plausible parameters are selected
• Eight original test images
• Eight judges
• Eight rating levels
• Data set 8(Judges) X 8(Images) X 4(Edge
  Detectors) X (612 parameter sets)
• Example of ANOVA table result

9
(No Transcript)
10
Experiment 1 Details(Contd.)

• Results
• The ratings of the judges are consistent
  (Bases)
• The ratings of an edge detector vary with the
  image
• The parameters of an edge detector have some
  influence on the ratings more or less
• The right parameter set varies with images

11
Experiment 2 Details

• Two set of parameters are used Fixed versus
  adaptive
• Results
• Detectors performance is deteriorated if
  fixed parameter set is used across all images
  instead of adaptive one
• The final rating of the edge detectors varies
  with images
• Final rating Canny, (Nalwa-Binford,
  Sarkar-Boyer), Sobel

12
Discussion and Conclusion

• There are statistically significant differences
  between the ratings of the edge detectors
• The optimal parameter settings of an edge
  detector are strongly depend on the image
• The relative performance of the edge detectors
  varied statistically significantly across the
  images