Relevance Feedback Content-Based Image Retrieval Using Query Distribution Estimation Based on Maximum Entropy Principle

1
Relevance Feedback Content-Based Image Retrieval
Using Query Distribution Estimation Based on
Maximum Entropy Principle

• Irwin King and Zhong Jin
• Nov. 2001

2
1. Introduction

• Content-Based Image retrieval (CBIR)
• The selection of images from a
  collection via primitive visual features
  representing color, shape, and texture extracted
  from images themselves.

Successful CBIR systems require the integration
of various techniques in the fields of

• Pattern Recognition (PR)
• Digital Image Processing (DIP)
• Information Retrieval (IR)

3
1. Introduction (Cont d)

• Relevance feedback (RF)
• an iterative and interactive process
  for query reformulation based on user's feedback.

RF techniques include mainly

• query moving technique
• similarity function re-weighting technique

4
1. Introduction (Cont d)

• Problems
• Since the retrievals under the commonly
  used nearest-neighbor rule cannot reflect the
  query distribution function properly , most of
  relevance feedback techniques may fail under the
  following assumption

• The number of relevant retrievals is small
• The number of iterations is required to be small

5
2. Background Review

• RF technique in CBIR system can be regarded as a
  form of two-stage automatic learning for the
  unknown query distribution function

• Estimate the query distribution function by using
  the Expectation-Maximization (EM) algorithm or by
  the classical statistics.
• Generate the inquiries to be returned to the
  user, where the nearest-neighbor rule is commonly
  used.

6
2. Background Review (Cont d)

• Limitation of estimation theories
• EM has its limitations in CBIR Because of a small
  number of labeled data in RF.
• Only relevant informaton can be utilized by
  classical statistical theory.

7
2. Background Review (Cont d)

• Limitation of the nearest-neighbor rule
• the retrievals generated cannot wholly reflect
  the Query Distribution Function (QDF) because the
  underlying QDF may not be isotropic in nature.

Note QDF is the statistical distribution
function deformed by all the images similar to
the given query image in high dimensional feature
space.
8
2. Background Review (Cont d)

• Shannons Entropy

Maximum Entropy Principle (MEP) To
obtain estimations by determining a probability
distribution associated with a random variable
over a discrete space which has the greatest
entropy subject to constraints on the
expectations of a given set of functions of the
variable.
The Maximum Entropy (MAXENT) solution with no
bias (or constraints) is
9
2. Background Review (Cont d)

• Some work on IR by MEP
• In the early 80's, Cooper et al. made a strong
  case for applying the maximum entropy approach to
  the problems of information retrieval.
• Kantor extended the analysis of the MEP in the
  context of information retrieval.
• Recently, Greiff and Ponte took a fresh look at
  modeling approaches to information retrieval and
  analyzed classical probabilistic IR models in
  light of the MEP.

10
3. Proposed Framework

• Our novel framework for image retrieval includes
  the following stages
• Estimation stage -- Estimate the query mean and
  the query covariance matrix by using accumulative
  relevance retrieval information and irrelevant
  retrieval information.
• Generation stage -- Generate a set of inquiries
  for relevance selection based on MEP.

11
3. Proposed Framework (Cont d)

• Estimation stage

When the number T of relevant retrieval is less
than the dimension M of the feature space, it is
assumed that
When T1, an estimation can be given by an
equal-probability constrain
12
3. Proposed Framework (Cont d)

• Generation stage

• For K number of retrievals, K1 points can be
  determined according to the following
  equal-probability conditions

• all similar images in the database can be divided
  in the following K subsets

where
13
4. Experiments and Analysis

• Database
• There are 1,400 trademark images with 128128.
  Here are ten samples

14
4. Experiments and Analysis (Cont d)

• Here are 10 deformation transformations

15
4. Experiments and Analysis (Cont d)

• 100 Test Images

16
4. Experiments and Analysis (Cont d)

• Feature Extraction 7 dimensional invariant
  moment

17
4. Experiments and Analysis (Cont d)

• Experimental Aim
• to evaluate the efficiency of the
  proposed generation stage, Generation MAXENT.

The retrieval performance is measured using the
following Average Retrieval Precision (ARP)
where K10
18
4. Experiments and Analysis (Cont d)

• In order to compare Generation MAXENT with
  Euclidean distance and Mahalanobis distance, a
  set of three-step experiments are designed as
  follows

Step 1 For a query image, return K retrievals by
Euclidean distance. Step 2 Perform the
estimation stage and return retrievals by
Euclidean distance, Mahalanobis distance, and
Generation MAXENT respectively. Step 3 Perform
the estimation stage and return K retrievals by
using the Mahalanobis distance
19
4. Experiments and Analysis (Cont d)

• Experimental results

Http//www.cse.cuhk.edu.hk/miplab/MAXENT
20
4. Experiments and Analysis (Cont d)

• Result analysis
• According to the ARP's in Step 3 in Table 1, the
  proposed generation stage Generation MAXENT
  outperforms the commonly used Euclidean distance
  and Mahalanobis distance
• The proposed generation stage Generation Based On
  MAXENT aims to retrieve image samples which can
  reflect the query distribution function. This is
  the reason why the ARP of MAXENT in Step 2 in
  Table 1 is lower than those of Euclidean distance
  and Mahalanobis distance.

21
5. Conclusion

• Novel two-stage relevance feedback framework for
  content-based image retrieval based on query
  estimation and the Maximum Entropy Principle is
  shown to be succeful in improving accuracy and
  speed on a trademark image database.

Future work to overcome the difficulty in image
retrieval for high-dimensional features in large
image databases
22
Acknowledgment

• This paper is supported in part by an Earmarked
  Grant from the Hong Kong Research Grants Council
  CUHK4407/99E.