Relevance Feedback based on Parameter Estimation of Target Distribution

1
Relevance Feedback based on Parameter Estimation
of Target Distribution

• K. C. Sia and Irwin King
• Department of Computer Science Engineering
• The Chinese University of Hong Kong
• 15 May
• IJCNN 2002

2
Agenda

• Introduction to content based image retrieval
  (CBIR) and relevance feedback (RF)
• Former approaches
• Tackling the problem
• Parameter estimation of target distribution
• Experiments
• Future works and conclusion

3
Content Based Image Retrieval

• How to represent an image?
• Feature extraction
• Colour histogram (RGB)
• Co-occurrence matrix texture analysis
• Shape representation
• Feature vector
• Map images to points in hyper-space
• Similarity is based on distance measure

4
Feature Extraction Model
5
Relevance Feedback

• Relevance feedback
• Architecture to capture users target of search
• Learning process
• Two steps
• Feedback how to learn from the users relevance
  feedback
• Display how to select the next set of documents
  and present to user

6
(No Transcript)
7
Former Approaches

• Multimedia Analysis and Retrieval System (MARS)
• Yong Rui et al. Relevance feedback A powerful
  tool for interactive content-based image
  retrieval. - 1998
• Using weight to capture users preference
• Pic-Hunter
• Ingemar J. Cox et al. The Bayesian image
  retrieval system, pichunter, theory,
  implementation, and psychophysical experiments. -
  2000
• Images are associated with a probability being
  the users target
• Bayesian learning

8
Comparison
9
The Model

• Feature Extraction
• I - raw image data
• ? - set of feature extraction method
• f - feature extraction operation
• Images ? data point in hyper-space (Rd)
• Problem scope is narrowed down to a particular
  feature

10
Feedback
11
Inconsistence in Feedback

• User tells lies
• Too many false positive or false negative
• Conflict of feedback in each iteration by
  careless mistake

12
Resolving Conflicts

• How to deal with inconsistent user feedback?
• Maintain a relevance measure for each data points
• Relevance measure 0 counted as relevant and use
  in estimation

13
Estimating Target Distribution

• Users target is a cluster
• Assume it follows a Gaussian distribution
• Model a distribution that fits the relevant data
  points
• Based on the parameterof distribution,
  systemlearns what user wants

14
Expectation Maximization

• Fitting a Gaussian distribution function using
  feedback data points
• By expectation maximization
• Distribution represent users target
• Expectation function match the display model

15
Updating Parameters

• Estimated mean is the average
• Estimated variance by differentiation
• Iterative approach

16
Display
17
Maximum Entropy Display

• Why maximum entropy display?
• Reason fully utilize information contained in
  user feedback to reduce number of feedback
  iteration
• Result near boundary images will be selected to
  fine tune parameters

18
Maximum Entropy Display

• How to simulate maximumentropy display in
  ourmodel?
• Data points 1.18 ? away from ? are selected
• Why 1.18?
• 2P(?1.18?)P(?)

19
Experiment

• Synthetic data generated by Matlab
• Mixture of Gaussians
• Class label of data points shown for reference to
  give feedback
• Dose it works and works better?

20
Convergence

• Is the estimated parameter (mean and variance)
  converge to the actual parameter of target
  distribution?
• Is the maximum entropy display correctly done?

21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
Performance

• Compares to Ruis intra-weight updating model
• Nearest neighbour search performed after several
  feedbacks (6-7 iterations)
• Data points outside 2 ? are discarded in our
  algorithm
• Precision-Recall graph

25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Future Works

• Modification to learn from information contained
  in non-relevant set
• To capture correlation in different features
• Apply in CBIR system for performance measurement

32
Conclusion

• Proposed an approach to interpret the feedback
  information from user and learn his target of
  search
• Compares our approach with Ruis intra-weight
  updating method

33
END

• Presentation file available at http//www.cse.cuhk
  .edu.hk/kcsia/research/