Content Based Image Retrieval

1
Content Based Image Retrieval

• Miguel Arevalillo-Herráez

2
Contents

• Introduction
• Information retrieval
• Image retrieval
• CBIR
• Approaches
• Combining similarity measures
• Full CBIR systems
• Possible extensions to 3D
• Results and Conclusions

3
Concepts

• Information retrieval
• Objects are documents
• Concept of a query
• Image retrieval
• Objects are images
• Concept of a query
• Content Based Image retrieval

4
Common setup for CBIR
5
The method

• How do we judge how similar two images are?

6
The method

• How do we judge how similar two images are?
• - feature vectors

7
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?

8
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  

9
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?

10
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?
• Normalization and combination

11
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?
• Normalization and combination
• How are multiple selections combined?

12
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?
• Normalization and combination
• How are multiple selections combined?
• Multiple selection approaches

13
The method

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?
• Normalization and combination
• How are multiple selections combined?
• Multiple selection approaches

14
Normalization and Combination Rules

• Classical normalization rules
• Gaussian
• Linear
• Classical combination rules
• Sum
• Product
• Linear combination

15
Probabilistic Approach

• For each distance function we estimate the
  probability that the user considers that two
  images are similar, for every possible distance
  value p(similar di)
• This is performed from a training set

16
Probabilistic Approach

• For each distance function we estimate the
  probability that the user considers that two
  images are similar, for every possible distance
  value p(similar di)
• This is performed from a training set
• p(similar d1, d2, d3,,dn) ?
• p(similar d1) x p(similar d2) x p(similar
  d3) x x p(similar dn)

17
Handling Multiple Selections

• Classical Approaches
• Query point movement and axis re-weighting
• Support Vector Machines
• Probabilistic and Regression Approaches
• Other interesting approaches
• SOM based
• Nearest neighbour

18
Fuzzy Approach - Concepts

• Need to deal with uncertainty of the data
• Classical set
• Elements are or are not in the set
• Fuzzy set
• Elements have a degree of membership to the set

19
Fuzzy approach

• Assumes an underlying search model
• Any image of interest should be perceptually
  similar to each of the pictures in the set
  Positive in at least kpos characteristics.
• Any image of interest should be perceptually
  different from each of the pictures in the set
  Negative in at least kneg characteristics.

20
(No Transcript)
21
Fuzzy approach

• Every iteration the user is more exigent
• Kpos and Kneg vary at each iteration

22
Fuzzy Approach
23
Genetic Approach

• An evolutionary algorithm attempts to solve a
  problem applying Darwins basic principles of
  evolution on a population of trial solutions to a
  problem, called individuals.

24
Genetic Approach
25
Genetic Approach

• Key issues
• Existence of fitness function
• Relevance feedback defines population and fitness
• Maintaining consistency
• How do we judge next generation?

26
Genetic Approach
27
Genetic Approach
28
Possible extensions to 3D

• How do we judge how similar two images are?
• - feature vectors
• How do we compare these vectors?
• distance funcions defined over the feature space.
  
• How are these distances combined to yield a
  composite similarity value?
• Normalization and combination
• How are multiple selections combined?
• Multiple selection approaches

29
Results and Conclusions

• Introduction to the CBIR problem
• Feature extraction
• Definition of distance funcions normalization and
  combination
• Handling multiple selections
• Posible extensions to 3D