Image Mining Intricacies and Innovations

1
Image Mining Intricacies and Innovations

SUNDARAM R M D M.Tech Computer Vision Image
Processing REcognition And Learning Lab Amrita
School of Engineering
2
Agenda

• Introduction
• 5-min Recap
• Problem Definition CBIR
• Need for CBIR
• Flow of work
• Feature Extraction A brief view
• Promising directions and open issues
• Performance Evaluation
• Conclusion

3
Introduction
4
5-min Recap

• Why is Image Information Retrieval more
  important?
• A Picture is worth thousand words
• Alternative form of communication
• Not everything can be described in text
• Popular medium of information on the Internet
• Well known search engines like Google fails to
  retrieve images based on their contents.

5
5-min Recap
6
5-min Recap
7
5-min Recap
8
Problem Definition

• Given a query image, with single / multiple
  object present in it mission of this work is to
  retrieve similar kind of images from the database
  based on the features extracted from the query
  image.
• - Content based Image Retrieval (CBIR)

9
Need for CBIR

• Early work on image retrieval can be tracked back
  to the late 1970s. Techniques used were not
  generally based on visual features but on the
  textual annotation of the images.
• Through text descriptions, images can be
  organized by topical or semantical hierarchies to
  facilitate easy navigation and browsing based on
  standard Boolean queries.
• But automatically generating texts for a wide
  spectrum of images is not feasible. Also,
  annotating images manually is a cumbersome and
  expensive task for large image databases, and is
  often subjective, context-sensitive and
  incomplete.
• Hence it is widely recognized that a more
  efficient and intuitive way to represent and
  index visual information is needed. This gave
  birth to a new concept called CBIR.

10
Applications of CBIR

• Crime prevention Automatic face recognition
  systems, used by police forces.
• Security Check Finger print or retina scanning
  for access privileges.
• Medical Diagnosis Using CBIR in a medical
  database of medical images to aid diagnosis by
  identifying similar past cases.
• Intellectual Property Trademark image
  registration, where a new candidate mark is
  compared with existing marks to ensure no risk of
  confusing property ownership.

11
Flow of Work

• Flow Chart

12
Feature Extraction 1

Extract Features (Primitives)
Similarity Measure
Matched Results
Query Image
Image Database
Relevance Feedback Algorithm
Features Database
13
Feature Extraction 2 -- Color

• Color is the most extensively used visual content
  for image retrieval.
• Its three-dimensional values make its
  discrimination potentiality superior to the
  single dimensional gray values of images.
• The Various parameters that can be extracted from
  color information are as follows
• 1) Color Moments
• 2) Color Histogram

14
Feature Extraction 3 -- Color

• Color is the most extensively used visual
  content for image retrieval
• Finding the Color moments
• 1) µi (1/N) S fij Mean
• 2) si (1/N) S (fij - µi )2 ½ Variance
• 3) Si (1/N) S (fij - µi )3 1/3 Skew
  ness
• Where fij is the value of the ith color
  component of the image pixel j, and N is the
  number of pixels in the image.
• Through Color Histogram
• Since any pixel in the image can be described by
  three components in a certain color space, a
  histogram, i.e., the distribution of the number
  of pixels for each quantized bin, can be defined
  for each component. Clearly, the more bins a
  color histogram contains, the more discrimination
  power it has.

15
Feature Extraction 4 -- Color
16
Why Fuzzy Color Histogram(FCH)?

• Existing Algorithm
• Given a color space containing n color bins, the
  color histogram of image containing N pixels
  is represented as, H(I) h1, h2, h3, .hn
  where hi Ni / N is the probability of a pixel
  in the image belonging to the ith color bin, and
  Ni is the total number of pixels in the ith color
  bin.
• Fuzzy Color Histogram
• Instead of using the normal probability values,
  we consider each of the N pixels in the image I
  being related to all the n color bins via Fuzzy
  set membership functions such that the degree of
  belongingness or association of the jth pixel to
  the ith color bin is determined by distributing
  the membership value of the jth pixel,µij to the
  ith color bin.

17
Feature Extraction 5 -- Color
Context Sensitivity
18
Feature Extraction 6 -- Texture

• Texture is another important property of images.
  Basically, texture representation methods can be
  classified into two categories structural and
  statistical.
• Structural methods, including
  morphological operator and adjacency graph,
  describe texture by identifying structural
  primitives and their placement rules. They tend
  to be most effective when applied to textures
  that are very regular.
• Statistical methods include, for example,
  in areas with smooth texture, the range of values
  in the neighborhood around a pixel will be a
  small value in areas of rough texture, the range
  will be larger. Similarly, calculating the
  standard deviation of pixels in a neighborhood
  can indicate the degree of variability of pixel
  values in that region.

19
Feature Extraction 7 -- Texture
Grey level co-occurrence matrix
20
Feature Extraction 8 -- Texture

• Tamura Features Three main features including
  Coarseness, contrast and directionality are
  extracted.
• Coarseness is the measure of granularity of an
  image, or average size of regions that have the
  same intensity.
• Contrast is the measure of brightness of the
  texture pattern. Therefore, the bigger the blocks
  that makes up the image, the higher the contrast.
• Directionality is the measure of directions of
  the gray values within the image.

21
Why Tamura Features ? 1

• Following are the features calculated from the
  normalized co-occurrence matrix P(i,j)

22
Why Tamura Features ? 2

• An image will contain textures at several scales
  coarseness aims to identify the largest size at
  which a texture exists, even where a smaller
  micro texture exists.
• Here we first take averages at every point over
  neighborhoods, the linear size of which are
  powers of 2. The average over the neighborhood of
  size 2k 2k at the point (x,y) is

23
Why Tamura Features ? 3

• Then at each point one takes differences between
  pairs of averages corresponding to
  non-overlapping neighborhoods on opposite sides
  of the point in both horizontal vertical
  orientations. In the horizontal case this is
• At each point, one then picks the best size which
  gives the highest output value, where k maximizes
  E in either direction. The Coarseness measure is
  then the average of Sopt (x,y) 2kopt over the
  picture.

24
Why Tamura Features ? 4

• Contrast aims to capture the dynamic range of
  gray levels in an image, together with the
  polarization of the distribution of black and
  white.
• Here µ4 is the fourth moment about the mean and
  s2 is the variance.
• Directionality is a global property over a
  region. Detect the edges in an image. At each
  pixel the angle and magnitude are calculated. A
  histogram of edge probabilities is then built up
  by counting all points with magnitudes greater
  than a threshold and quantizing by the edge
  angle. The histogram will reflect the degree of
  directionality.

25
Feature Extraction 9 -- Texture

26
Feature Extraction 10 -- Shape

• Compared with Color and Texture features, shape
  features are usually described after images have
  been segmented into regions or objects.
• Since robust and accurate image
  segmentation is difficult to achieve, the use of
  shape features for image retrieval has been
  limited to special applications where objects or
  regions are readily available.
• The state-of-art methods for shape
  description can be categorized into either
  boundary-based polygonal approximation, finite
  element models and Fourier-based shape
  descriptors or region-based methods (statistical
  moments.)
• A good shape representation feature for
  an object should be invariant to translation,
  rotation and scaling. Phase Congruency is one
  such feature.

27
Feature Extraction 11 -- Shape

• Algorithm phase_congruency
• Input
• Amplitude An(x)
• Phase angle ?n(x)
• Weighted mean phase angle ??(x)
• Weights for frequency spread w(x)
• To avoid division by zero, add a small ?
• Compute
• PC1 (x) ?E (x)? / ?n An(x)
• Where?E (x)? ?n An (cos (?(x) - ??(x))
• To produce more localized response,
• PC1 (x) w (x)?E (x) -T? / ?n An(x) ?
• Compute distance measurement between query image
  and image in the database.
• Output
• Retrieve similar images.

28
Similarity / Distance Measure

• Instead of exact matching, content-based image
  retrieval calculates visual similarities between
  a query image and images in a database.
  Accordingly, the retrieval result is not a single
  image but a list of images ranked by their
  similarities with the query image.
• If each dimension of image feature vector is
  independent of each other and is of equal
  importance, the Minkowski-form distance is
  appropriate for calculating the distance between
  two images. This distance is defined as
• D( i, j ) S fi (I) - fi (J) p 1/p

29
Highlights and Novelty in the work

• Using Rodriguez formula to normalize the images
  before taking the color Histogram.
• Analysis of Statistical methods, including
  Fourier power spectra, co-occurrence matrices,
  Tamura feature, and multi-resolution filtering
  techniques such as Gabor and wavelet transform,
  for characterizing texture.
• Phase Congruency measurement for Shape
  Information.

30
Promising Directions and Open Issues

• Relevance feedback is proposed as a technique for
  overcoming many of the problems faced by fully
  automatic systems by allowing the user to
  interact with the computer to improve retrieval
  performance.
• More interestingly, the semantic gap is need to
  be bridged. Semantic gap here refers to the
  large disparity between the low-level features or
  content descriptors that can be computed
  automatically by current machines and algorithms,
  and the richness and subjectivity of semantics in
  user queries and high-level human interpretations
  of audiovisual media

31
Performance Evaluation

• The proposed technique is tested on two types of
  data sets, first one consisting of different
  animals and the second dataset consisting of
  birds, flowers and buildings.
• The retrieval accuracy is found to be 96.4 and
  92.2 for a database size of 55 each.

32
Database (Partial Set)
33
Results 1

• QUERY IMAGE

34
Results 2 -- Color
35
Results 3 Color Texture
36
References

• M. Flickner, H. Sawhney, W. Niblack, and P.
  Yanker, Querying by image and video content The
  QBIC system, IEEE Trans. Comput., vol. 28, pp.
  2332, 1995.
• James Z. Wang, Jia Li and Gio, SIMPLIcity
  Semantics sensitive Integrated Matching for
  picture Libraries, IEEE Transactions on pattern
  Analysis and Machine Intelligence,
  Vol.23,no.9,pp.947-963,2001.
• Smeulders, Senior Member, IEEE, Santini, Member,
  IEEE, Content-Based Image Retrieval at the End
  of the Early Years, IEEE transactions on pattern
  analysis and machine intelligence, vol. 22, no.
  12, december 2000
• K.Satya Sai Prakash, RMD. Sundaram, Shape
  Information from Phase Congruency and its
  application in Content based Image Retrieval,
  Proceedings of the 3rd Workshop on Computer
  Vision, Graphics and Image Processing WCVGIP
  2006, Hyderabad PP.88-93, 2006.

37
References

• J. van de Weijer and J. M. Geusebroek. Color edge
  and corner detection by photometric
  quasi-invariants. IEEE Trans. Pattern Anal.
  Machine Intel. 27(4)625-630, 2005.
• Barbeau Jerome, Vignes-Lebbe Regine, and Stamon
  Georges, A Signature based on Delaney Graph and
  Co-occurrence Matrix, Laboratories Informatique
  et Systematique, University of Paris, Paris,
  France, July 2002, Found at http//www.math-info.
  univ-paris5.fr/sip-lab/barbeque/barbeque.pdf

38
Discussions????


THANK YOU sundaram.rmd_at_gmail.com