Image Indexing and Retrieval using Moment Invariants

1
Image Indexing and Retrieval using Moment
Invariants

• Imran Ahmad
• School of Computer Science
• University of Windsor Canada

2
Outline

• Introduction
• Shape-based Retrieval
• Image Representation
• Moment Invariants
• Proposed Approach
• Experimental Results
• Conclusions

3
Introduction

• Information Characteristics
• Nature
• Multiple formats
• Complex
• Types
• Image Database
• Time dependent sequence
• Video Database

4
Introduction (contd.)

• Still information
• Dynamic information
• Temporal Evolution of information
• Changes in features and characteristics

5
Introduction (contd.)

• Image contents
• Real world information
• Unique features
• Need retrieval based on contents

6
Image Retrieval

• Exact match
• Similarity-based retrievals
• Color / texture similarity-based retrievals
• Spatial similarity-based retrievals
• Shape similarity-based retrievals

7
Shape-based retrievals

• Model-based Object Recognition Approach
• Models based on global and local features
• Unknown object compared against known ones
• Data-Driven Approach
• An index for known shapes
• Search utilizes such indices

8
Image Representation

• Images can be defined in terms of
• Global features
• Based on overall image composition
• Easier to compute
• Local features
• Based on individual image components
• Incorporate spatial information
• Computationally expensive

9
Shape

• How to define a shape?
• A geometric property of a figure
• Formal definition independent of language
• Described in terms of properties invariant under
  a group of coordinate transformations
• Let is a characteristic function such
  that

For points in the figure Otherwise
10
Shape (contd.)

• Definition Let Y be a group of coordinate
  transformations. The function I is invariant
  w.r.t. Y if
• for all characteristic functions
  and all transformations y e Y
• Definition A shape of a figure is a pair ltI, Ygt,
  where I is invariant under the group of
  coordinate transformations Y.

11
Moments

• Can capture global information about image
• Do not require closed boundaries.
• Regular moments introduced by Hu.
• Invariant to translation, rotation and scaling
• Algebraic moments
• Do not depend on actual values of the
  coefficients
• Central moments
• Equivalent to regular moments of an image that
  has been shifted

12
Moments (contd.)

• Applications
• Image reconstruction
• Shape identification such as aircrafts, etc.
• Shape recognition
• Classifiers

13
Moments (contd.)

• Let
• be the image intensity distribution
  function
• p q is the order of moments (for p, q 0, 1, 2,
  )
• the algebraic moment of functions are given as
• For a digital image of size M x N

14
Moments (contd.)

• For centralized moments, we can write
• with its digital form as

15
Moments (contd.)

• Central moments up to 2nd order are defined as

16
Moments (contd.)

17
Moments (contd.)

• Algebraic moments by Hu

18
Moments (contd.)

• Moment Invariants
• Time complexity in computing MI is directly
  proportional to the number of pixels in the
  silhouette or forming the boundary.
• Let N be the perimeter of the closed boundary
• To calculate 2nd order moments, we need
• 4(N-1) real additions and 3N real
  multiplications
• To calculate 3rd order moments, we need
• 6(N-1) real additions and 12N real
  multiplications

19
Moments (contd.)

• A Sample binary image moment invariants
• F0.259179343138514,
• 0.00801986505055,
• 0.012354456089699,
• 0.00827468547136,
• -0.000000750728194,
• -0.00005777268349,
• 0.00000025369430

20
Clustering

• Main Categories
• Hierarchical methods
• A nested sequence of partitions
• Involves multiple iterations to cluster objects
• Non-hierarchical methods
• Assume desired number of clusters at the
  beginning
• Data is reallocated until a particular clustering
  criteria is optimized.
• Objects in a cluster are more similar to each
  other.

21
Clustering (contd.)

• K-means clustering
• Let a set of N objects in d-dimensional space Rd
• k is an integer
• Determine a set of k points in Rd, called
  centers, so as to minimize the mean squared
  distance from each data point to its nearest
  center.
• Also known as squared-error distortion.

22
K-means Clustering (contd.)

• ALGORITHM K-means clustering
• For pattern vectors P1, P2,, Pm, set first
  k pattern vectors to the initial clusters C1P1,
  C2P2, C3P3,, CkPk, where m gt k
• Assign each pattern vector to the nearest cluster
  
• Compute new cluster means
• If new cluster means old cluster means
• stop,
• else
• go to step 2

23
K-means Clustering (contd.)

• K-means Clustering Tree (KCT)
• The KCT is a hierarchical data structure similar
  to a combination of binary search tree and B
  -tree.
• Data pointers are stored only at the leaf nodes
  of tree.
• Non-leaf nodes that contain weight vectors.
• Non-leaf nodes have links to other nodes.
• Unidirectional links
• Left child nodes with lesser threshold values
• Right child nodes with greater threshold values.

24
K-means Clustering (contd.)

• K-means Clustering Tree (KCT) contd.
• KCT has a single root node
• Leaf nodes of KCT contain features for an image
  shape and a pointer to the images.
• Non-leaf nodes of the tree correspond to the
  other levels of the index.
• The nodes correspond to disk pages and the
  structure is designed so that search requires
  visiting only a small number of pages.

25
K-means clustering (contd.)

• KCT Tree creation
• Assume 2-D feature vector for each of 19 object
  as
• -3.0, 3.0, -2.5, 3.0, -2.0, 2.0,
• -1.5, 2.0, -3.5, 1.5, -4.0, 1.0,
• -3.0, 0.5, -3.0, 0.0, -1.5, 0.5,
• 2.5,-0.5, 2.5,-1.0, 4.5,-1.0,
• 0.5,-1.5, 1.0,-2.0, 3.0,-2.0,
• 4.0,-2.0, 0.5,-2.5, 1.0,-2.5,
• 2.0,-3.0

26
K-means clustering (contd.)

• Same 2-dimensional numerical values
• Each object value will be in leaf node of KCT

27
K-means clustering (contd.)

• Corresponding KCT tree

28
K-means Clustering (contd.)

• Insertion possibilities
• A leaf node is full and a new object is to be
  inserted in that leaf node. In such case, an
  insertion results in overflow and, therefore, the
  node must split. As a result, a new non-leaf node
  and two leaf nodes are constructed and linked to
  that part of the tree. We also need to train the
  non-leaf node to get weight values.
• When the node in which the object has to be
  inserted has only one object in it. In this case,
  object can be simply added into node without any
  additional cost.

29
K-means Clustering (contd.)

• Deletion possibilities
• When the sibling node is full-leaf node or
  non-leaf node and we delete an object from full
  leaf node
• ? delete object from that node.
• When the sibling node is leaf node with an object
  and we delete an object from full- leaf node
• ? delete object from that node, combine two leaf
  nodes, delete its parent non-leaf node, and
  connect remaining full-leaf node to deleted
  non-leaf node's parent node.

30
K-means Clustering (contd.)

• Deletion possibilities contd.
• When the sibling node is full-leaf node and we
  delete an object from leaf node with itself
• ? delete object from that node, delete its
  parent non-leaf node, and connect remaining
  full-leaf node to deleted non-leaf node's parent
  node.
• When sibling node is non-leaf node and we delete
  an object from leaf node with one object
• ? Delete an object from that node, delete its
  parent non-leaf node, and connect deleted
  non-leaf node's child node to deleted non-leaf
  node's parent node.

31
K-means Clustering (contd.)

• Algorithm for retrieving images
• // Search an object with feature F using KCT
• b ? block containing root node of a KCT
• read block b
• while (b is not a leaf node of the KCT) do
• next ? recall Backpropagation using weights
  in block b
• b ? next
• read block b
• search block b for the most similar object with
  feature F // search leaf node
• if found then
• read index file block display images with
  object

32
Experimental Results

• Experimental Setup
• Environment PC with Microsoft Windows 98
• Language C / C
• Images
• Normalized to grayscale and 128 x 128
• Grayscale images to binary images for chain-codes
• Data set size
• 100 original images
• 5 variants involving translation, rotation and
  scaling.

33
Experimental Results (contd.)

• Sample image shapes and their seven moment
  invariants

34
Experimental Results (contd.)
A subset of grouping results using database
images at the root level of KCT. Objects in top
row occupy the left subtree while the bottom row
objects become right subtree.
35
Experimental Results (contd.)

• Grouping results with database images at the 3rd
  level of KCT. The top row of objects forms the
  left subtree of KCT while the bottom row is the
  right subtree.

36
Experimental Results (contd.)

• Results of sample queries. Query shape is given
  in row 1, column 1 of each image while the
  retrieved images include the query shape.

37
Experimental Results (contd.)

38
Conclusions

• Presented a moment invariants based image
  indexing scheme.
• Chain codes are used to reduce size of database
• Indexing is based on K-means clustering with k
  2 and Backpropagation to get weights for each
  node.
• Retrieval of images was based on finding the leaf
  node that includes similar images.
• Limitation Small image collection and limited
  training data.