Recognizing hand-drawn images using shape context

1
Recognizing hand-drawn images using shape context

• Gyozo Gidofalvi
• Computer Science and Engineering Department
• University of California, San Diego
• gyozo_at_cs.ucsd.edu
• November 29, 2001

2
Shape Context by Mori et al.
Key idea represent an image in terms of
descriptors at certain locations that describe
the image relative to those locations Shape
context of a point is the histogram of the
relative positions of all other points in the
image. Use bins that are uniform in log-polar
space to emphasize close-by, local structure.

3
Representative shape context efficient retrieval
of similar shapes by Mori et al.

• Matching Given two images, represented as n
  shape context descriptors, we want to find a
  one-to-one assignment of these descriptors, such
  that the X2 distance for the assignment is
  minimized ? O(n3) algorithm.
• Fast Pruning
• Represent the query image by a small number of
  shape context descriptors
• To calculate the cost of a match between the
  query image and an image in a DB perform nearest
  neighbors search
• Return a short list of the fist K best matches

4
Hand drawn images
Mori et al. tested the representative shape
context method on the Snodgrass and Vanderwart
line drawings. Queries were distorted versions
the original images. We gathered 6 sets of
samples for these line drawings and used them as
queries. All images were cropped and scaled to
500 by 500 pixels.
5
Performance on on hand-drawn images
Results of 300 queries for varying length of
short list returned Pruning factor ( number of
images in DB ) / ( length of the short list )
6
Sampling shape context
Can we improve performance? Shape context of
which points should represent and
image? Pixel-density based sampling Promote
points with higher or lower densities?
spread out
promote higher density
promote lower density
7
Density-based sampling results
promoting points with higher pixel-densities
promoting points with lower pixel-densities
8
Finding embedded objects
Task Given a query image that may contain some
clutter around an hand-drawn object find objects
from the DB that are most similar to it. How does
the presence of clutter affect the recognition of
the hand-drawn object? To obtain images for
embedded objects we find the outline of the
object, construct a binary mask for it, and using
logical operations (AND ? OR) we copy the clutter
around the object. Finding the outline of objects
is done using a method similar to flood-fill.
9
Embedding objects into some clutter
10
Acknowledgments

• Thanks to Mori at al. for providing me with
  source code for the shape context matching.
• Thanks to Serge for guidance, ideas.
• Special thanks to my Dad, my brother, and Hector
  for drawing me sample objects.