Tattoo Image Matching and Retrieval for

1
Department of Computer Science
EngineeringCollege of Engineering
Tattoo Image Matching and Retrieval for Victim
and Suspect Identification
Jung-Eun Lee and Anil K. Jain
The Problem
Scars, Marks and Tattoos (SMT)
Experimental Results

• ANSI/NIST standard was proposed to ensure
  uniformity in capture and exchange of SMT data
• Eight major tattoo classes HUMAN, ANIMAL, PLANT,
  FLAG, OBJECT, ABSTRACT, SYMBOL, and OTHER 80
  subcategories

• Tattoo images are often captured under non-ideal
  conditions (e.g., by a surveillance video camera)
• Image deformations are classified as blurring,
  changes in illumination, color, aspect ratio,
  rotation and affine transformation

• SMT are imprints on the skin that can be useful
  for identifying a suspect or a victim
• Rising popularity of tattoos
• 16 of the adults in the US have tattoo(s)
• The highest incidence was found among Americans,
  between the ages 25 to 29 (36) and 30 to 39
  (28)
• More delinquents than non-delinquents have tattoos

(a) (b) (c)
(d) (e) (f)
(g)
Examples of image transformation (a) original
variations due to (b) blurring, (c)
illumination, (d) color component changes, (e)
affine transformation, (f) aspect ratio change,
and (g) rotation

• Limited number of categories in the ANSI/NIST
  standard and large intra-class variability make
  it difficult to label tattoos consistently

• Image Feature Extraction Scale Invariant Feature
  Transformation (SIFT)
• Extract salient keypoints which are Invariant to
  image scale, rotation, changes in illumination,
  noise, occlusion and minor changes in viewpoint

Illustration of large intra-class variability in
tattoo images. All the above images belong to the
FLAG category
Tattoos for Human Identification
Examples of SIFT Keypoints
Our Approach Content-based Tattoo Image Retrieval

• Keypoint Matching
• Matched by finding the nearest neighbors using
  Euclidean distance
• We use the number of matching keypoints as the
  similarity between two images

• Tattoos are being used for victim identification
• e.g., Tsunami (2004) and 9/11 Terrorist attacks
• Tattoos give specific identifying clues as to the
  social status, gang membership, religion beliefs,
  previous convictions and military service
• It is now a common practice for law enforcement
  agencies to photograph and catalog tattoo
  patterns. It is an effective tool for gang
  membership identification
• Current systems match tattoo images based on
  human-assigned class labels with the text query
  (a black dragon)

• Build an image (pattern)-based retrieval system
  for a large tattoo database
• Given an image query and an associated class
  label, retrieve the top-N most similar tattoos
  from the database
• Image query contains the content of the tattoo
  pattern
• SMT Database
• 5,300 tattoo images downloaded from the Web
  (4,700 common tattoos and 500 gang tattoos)
• 100 real tattoo pictures taken under different
  imaging conditions
• 250 scars and marks images from the Web

Matching examples with the numbers of matching
points between (a) similar and (b) different
images

• 86,460 transformed image queries submitted to a
  database with 4,323 original tattoo images
• SIFT features are more robust than simple image
  features, i.e., color, shape and texture
• 97 rank-1 retrieval accuracy is achieved

(a) (b) (c)
(d) (e) (f)
(g)
(a) (b)
(c)
(d)
Gang Tattoos (a) and (b) Ambrose, (c) Adidas
boys, (d) Brazers, (e) Latin Kings, (f) Family
Stones, (g) Insane Deuces (From GangInk.com)
Examples of tattoos used for victim and suspect
identification (a) Tsunami victim, (b)
unidentified murdered woman, (c) Teardrop
criminal tattoo (person has killed someone or had
a friend killed in prison), (d) Texas syndicate
(TS) gang tattoo
April. 18. 2008