Object Recognition a Machine Translation

1
Object Recognition a Machine Translation

• Learning a Lexicon for a Fixed Image Vocabulary
• Miriam Miklofsky

2
Lexicons

• A vocabulary of terms used in a subject
• A specialized list of terms
• Devices that predict one representation given
  another representation

3
Dataset

• Aligned bitext
• Annotated images
• Images with regions
• Unknown which region of image goes with which
  word from text

4
EM
5
Clustering

• K means clustering
• Vector quantize the image region representation
• Kullback-Leibler divergence
• Relative entropy
• Measure of difference of two probability
  distributions over the same event space

6
Evaluation

• Auto annotate images
• Quantize regions
• Use lexicon to determine word
• Annotate image with word

7
Results - Annotation

• Base results
• 80 words of 371 word vocabulary could be
  predicted
• Retraining
• Similar results but some words with higher recall
  and precision

8
Results(cont.)

• Null probability
• Recall decreases
• Precision increases
• Clustering of like words
• Recall values of clusters higher than for single
  words

9
Results -Correspondence

• Base results
• Some good words up to 70 correct prediction
• Null prediction
• Predict good words with greater probability
• Word clustering
• Prediction rate generally increases

10
Evaluation

• Human evaluation
• Images viewed by hand
• Somewhat subjective

11
(No Transcript)
12
EM (cont.)
13
KL Divergence