Comparing feature sets for contentbased image retrieval in a medical case database

1
Comparing feature sets for content-based image
retrieval in a medical case database

• Presentation at the SPIE Medical Imaging
  conference 2004

Henning Müller Service of Medical
Informatics Geneva University Hospitals
2
Overview

• The problem
• The systems we use
• medGIFT
• Casimage
• Features and variations tested
• Evaluation methodology
• Results
• Comparison and best feature sets
• Conclusions

3
The problem

• Which visual features can be chosen to represent
  images for content-based medical image retrieval?
• Extreme importance for retrieval quality
• Very little literature, especially in focused
  domains
• Abundance of visual descriptors
• Many of them model the same information
• How to evaluate image retrieval applications and
  their features?
• No real standard yet
• No benchmarking event such as TREC (Text
  REtrieval Conference)
• No standard datasets
• What can we do?

4
medGIFT

• Based on the GIFT (GNU Image Finding Tool)
• http//www.gnu.org/software/gift/
• Framework for image retrieval (indexing
  structures, features, tools, interfaces)
• MRML as a communication language
• Available free of charge
• Outcome of Viper project http//viper.unige.ch/
• Uses technologies well-known from text retrieval
  for image retrieval
• Changes for images in the medical domain
• More importance on grey level features than on
  colours
• More importance on the texture features
• Link with a medical case database
• Changes in the user interface to display
  diagnosis
• Connection with the full textual description and
  other images of the case

5
The user interface
Link to the Case description
Similarity score
Diagnosis
Choice of user (pos, neg, neutral)
6
Casimage

• Medical case database, teaching file system
• MIRC compatible
• Used in every-day practice
• More than 50,000 images stored in radiology alone
• Images in jpg, level/window settings on insertion
• Textual descriptions vary in quality and language

7
Features and variations

• Gabor filters
• Directions (4)
• Scales (3)
• Changes in the computation and quantization
  (complex or not)
• Grey level and colour features
• HSV colour space (H18, S3, V3, Gray4)
• Various quantizations of mainly grey levels
• Colour is not extremely important
• Data sets
• We have two data sets that give differing results
• One data set is now publicly available for system
  comparison (imageCLEF, 8751 images)
• First query step and relevance feedback

8
Evaluation methodology

• Based on the experiences from TREC for text
  retrieval
• http//trec.nist.gov/
• Definition of a medical image dataset
• Definition of query topics
• Execution of experiments with the query topics
  for different system configurations
• Generation of ground truth (gold standard) based
  on the first N results of the systems (pooling)
• Evaluation of the results with the ground truth
• Generation of relevance feedback using the
  ground truth data
• Comparison results as well for queries using
  relevance feedback

9
Evaluation measures

• Based on the measures used by TREC
• Statistical dependence but they show various
  aspects of the system
• Precision/Recall Graphs
• Precision and Recall Measures P(20), P(50),
  R(100), R(P(.5)), P(NR)
• Rank measures Rank1, average rank measures
• Query response time
• System description used
• Number of images in db, number of relevant

10
Results (1)
11
Results (2)
12
Results (3)
13
Results (4)
14
Summary of the results

• Grey scale
• In the first query step, the results are very
  close together
• Feedback queries are best with 16-64 grey levels
• Changing quantisation between query steps seems
  to be a good technique (could use query
  expansion)
• Gabor filters
• More scales and directions improve results but
  make the queries slower
• More directions are less good for feedback
  queries than scales
• Standardized testing of features on varied
  datasets is needed for feature comparisons

15
Conclusions

• A proper evaluation methodology is important
• Databases (freely available!)
• Gold standard
• Evaluation measures
• To really optimize systems, they need to be
  component-based (to plug-in features as needed)
• Feature comparisons for specialized tasks are
  needed
• A benchmarking event will even be better
• Benchathlon
• imageCLEF can be a start for medical images