Wei Wang, Payam Barnaghi The University of Nottingham Malaysia Campus

1
Semantic Support for Medical Image Search and
Retrieval

• Wei Wang, Payam BarnaghiThe University of
  Nottingham (Malaysia Campus)

The Fifth IASTED International Conference on
Biomedical Engineering BioMED 2007 Feb. 2007,
Innsbruck, Austria
2
Outline

• Introduction
• Semantic Enhanced Image Annotation
• Information Search and Retrieval
• Implementation and Evaluation
• Conclusion

3
Information Search Retrieval

• High recall, low precision
• Meta-data, vocabulary dependent, search context
• Content-based search expensive computation, low
  level features, dealing with high level concepts
• Semantic gap

4
Goals and Principles

• Representation of interoperable and re-usable
  meta-data, and device independent constructions
• Employing standard vocabularies to annotate the
  medical images.
• Enhancing the information search and retrieval
  process by considering the meaningful
  relationships

5
Challenges

• Identifying and constructing the meta-data
  structure
• Knowledge representation for the domain of
  discourse (e.g. Lung Cancer)
• Observing and exploiting the semantic
  relationships

6
XML limitations for semantic markup

• XML representation makes no commitment on
• Domain specific ontological vocabulary
• Which words shall we use to describe a given set
  of concepts?
• Ontological modelling primitives
• How can we combine these concepts, e.g. car is
  a-kind-of (subclass-of) vehicle
• ? requires pre-arranged agreement on vocabulary
  and primitives

• Only feasible for closed collaboration
• agents in a small stable community
• pages on a small stable intranet.. not for
  sharable distributed-resources

Davies, 03
7
A Semantic Web Vision

• Explicit information and reusable knowledge
  representation
• This knowledge should be
• Machine readable/processable
• Reusable
• Interchangeable
• Semantic Web provides
• Representation formats for resource content
• Expressing the knowledge using ontologies

8
Semantic-based Search

• Using semantic web as a universal language
• Using an ontology to represent our knowledge
  about a particular domain (i.e. lung cancer)
• Multimedia annotation (i.e. images)
• defining the ontology associations
• Information retrieval is based on the enhanced
  annotation and semantic inference

9
Domain Ontology

• The domain ontology represents the concepts and
  their relationships in a particular domain.
• We use the US National Cancer Institute (NCI)
  meta-thesaurus (converted to OWL form) as our
  domain ontology.
• Comprehensive knowledge about Anatomy, Disease,
  Finding, etc.

10
Annotation Framework

• Media dependent information (extracted
  automatically)
• Subject (based on the domain ontology)
• Content descriptions (keywords, meta-data)

11
Domain Ontology and Annotation Representation
A fragment of the media annotation
A fragment of the domain ontology
12
The System Architecture
Application
Middleware
Storage
Ontology
13
The User Interface Query Tool
14
System Implementation

• Sample CT scan images for lung cancer
• Annotation is done using the implemented
  annotation tool
• The search result is compared with an online
  medical image database, MedPix

London South Bank University Radiology
Database, http//myweb.lsbu.ac.uk/dirt/museum/
15
Evaluation of the results (II)
16
Conclusion

• We have shown an approach for IR from multimedia
  resources using
• Knowledge-based query
• Combination of IR results and semantic inference
• Context oriented (?)
• Use of the Semantic Web technologies to specify
  domain knowledge, media resources and meaningful
  relationships
• Semantic-based search technique provides better
  support for multimedia information retrieval
• Evaluation of the system with a large data set

17

• Thank you for your attention.

18
Acknowledgement

• John Davies, BT Research, Semantic Web,
  Presentation, 2003