MPEG7 Interoperability Use Case

1
MPEG-7 Interoperability Use Case
2
Motivation

• MPEG-7 set of standardized tools for describing
  multimedia content at different abstraction
  levels
• Implemented in the form of XML Schemas
• Lack of explicit semantics intended semantics in
  form of documentation
• Lack of precise semantics ambiguities resulting
  from flexibility in structuring the descriptions

3
Motivation (contd)

• Attach formal semantics to mpeg7 by translating
  it to an ontology
• The lack of precise and explicit semantics leaves
  room for different interpretations
• ? interoperability issues raised among the
  developed mpeg7 ontologies

4
Examples I

• Semantically identically metadata can be
  represented in multiple ways
• E.g. annotating an image depicting Zidane
  scoring against England
• Possible ways using free annotation, keywords,
  structured annotations etc.

5
Examples I (contd)
ltStructuredAnnotationgt ltWhogt ltName
xmllang"en"gtZinedine Zidanelt/Namegt lt/Whogt
ltWhatActiongt ltName xmllang"en"gt
Zinedine Zidane scoring against England.
lt/Namegt lt/WhatActiongt lt/StructuredAnnotati
ongt

• ltFreeTextAnnotation xmllang"en"gt
• Zinedine Zidane scoring against England
• lt/FreeTextAnnotationgt
•  
• ltKeywordAnnotation xmllang"en"gt
• ltKeywordgtZinedinelt/Keywordgt
• ltKeywordgtZidanlt/Keywordgt
• ltKeywordgtscoringlt/Keywordgt
• ltKeywordgtEnglandlt/Keywordgt
• ltKeywordgtgoallt/Keywordgt
• lt/KeywordAnnotationgt

6
Example II

• The image representation in itself as a
  multimedia entity can be performed in multiple
  ways as well
• Possible ways using the Stillregion DS or even
  the VideoSegment DS no formal semantics to
  prevent one from selecting to interpret an image
  as a still region or as a frame of a video
  sequence

7
Example III

• Semantically different metadata can be
  represented using the same description tool
• E.g., the Multimedia Description
  SchemesVideoSegment DS can be used to represent
  a single frame in a video, an arbitrary sequence
  of frames, or the full video sequence
• Three different semantic concepts frame, shot,
  video that cant be discriminated based on the
  provided XML Schemas

8
Example IV

• Loose (intended) semantics of MPEG-7 e.g. with
  respect to the intended meaning of the structure
  decomposition schemes
• E.g., not part-whole semantics having an image
  annotated as Zidane scoring and a second image,
  having a segment annotated the same, a single
  query wouldnt be adequate to retrieve both

9
Approaches to building an MPEG-7 ontology

• The Hunter approach
• Initially a DAMLOIL enriched RDFS ontology,
  subsequently translated into OWL
• Intended to be used under a core ontology (ABC)
  to enable harmonization with domain ontologies
• Decomposition captured using subclass constructs
• However no axioms included (e.g. an image
  consists of a set of spatial, and not temporal,
  segments)

10
Approaches to building an MPEG-7 ontology (contd)

• The SMART Web project approach
• The MultimediaContent and Segment classes along
  with corresponding properties implement the
  recursive nature of multimedia content
  decomposition
• Partial axioms

11
Approaches to building an MPEG-7 ontology (contd)

• Tsinarakis approach
• Translate the MPEG-7 MDS into OWL to serve as a
  core ontology for the integration of domain
  specific knowledge into the MPEG-7 semantic
  metadata
• Domain ontologies need to be mapped to the
  MPEG-7 schemas

12
The Hunters and Tsinarakis approaches in
practice

• Use Case Annotate and image of Zidane scoring

image01 mpeg7depicts goal01 goal01
abchasAction scoring01 scoring01 abchasAgent
s_b1 s_b1 hasName Zinedine Zidane
image01 rdftype mpeg7Image goal01 rdftype
sGoal scoring01 rdftype sScoring
mpeg7Image rdfssubclass mpeg7MultimediaContent
mpeg7MultimediaContent rdfssubclass
abcManifestation sScoring rdfssubclass
abcAction sGoal rdfssubclass abcEvent
region01 rdftype mpeg7StillRegion
(Hunters approach)
13
(contd)

• Zidane rdftype mpeg7PlayerObject
• mpeg7PlayerObject owlsubclass
  mpeg7AgentObjectType
• mpeg7 AgentObjectType rdfssubclass
  mpeg7SemanticType
• Zgoal rdftype sGoal
• sGoal rdftype mpeg7EventType
• mpeg7 EventType rdfssubclass mpeg7SemanticType
• Zidane mpeg7Relation Zgoal

(Tsinarakis approach)
14
Possible solutions

• Define semantic mappings between the various
  mpeg7 ontologies (exploit the potential of using
  rules on top of ontologies and reason on them)
• Built a core multimedia ontology (exploit perhaps
  a foundational ontologys axiomatization to
  ensure not only explicit but precise semantics as
  well)
• Use of semantically enabled profiles (again
  formal ways to specify the semantics constraints
  are required)

15
Thank you!