Semantic Integration and Retrieval of Multimedia Metadata

1
Semantic Integration and Retrieval of Multimedia
Metadata

• Roberto García and
• Universitat de Lleida, Lleida, Spain
• Òscar Celma
• Universitat Pompeu Fabra, Barcelona, Spain

2
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

3
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

4
Introduction

• Digital Media requires Multimedia Metadata.
• Main requirements
• Great amounts of metadata
• Low-level to user-friendly metadata
• Interoperability
• Proposal semantic metadata and an ontological
  framework for integration.

5
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

6
Ontological Framework

• Multimedia ontology.
• Profit from non-semantic initiative MPEG-7.
• XML ? Semantic Web
• XSD2OWL schema to ontology.
• XML2RDF XML instance data to RDF instances.
• Complete MPEG-7 OWL Ontology

7
Ontological Framework

• MPEG-7 Multimedia Content Description Standard
• content-based description for audiovisual
  information
  
• content searching, filtering and identification
• Descriptors
• low-level features
• structure
• semantic

• models
• collections
• creation...

8
Ontological Framework

• Other semantic multimedia metadata initiatives
• Hunter02 not complete, RDFDAML.
• Tsinaraki04 not complete, semantic part of MDS.
• Troncy03 not complete, from an ontology to
  MPEG-7.
  
• Direct benefits XQuery vs. Semantic Query,
  example next

9
Ontological Framework
10
Ontological Framework

• ReDeFer XSD2OWLMappings

11
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

12
Integration Framework

• MPEG-7 OWL Ontology as integration base for
• Semantic metadata align ontologies with MPEG-7
  ontology.
  
• MusicBrainz (http//musicbrainz.org)
• Simac (http//www.semanticaudio.org)
• Music Vocabulary (http//www.kanzaki.com/ns/music)
  
  
• Etc.
• XML2RDF RDF from MPEG-7 XML instance metadata.

13
Integration Framework. Architecture
14
Integration Framework

• MusicBrainz to MPEG-7 OWL mappings

15
Integration Framework

• Simac to MPEG-7 OWL mappings

16
Integration Framework

• Music Vocabulary ontology to MPEG-7 OWL mappings

17
Integration Framework. Architecture
18
Integration Framework

• XML2RDF XML tree ? RDF graph.
• Deduce blank node types from XSD2OWL ontologies
  restrictions.

19
Integration Framework

• XML2RDF example

20
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

21
Integration Retrieval Scenario

• Low-level descriptors from audio track signal
• Track editorial metadata

a simacTrack
simactempo 122 musicbrainztrmid e3c41bc1-
4fdc-4ccd-a471-243a0596518f.
a musicbrainzTrack
dctitle Blowin in the wind dcauthor
musicbrainzsortName Bob Dylan
musicbrainztrmid e3c41bc1-4fdc-4ccd-a471-243a05
96518f.
22
Integration Retrieval Scenario

• Generic MPEG-7 based integration rules.
• Example audio track with the same fingerprint
  are the same individual, their metadata is merged.

23
Integration Framework
24
Integration Retrieval Scenario

• Transparent retrieval, MPEG-7 based queries.
• RDQL example
• SELECT ?title
• WHERE (?track,,mpg7AudioSegmentType),
  (?track,,?title),
  (?track,,?author),
  (?author,,Bob Dylan), (?track
  ,,?tempo)
• AND (?tempo 120)
• USING mpeg7 FOR tologies/2005/03/Mpeg7-2001.owl

25
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

26
Integration Framework
27
Conclusions

• MPEG-7 OWL ontology, an upper-level multimedia
  ontology for
  
• Semantic metadata integrationalign ontologies to
  MPEG-7 OWL.
  
• XML metadata integrationXML2RDF.
• Semantic queries for retrievalgeneric queries
  based on MPEG-7.

28
Contents

• Introduction
• Ontological framework
• Integration framework
• Integration Retrieval Scenario
• Conclusions
• Future Work

29
Future Work

• Semantics-based recommendation systems
• http//foafing-the-music.iua.upf.edu
• http//rhizomik.net
• Infer user-friendly metadata from low-level
  descriptors.

30
Thank you for your attention

• Contact
• roberto_at_rhizomik.net
• ocelma_at_iua.upf.edu