First Insights into the Library Track of the OAEI

1
First Insights into the Library Track of the
OAEI

• Dominique Ritze
• Mannheim University Library

2
Motivation
Ontology Mapping
Publication x
Search
0 results
subject (thesaurus 2) ontology alignment
Thesaurus 1
Thesaurus 2
Ontology Mapping

Ontology Alignment
Ontology Mapping
Publication x
Search
subject (thesaurus 1) ontology alignment
3
Overview

• Ontology Matching
• OAEI
• Thesaurus vs. Ontology
• OAEI Library Track 2012
• Lessons learned and Future Work

4
Ontology Matching
Person
People
Author
Author
lt Author, Author, , 0.97 gt lt Paper, Paper, ,
0.94 gt lt reviews, reviews, , 0.91 gt lt writes,
writes, , 0.7 gt lt Person, People, , 0.8 gt lt
Document, Doc, , 0.7 gt lt Reviewer, Review, ,
0.6 gt
CommitteeMember
writes
Reviewer
PCMember
reviews
Doc
reviews
Document
Paper
Paper
writes
Review
5
Ontology Matching Evaluation
O1
Tool
A
Test
O2
m
R
Result
6
Ontology Alignment Evaluation Initiative (OAEI)

• Annual campaign started 2005
• Different tracks/datasets
• Benchmark, Anatomy, Conference, Multifarm, Large
  BioMed, Library, Instance Matching
• 21 submitted systems (2012)
• Goal Improving the performances of the ontology
  matching field
• Through comparison of algorithms
• New challenges for the systems

7
Thesaurus Ontology?
SKOS OWL
skosconcept owlclass
skosprefLabel skosalternativeLabel rdfslabel
skosscopeNote skosnotation rdfscomment
A skosnarrower B A rdfs subClassOf B
A skosbroader B B rdfssubClassOf A
skosrelated rdfsseeAlso
Germany
Commodities
Tropical Fruit
Ananas
Metal Product -gt Metal
8
OAEI Library Track

• Are current state-of-the-art ontology matching
  tools able to match thesauri?
• Dominique Ritze, Kai Eckert,
• Benjamin Zapilko, Joachim Neubert

9
Data Set

• Thesaurus for economics (STW)
• 6.000 concepts with 19.000 additional keywords
  (EN, DE)
• Thesaurus for the Sociel Sciences (TheSoz)
• 8.000 concepts with 4.000 additional keywords
  (EN, DE, FR)
• Reference alignment manually created in 2006
• Both actively used in libraries for keyword
  indexing

10
Execution

• 7GB Debian machine
• Timeframe 1 week
• 13 of the 21 submitted systems were able to
  generate an alignment
• No system had a heap space problem
• Evaluation Precision, Recall, F-Measure, Runtime

11
Results
System Precision Recall F-Measure Time (s) Size 11
GOMMA 0.537 0.906 0.674 804 4712
ServOMapLt 0.654 0.687 0.670 45 2938
LogMap 0.688 0.644 0.665 95 2620
ServOMap 0.717 0.619 0.665 44 2413 yes
YAM 0.595 0.750 0.664 496 3522
LogMapLt 0.577 0.776 0.662 21 3756
G02A 0.675 0.645 0.660 32773 2671
Hertuda 0.465 0.925 0.619 14363 5559
WeSeE 0.612 0.607 0.609 144070 2774 yes
HotMatch 0.645 0.575 0.608 14494 2494 yes
CODI 0.434 0.481 0.456 39869 3100 yes
MapSSS 0.520 0.184 0.272 2171 989 yes
AROMA 0.107 0.652 0.184 1096 17001
Optima 0.321 0.072 0.117 37457 624
How to evaluate the results? F-Measure of
0.67 good?
12
Results
System Precision Recall F-Measure Time (s) Size 11
MatcherPref 0.820 0.642 0.720 75 2190
MatcherDE 0.891 0.601 0.717 42 1885
MatcherAll 0.544 0.896 0.677 735 4605
GOMMA 0.537 0.906 0.674 804 4712
ServOMapLt 0.654 0.687 0.670 45 2938
LogMap 0.688 0.644 0.665 95 2620

MatcherEN 0.808 0.439 0.569 36 1518
CODI 0.434 0.481 0.456 39869 3100 yes
MapSSS 0.520 0.184 0.272 2171 989 yes
AROMA 0.107 0.652 0.184 1096 17001
Optima 0.321 0.072 0.117 37457 624
13
Manual Evaluation

• Between 38 and 269 new correct correspondences
  found per matcher
• Up to half of the correspondences correct
• Many new correspondences are quite simple
• Some more complex and interesting ones
• Automated production CAM
• Several incorrect ones if the labels are quite
  similar
• Difficult to distinguish the names of countries,
  their inhabitants and the languages

14
Lessons Learned

• Transformation SKOS to OWL causes some problems,
  especially regarding the labels
• Ontology matching systems are nevertheless able
  to match the thesauri and even discover unknown
  correct correspondences
• Interest of the community in this topic

15
Future Work

• Update reference alignment
• adapted results
• SKOS import for matching systems
• Use instance data to match thesauri?
• Other thesauri?

16

• Thank you for your attention!
• dominique.ritze_at_bib.uni-mannheim.de