Aucun titre de diapositive

1
(No Transcript)
2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
(No Transcript)
7
(No Transcript)
8
(No Transcript)
9
GENERAL ARCHITECTURE (indexing and retrieval)
10
GENERAL ARCHITECTURE (general ontology
construction)
11
Audiovisual Grounded Ontology project AVONTO

• Object ontology reworked from WordNet

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
Examples of interrogation for extraction
representative images
Google image search using the French word
 avocat 
Google image search using the English word
 avocado 
The word is now not ambiguous but images show
both the fruit and the tree
16
Toward a more focused interrogation
Google image search using the English word
 avocado tree 
Google image search using the English word
 avocado fruit
17
An other example of disambiguation using the
ontology
18
Images tagged by dog are too much ambiguous

• Current annotation  dog  from Picsearch

19
The concept of dog is much more well
represented by the addition of images linked with
the leaves of the taxonomy
20
Filtering retrieved images
21
clustering of Objects
22
Background Removal after segmentation
23
Automatic Segmentation Results
24
Choice of good representatives (we need not the
full set of images but good set of prototypes)
25
Use of commentaries in video
26
Semantics extraction from texts
Affenpinscher Airedale terrier Akita Inu
American staffordshire terrier
27
Semantics extraction from texts
Use results of syntactic analysis to extract
semantics

•  you have decided to adopt a dog 
• ?A dog can be adopted
•  for active and big sized dog  
• A dog can be active 
• A dog can have a big size 
•  Dogs like medulla marrowbones 
• A dog like marrowbones
•  when the dog wags tail, it is happy 
• A dog can be happy
• When a dog is happy it wags tail
• .

28
The color of things
29
Multilingual aspect
Spanish and Italian wordnet are aligned with a
part of English wordnet

• ?Use several bilingual dictionaries to add
  languages
• Ex heifer (EN), giovenca or manza (IT), novilla
  (ES)
• gives génisse (FR) using EN-FR, IT-FR, ES-FR
  dictionary
• Use bilingual dictionaries for the most precise
  concepts that have no translation ambiguities
• Ex racoon (EN) gives raton laveur (FR)
• Use scientific names to find translations into
  languages for animals, plants,
• Ex oryx gazella (Sc) gives orice gazella (IT)

30
Multilingual aspect in the ontology
ltowlClass rdfabout"bullock__1"gtltrdfscomment
rdfdatatype"http//www.w3.org/2001/XMLSchemastr
ing"gt Italian Translation manzo__2
lt/rdfscommentgt ltrdfscomment rdfdatatype"http/
/www.w3.org/2001/XMLSchemastring"gt Spanish
Translation novillo__2 lt/rdfscommentgtlt/owlClas
sgt
31
State of the construction in our lab
Images 4800 objects 1.2 Million
pictures Automatically collected, segmented,
clustered 250 Gbytes Video 6 channels 60
video/day 2GB/day Processing by LIMSI/VECSYS
speech to text technologies
32
State of the construction in our lab
Gregory Grefenstette 2006-2007 Morphosyntactic
(dependency relations) of the full French
web Extraction of semantics from this
analysis 2007-2008 Same processing for English
33
Conclusion
Very huge task Necessitate the combination of
various knowledge Collaboration of several teams
at the European level (even international) Link
with other initiative like UNL (Universal
word) Need a big computing and storage
capacity ? Super computer, large clusters, grid