information

1
information
retrieval
2
perspectives information retrieval

• application(s) - surveillance
• psychological - information overload
• experimental - search engines
• algorithmic - (media) similarity metrics
• system - unified (media) repository
• presentation - user profile / integration
• search - precision recall
• commercial - applications certification

3
(No Transcript)
4
media types
text, images, audio, video, structured data
retrieval

• image query -- all images with this person
• audio query -- identity of speaker
• text query -- all transactions with BANK Inc.
• video query -- all segments with victim
• complex queries -- convicted murderers with BANK
  transactions
• heterogeneous queries -- photograph murderer
  transaction
• complex heterogeneous queries -- in contact with
  murderer transaction

5

• precision -- how many answers are correct
• recall -- how many of the right documents are
  returned

precision ( returned and relevant ) / returned
recall ( returned and relevant ) / relevant
6
frequency table
term/document d0 d1 d2 snacks 1 0 0 drinks 1 0 3
rock-roll 0 1 1
indexing
7
(No Transcript)
8
video content
video v, frame f f has associated objects
and activities objects and activities have
properties

• 
  object instance
  (oid,os,ip)
• object-id -- oid
• object-schema -- os (fd,fi)
• set of statements -- ip name v and name v
  IN f

video
9
frame objects frame-dependent properties 1 Jane h
as(briefcase), at(path) - house door(closed) - b
riefcase 2 Jane has(briefcase),
at(door) - Dennis at(door) - house door(open) -
briefcase
10
object frame-independent properties value Jane ag
e
35 height
170cm house address
... color
brown briefcase color
black size
40 x 31
11
query language for video libraries

• 
  
• segment retrievals -- exchange of briefcase
• object retrievals -- all people in vs,e
• activity retrieval -- all activities in vs,e
• property-based -- find all videos with object
  oid

SQL, XML, logic-based, ...
12
media
abstractions
13
(No Transcript)
14
media abstraction

• state -- smallest chunk of media data
• feature -- any object in a state
• attributes -- characteristics of objects
• feature extraction map -- to identify content
• relations -- to capture state-dependent
  information
• (inter)relations between 'states' or chunks

• VO (O_i,Q_i,C_i) 1 lt i lt k
• where
• C_1,...,C_k -- mutually exclusive conditions
• Q_1,...,Q_k -- queries
• O_1,...,O_k -- objects

virtual objects
15
user
interface
16
(No Transcript)
17
augmented virtuality
18
(No Transcript)
19
information retrieval issues

• retrieval -- indexing success metrics
• video -- (temporal) description logic
• applications --

digital dossier
Create a VR that realizes a digital dossier for a
work of a particular artist. A digital dossier
represents the information that is available for
a particular work of art, or a collection of
works, of a particular artist.
http//www.cs.vu.nl/eliens/casus/
20
perspectives information retrieval

• application(s) - digital dossier
• psychological - focus
• experimental - user interaction
• algorithmic - (information) access
• system - unified presentation space
• presentation - embodied agents
• search - semantic annotation
• commercial - future systems