An Integrated Framework for the Management of Video Collection

1
An Integrated Framework for the Management of
Video Collection
Nicolas Moënne-Loccoz, Bruno Janvier, Stéphane
Marchand-Maillet and Eric Bruno Viper
group Computer Vision Multimedia Laboratory
University of Geneva

• Managing large collection of Video Documents to
  ease Computer Vision Research

2
Outline

• Motivations
• Modelling the Data
• Storing the Data
• Accessing the Data
• Applications
• Conclusion

3
Motivations

• Content-based Video Retrieval Research
• Storing Descriptors Annotations
• Evaluating Index structures Retrieval
  algorithms

Storing
?
Modeling
Accessing
Mutlimedia Documents
Descriptors Annotations Index structures
Management Framework
4
Motivations

• MPEG-7 multimedia documents model
• Standard
• Extensible
• Weak ontology support
• Native XML-Database storage
• Well suited for MPEG-7 files
• Document-centred
• Indexing capabilities
• Lack of well-defined global solution

5
Overview

• Generic Data Model
• Relational DBMS
• Raw Data Access
• External Indexing Processor

Data Model
MaxDB DBMS
OVAL
Indexing Processor
Management Framework
6
Data Model

• Subset of MPEG-7 model
• MPEG-7 compliance (XML-enabled model)
• Temporal Segments
• Central Data Unit (Data-centred model)
• Ontology Capabilities
• W3C-Ontology Web Language compliant
• Feature Space Indexing structure

7
Data Model
Document
Media Stream
Information
Temporal Segment
Composed by
Structure
Ontology
Annotation
Descriptor
Feature Space
Description
Index
Role
Uses
8
Data Storage

• Videos Documents
• TRECVid-2003, MPEG-7, IM2/M4 Corpus
• 100 hours MPEG-1 Videos
• Semantic Descriptions
• TRECVid-2003 Annotations, Manual Annotations
• Feature-based Descriptions
• Shot segmentation
• Visual Content Decomposition (Camera Motion,
  Mosaic, Events)
• Global Activity

9
Data Access

• DBMS Access Facilities
• (SQL queries, Query optimization, Indexes)
• OVAL Raw Data Access
• Object-based Video Access Library (C, Java,
  Matlab)
• Exact and Random access to raw media stream data
• Plugins based (MPEG-1, MPEG-2)
• Generic efficient access to raw data

10
Data Access

• External Indexing Access
• Semantic Description
• Use of ontology structure
• Feature Based Description
• Index structure
• (Distance Matrix, Vantage Point Tree)
• ? Index processor MATLAB

Index file
Query Data
Index Data
Access Script
Index Processor
11
Application (1)Evaluation of Activity-based
descriptors

• Activity content is extracted from shot segments
• Activity is derived from optical flow estimation
• Allow to define an activity based similarity
  measure
• How this measure is efficient for video
  retrieval ?

12
Application (1)Evaluation of Activity-based
descriptors

• Considering the activity related concepts
•  Action ,  Talking head ,  Human motion 
• 900 temporal segments
• Precision-Recall graph from similarity ranked
  queries
• Using Distance Matrix index

13
Application (2)Collection Guide

• Browsing a collection of Video Temporal Segments
• According to their proximity in a feature space
• Projection of a distance matrix in a 2D
  representation space
• Curvilinear Component
• Analysis
• Direct use of the index
• Direct use of OVAL

14
Conclusion

• Multimedia Documents Management Framework
• Relational DBMS facilities
• MPEG-7 compliant
• Exact, random and efficient access to raw data
• Standard ontology support
• Complex indexing structures
• No raw data access across network
• Lack some MPEG-7 features
• Light and efficient framework for computer
  vision research

? Questions ?