Project Overview

1
Project Overview
Video Understanding Evaluation
David CHER RD department SILOGIC S.A.S ,
Toulouse, France
2
ETISEO / Techno-Vision
ETISEO is part of the Techno-Vision program
Partially funded by the French government. 2
years duration, started in January 2005 To
evaluate vision techniques for video
surveillance applications.
3
Project committee
4
ETISEO Main objectives

• To promote evaluation usage in image processing
  domain,
• To diffuse good practices,
• To create perennial evaluation structure.

5
The way

• To support resources
• generation diffusion,
• - To create specific tools metrics,
• To support evaluation cycles,
• To encourage a large active participation
• International open participation,
• To communicate evaluation results.

6
Detailed Objectives

• To conceive automatic evaluation tools for
  vision algorithms to allow a fair and
  quantitative comparison between algorithm results
  and reference data.
• To acquire precise knowledge of vision
  algorithms capabilities to assess technological
  limits.

To have a meaningful evaluation
7
Main steps

• Nice, May 2005
• first INRIA seminar

• Lille, December 2005
• INRETS Seminar

• Evaluation cycle 1 January 2006
• to evaluate the process

• ETISEO Evaluation May 2006
• final evaluation

8
Resources production

• ETISEO Metrics definition

• ETISEO video annotation rules

• ETISEO data-set collection
• Content definition
• Partners equipment installation
• Video sequences recording
• Data set management

• ETISEO corpus annotation

• Automatic comparison tool.

9
Evaluation schedule

• First evaluation cycle ( for test )

• ETISEO evaluation cycle ( 2nd cycle).

Seminar
CYCLE 1
First evaluation
Evaluation review
CYCLE 2
Etiseo evaluation
Evaluation review
Seminar
10
Dissemination
VS-PETS 2005 2nd Joint IEEE International
Workshop on Visual Surveillance and Performance
Evaluation of Tracking and Surveillance. October
15-16 2005 Beijing, ChinaIn Conjonction with
ICCV 2005 (10th IEEE International Conference
on Computer Vision )
ASTI2005 2èmes meetings of Sciences and
Information Technologies Techno-langue /
Techno-vision Journey October 21th 2005,
Clermont-Ferrand, France
11
Dissemination
RFIA 2006 15th French-speaking congress
AFRIFAFIA Pattern recognition and Artificial
Intelligence "Performance evaluation in image
processing and vision by computer" Rond
table January 25-27 2006, Tours, France
PETS 2006 Workshop on Visual Surveillance and
Performance Evaluation of Tracking and
Surveillance June 18 -2006 New-York , USAIn
Conjonction with CVPR 2006 (IEEE Conference on
Computer Vision and Pattern Recognition 2006 )
12
Dissemination
CLEAR 2006 Evaluation Campaign and Workshop -
Classification of Events, Activities and
Relationships- supported by the European
Integrated project CHIL, the US ARDA VACE program
and the US National Institute of Standards and
Technology (NIST). April 2006, Southampton, U.K.
13
ETISEO Participation

• - All participants are welcome to participate.
• Voluntary is the rule
• participants are expected to test their
  algorithms on their own and to send their results
  to the evaluator.
• After the participants commitment,
• video database with ground truth and metrics with
  evaluation tools will be transmitted.
• - All participants will be welcome to attend
  freely workshops organized during the project.

14
ETISEO participants