Recent Advances in ViPER

1
Recent Advances in ViPER

• David Mihalcik
• David Doermann
• Charles Lin

2
What is ViPER?

• A tool for evaluating video understanding
  algorithms.
• Includes
• An Annotation Tool
• For labeling ground truth and browsing results.
• A Comparison Tool
• For evaluating result data with respect to ground
  truth.

3
What is the Problem?

• Lots of people, here and elsewhere, are working
  on video processing algorithms for information
  extraction, etc.
• Evaluating performance of the algorithms requires
  a lot of work, usually with tools developed by
  the algorithm designer.
• Is your solution any good? Prove it.

4
How to evaluate the algorithm?

• What is the problem?
• ViPER focuses on evaluating solutions to
  detection and tracking problems these determine
  if or where in the video some entity or event
  appears.
• Evaluation
• Comparison of the result data set against a truth
  data set.
• Truth, metrics, and rules for comparison are task
  dependent.

5
Goal of the ViPER Project

• To make evaluation of video algorithms simple,
  repeatable, and ubiquitous.
• As ground truth is required for evaluation,
  annotation must be made simple, as well.
• Avoid tedium.
• Avoid frustration.
• Support expert usage.

6
Ground Truth Annotation

• ViPER-GT supports annotation of temporally
  qualified spatial and nominal data on video files
  and still images.
• Go from the simple per-frame or shot annotations
  to detailed spatial markup.
• You can quickly indicate which frames contain
  people.
• Then, you add how many people per frame.
• With a lot of time and money, you can put boxes
  around them.

7
ViPER-GT Video Ground Truth Annotation Tool
8
For Example Person Tracking

• How well does an algorithm find and track humans
  moving through a video?
• To evaluate detection
• Truth must indicate which frames contain the
  person.
• To evaluate tracking
• Truth must contain spatial information,
  indicating where a person may be found.

9
Example of Annotation Person Detection
10
Example of Annotation Person Tracking
11
The ViPER Data Model

• Similar to a relational database
• Tables are Descriptor Definitions.
• Columns are Attributes.
• Rows are Descriptor Instances.
• Most descriptors are OBJECT descriptors
• Attributes are temporally qualified.
• Static OBJECTS have a frame range, but their
  attributes are not temporally qualified. Useful
  for events, etc.

12
The ViPER File Format and API

• Uses XML.
• I wont go into it here.
• There is a Java API.

13
Related Work

• VideoAnnEx
• OntoLog
• PhotoStuff
• Informedia

14
VideoAnnEx

• IBMs MPEG-7 annotation tool.
• Cool Features
• Cut detection makes it easy to add per-shot
  markup.
• Supports MPEG-7.
• Annoyances
• Not very good for spatial attributes.
• Commercial software. Not as extensible as ViPER.

15
OntoLog

• Jon Hegglands Tool for Temporal Markup with
  Ontologies
• Advantages
• Good support for key bindings and playback.
• Data model supports inheritance.
• Annoyances
• No spatial data support.

16
PhotoStuff

• MINDSWAPs Tool for Adding Semantic Web Markup to
  Images
• Cool Features
• Semantic Markup!
• Spatial Data!
• Annoyances
• Buggy and beta.
• No support for video.

17
Informedia

• CMUs tool for browsing video libraries
• Cool Features
• Advanced browsing functionality.
• But
• Focus on video library, not annotation.
• Not available for download, from what I can tell.
• Not terrifically extensible.
• See also Silver and Malach

18
Extending the Interface

• ViPER provides a lot of functionality, but is
  very general.
• It may be appropriate to extend viper-gt to
  better support marking up a different type of
  annotation.

19
Example Extension Adding Text Zones

• Adds a toolbar that allows typed bounding boxes.
• Instead of having to click create, auto-creates a
  new box.

20
Architecture of ViPER-GT

• Application Launcher
• Loads a set of javabeans from an RDF model.
• Allows modifying menus, i18n, etc.
• Is a bit of a pain to handle 'menu change'
  events, like for most recently used menu.
• Viper View Mediator
• Javabean container for ViPER API.
• Adds 'user interaction' methods to keep track of
  things not in API (focus, filters).

21
Continual Improvement

• SourceForge web site
• http//viper-toolkit.sf.net/
• Mail suggestions/comments to
• viper_at_cfar.umd.edu