ViPER Video Performance Evaluation Toolkit

1
ViPERVideo Performance Evaluation Toolkit

• viper-toolkit.sf.net

2
The ViPER Toolkit

• Unified video performance evaluation resource,
  including
• ViPER-GT a Java toolkit for marking up videos
  with truth data.
• ViPER-PE a command line tool for comparing
  truth data to result data.
• A set of scripts for running several sets of
  results with different options and generating
  graphs.

3
The Video Performance Evaluation Resource
Schema Mapping
Metrics
Filters
Ground Truth Editor
Truth Data
Performance Evaluation Tool
Video Analysis Algorithm
Result Data
Video Analysis Algorithm
Result Data
Video Analysis Algorithm
Result Data
Evaluation Results
4
ViPERGround Truth Editing

• viper-toolkit.sf.net

5
Ground Truth Editing
6
Ground Truth Editing

• Lets the user specify the task and the
  interpretation.
• Improved (Verson 4)
• Schema editor.
• Timeline view.
• Supports undo/redo.
• New video annotation widget.
• MPEG-2 support.

7
Latest Version in Series
8
Time Line View

• Provides summary of ground truth.
• Direct manipulation across frames.
• Feedback for indirect manipulation.

9
Time Line View

• Provides summary of ground truth.
• Direct manipulation.
• Quick editing of activities, events, and other
  CONTENT descriptors.
• Some ability to modify descriptors with dynamic
  attributes directly, if not the attribute values.
• Feedback for indirect manipulation.
• Easier to notice massive changes.

10
Enhanced Keyboard Editing

• Support for real-time mark-up of events and
  activities.
• Keys for creating and deleting activities.
• Keys for controlling rate of display (jog dials).
• Enhance mark-up of spatial data.
• Keys for creating, editing of a single
  descriptor's attribute.
• Overall attempt to minimize effort in a GOMS
  model.
• Mouse events unnecessary except for polygon
  editing.

11
Frame View
12
Schema Editor
13
ViPER File Format

• Represents data as set of descriptors, which the
  user defines in a schema.
• Each descriptor has a set of attributes, which
  may take on different values over the file.
• Like a temporally qualified relational database
  for each media file, where each row is an
  instance of a descriptor.

14
ViPER File Format Descriptors

• Descriptor Types
• FILE (Video Level Information)
• CONTENT (descriptors of the scene)
• Static attribute values
• Single instance of one type for any frame
• OBJECT (descriptors of instances, including
  events)
• Attributes are dynamic by default
• Multiple instances can exist at a single frame.

15
Attributes

• Attribute Types
• Strings, numbers, booleans, and enumerations
• Shape types, including bounding boxes and
  polygons
• Relations (Foreign keys)

16
Demonstration
17
ViPER-GT Internals
ViPER-GT A Video Ground Truth Annotation Tool
ViPER Metadata API
AppLoader Plug-In Manager
Jena
Core GT API
Pure Java MPEG Decoder
Schema Editor
Plug-Ins
Plug-Ins
Plug-Ins
Native Decoders VirtualDub QuickTime JMF
18

• VideoAnnEx
• IBM AlphaWorks MPEG-7 Editor
• OntoLog (OWL)
• Jon Hegglands RDF Video Ontology Editor
• Informedia
• CMU Digital Video Library
• PhotoStuff
• Still image annotation for the semantic web
• ANVIL
• Any one-off tool a developer has created to test
  their stuff