ETISEO%20Annotation%20rules

1
ETISEO Annotation rules

• Data structure
• Annotation tool and format
• Ground truth creation rules
• Reference data generation

2
Data structure - context
Context

• Id
• Type (ex zone)
• Name (ex exit_zone)
• Polygon

3
Data structure - object
Physical object

• Id
• Start time
• Type (ex person, vehicle)
• Subtype (ex car, van)
• Info2D
• Info3D

• Links
• Speed
• GT Nature
• GT Status
• GT Occluded (partially or completely)
• GT Connected with

GT specific to annotation purpose
4
Data structure - event
Event

• Id
• Name
• Start and end time
• Status
• Involved objects (physical object of interest and
  contextual object)

5
ETISEO Annotation rules

• Data structure
• Annotation tool and format
• Ground truth creation rules
• Reference data generation

6
Annotation tool and format
Annotation delivered with video

• Database - Excel file
• Context annotation - Viper-GT tool and format

Annotation delivered at the end of Etiseo

• Ground truth - Viper-GT tool and format
• Reference data (computed from GT) Specific tool
  and Viper-GT format

7
Annotation tool and format
8
Annotation tool and format
Viper-GT format example
lt?xml version"1.0" encoding"UTF-8"?gt ltviper
xmlns"http//lamp.cfar.umd.edu/viper"
xmlnsdata"http//lamp.cfar.umd.edu/viperdata"gt
ltconfiggt ltgt lt/configgt ltdatagt ltsourcefile
filename"fileETI-VS1-CR-4-C1.avi"gt ltobject
framespan"1100" id"4" name"physical_object"gt
ltattribute name"nature"gt ltdatasvalue
value"physical_object_of_interest"/gt lt/attrib
utegt ltattribute name"type"gt ltdatasvalue
value"person"/gt lt/attributegt ltattribute
name"info2D"gt ltdatabbox framespan"11"
height"182" width"86" x"138"
y"59"/gt ltdatabbox framespan"22"
height"183" width"86" x"139" y"59"/gt
ltdatabbox framespan"33"
height"185" width"86" x"141" y"60"/gt
ltdatabbox framespan"417"
height"187" width"87" x"143" y"60"/gt
ltgt ltdatabbox
framespan"40100" height"212" width"10" x"3"
y"10"/gt lt/attributegt ltgt lt/sourcefilegt
lt/datagt lt/vipergt
9
ETISEO Annotation rules

• Data structure
• Annotation tool and format
• Ground truth creation rules
• Reference data generation

10
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

11
Ground truth creation Rules for objects
Contextual object

• Annotated during its period of movement

12
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

13
Ground truth creation Rules for objects
Static physical object of interest

• Any physical object of interest is annotated on
  the whole video clip (ex a parked vehicle)

First frame and last frame of a video clip
14
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

15
Ground truth creation Rules for objects
Occlusions general rules

• Apparition of an object the BBox increase with
  object apparition including only its visible
  part,
• All BBox are limited by the image boundary,
• Partial and complete occlusion will be
  differentiate in the evaluation (attribute
  occluded).

16
Ground truth creation Rules for objects
Static occlusions (by a contextual object)

• Partial occlusion full BBox is drawn including
  hidden part,

17
Ground truth creation Rules for objects
Static occlusions (by a contextual object)

• Complete occlusion - two cases
• all escapes of the contextual object are
  visible full BBox is drawn all the time,

18
Ground truth creation Rules for objects

• all escapes of the contextual object are not
  visible the object is no more annotated when
  completely occluded

19
Ground truth creation Rules for objects
Dynamic occlusion (by another object)

• Partial or complete occlusion in both cases, the
  full BBox is drawn including hidden part.

20
Ground truth creation Rules for objects
Split object due to occlusion

• A unique BBox including all sub part is drawn.

21
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

22
Ground truth creation Rules for objects
Artefacts

• Only the part constituting the objects forms the
  BBox, artefacts (shadow, reflection) are ignored
  for instant.
• Annotated in video data set 2

23
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

24
Ground truth creation Rules for objects
Sudden illumination changes (tagged frames)

• Static object

• Object in movement position interpolation

25
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

26
Ground truth creation Rules for objects
Held object

• The holder and the held object are both annotated
  (if visible),
• The carried object is connected to the holder.

Example bag-person, person-bike, door-vehicle
27
Ground truth creation Rules for objects
Specific case

• Person getting in/out a vehicle annotated only
  when visible

28
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

29
Ground truth creation Rules for objects
Group - Crowd

• Group multiple bounding boxes (persons group)

• Crowd unique Bbox when individual annotation
  is impossible.

30
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

31
Ground truth creation Rules for objects
Crowded video clip

• Only object involved in event of interest are
  annotated. No false detections evaluation

32
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

33
Ground truth creation Rules for objects
Multi-view annotation
34
Ground truth creation rules

• Rules for objects
• Contextual object
• Static physical object of interest
• Occlusion
• Artefacts
• Sudden illumination changes
• Held object
• Group - Crowd
• Crowded video clip
• Multi-view annotation
• Rules for events

35
Ground truth creation Rules for events
Event data structure

• Id
• Name
• Start and end time
• Status
• Involved objects
• Id of the physical object of interest in GT files
• Id of the contextual object in context
  information files

36
Ground truth creation Rules for events

• Choice of event for each video clip in the event
  ontology gt defined in the database xls file
  (provided with video)

Sequence Name ETI-VS1-RD-15

Content Physical objects type Person, vehicle
Content Events of interest gets_in, gets_out, parked
37
Ground truth creation Rules for events

• Automatic annotation of events with quantitative
  constraints, example
• Related to kinematics properties stopped,
  running
• Related to spatial properties close_to,
  inside_zone
• Manual annotation of other events holding,
  pushing, waiting, gets_in/out,

38
Ground truth creation Rules for events
Example
39
ETISEO Annotation rules

• Data structure
• Annotation tool and format
• Ground truth creation rules
• Reference data generation

40
Reference data generation

• Filtering on object size
• Object speed computation
• Group generation
• Multi-view and 3D reconstruction
• Events with quantitative constraints generation

41
ETISEO Annotation rules

• Documentation
• ETI-Data structure and output format
• ETI-Video Annotation Rules

42
ETISEO Annotation rules
Questions