AMI Tracking Evaluation

1
AMI Tracking Evaluation

• Vítezslav Beran
• SERVITE, 13.1.2006
• Department of Computer Graphic and Multimedia
• Faculty of Information Technology
• Brno University of Technology

2
Content

• MULTI-OBJECT TRACKING EVALUATION
• Coverage test
• Configuration errors
• Identification errors
• AMI TRACKING EVALUATOR
• User manual
• Data formats
• Implementation
• Console Evaluator

3
Multi-object Tracking Evaluation

• Introduced in AMI WP4 Tracking Evaluation
  scheme 1.0, 2006
• Two data sets
• Ground Truth (GT)
• Estimates (E)

4
Coverage test

• Two measurements
• Recall
• Precision
• F-measure

5
Configuration errors

• Frame errors

• False positive (FP)
• False negative (FN)
• Multiple trackers (MT)
• Multiple objects (MO)
• Configuration distance (CD)

6
Configuration errors (cont.)

• tI is coverage threshold

7
Configuration errors (cont.)

• Configuration Map
• Configuration Errors

E GT occ
1 1 0
3 2 0
4 2 0
4 3 0
Error type value
FP 1
FN 0
MT 1
MO 1
CD 0,33
8
Configuration distance

• Relevant only together with other errors.

9
Occlusion

• t0 is occlusion threshold
• when occlusion is set ? no error is computed

10
Normalization

• for easy comparison of tracking algorithm errors

11
Identification errors

• Sequence errors
• Falsely identified tracker (FIT)
• Falsely identified object (FIO)

12
Identification errors (cont.)

• Object purity (OP)
• Track purity (TP)
• Not interesting for our purpose.

13
Example sequence

14
Example sequence results
15
AMI Tracking Evaluator

• Windows platform application
• tracking video data evaluation
• load / save data sets from file source
• GT and E objects visualization
• errors visualization
• console version

16
User interface
17
Data formats - XML

• lt!ELEMENT AVEvents (EventGroups?, EventTypes?,
  File?, Editor?)gt
• ...
• lt!ELEMENT File (Source, TimeFormat?, Event,
  Title)gt
• lt!ELEMENT Source (PCDATA)gt
• lt!ELEMENT TimeFormat (PCDATA)gt
• lt!ELEMENT Event (ID, Time, Text?, Parameters?)gt
• lt!ELEMENT ID (PCDATA)gt
• lt!ELEMENT Time (PCDATA)gt
• lt!ELEMENT Text (PCDATA)gt
• lt!ELEMENT Parameters EMPTYgt
• lt!ELEMENT Title (Time, Text)gt
• ...
• lt!ELEMENT Editor ANYgt

18
Data formats - TXT

• Three format supported
• format 1
• frame frameID
• object objectID BoxCenterX BoxCenterY
  Width/2 Height/2
• object ...
• format 2
• imageframeID. objectID minXPos minYPos
  maxXPos maxYPos
• format 3
• frameID objectID visibility minXPos minYPos
  maxXPos maxYPos

19
Conclusion

• tool for multi-object tracking evaluation
• same tool by IDIAP is much more strict to data
  source formats
• presentation, documentation and downloads
  available onhttp//www.fit.vutbr.cz/research/gran
  ts/ami/evaluator.html

20
Future work

• evaluate our evaluation tool

21
Thank you for your attention.

• Any questions, please?