Salient event detection in video surveillance scenarios

1
Salient event detection in video surveillance
scenarios

• Kenneth Ellingsen
• Masters thesis presentations - 05.06.2008

Supervisor Faouzi Alaya Cheikh, Dr.
Tech. Department of Computer Science and Media
Technology Gjøvik University College, Norway
2
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Feature extraction
• Feature analysis
• Results
• Conclusions

3
Introduction

• Large amounts of surveillance data
• accumulate each day.
• Monitored by very few observers relative to the
  number of cameras which makes it impossible to
  detect and respond to all abnormal event when
  they occur.
• Propose a system for automatic detection of
  abnormal events in video surveillance scenarios.

4
Introduction

• Goal is to extract simple and reliable features
  which are descriptive and that can be used by an
  unsupervised algorithm to discover the important
  and unusual events.
• Examine the possibility of modeling abnormal
  events.
• Analyze objects behaviors in video sequences
• over time.
• Define criterias that characterizes the event.
• Compare features against predefined criterias.

5
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Feature extraction
• Feature analysis
• Results
• Conclusions

6
Abnormal events

• Abnormal events are something that deviates from
  the normal behavior. What is abnormal?
• Unpredictable behavior.

7
Abnormal events

• Types of events
• Chasing
• Exchange of objects
• Fighting
• Loading/unloading
• Object dropping
• Sneaking
• Stealing
• Focus on the event of object dropping in public
  places such as airports and train stations etc.

8
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Feature extraction
• Feature analysis
• Results
• Conclusions

9
Proposed system

• System overview
• Four main blocks
• Background estimation
• Object tracking
• Feature extraction
• Feature analysis

10
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Feature extraction
• Feature analysis
• Results
• Conclusions

11
Event classification

• Object dropping
• Subjective analysis of several surveillance
  datasets.
• Derive a general description of object behavior
• during the event.
• The extracted low-level features
• Area
• Center of mass
• Displacement information
• Width-height-ratio
• Numel
• Minor axis

12
Event classification

• Object dropping criterias

Name Description
Area Significant decrease in first objects size, no sudden significant increase during previous frames. Second objects size equal first objects decrease.
Center of mass Examine distance between first and second objects. It must be equal or less then 2 times the minor axis to be considered as close.
Directional information First objects translation history show a gradually increase in displacement before the object drop point.
Width-height-ratio A ratio equal or higher then 0.5 accounts for an object standing, or bending slightly. Ratio increase before the drop.
Numel Number of objects increases by one when the object dropping occur.
Minor axis Representative for the first objects width (when standing). Used as threshold with Center of mass-feature, as relation to the objects actual height.
13
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Feature extraction
• Feature analysis
• Results
• Conclusions

14
Feature extraction

• Extract and save feature data of all object for
  each frame.
• Filter feature data to remove noise elements.
• Sort feature data to obtain correct labeling of
  objects.
• Plotting of data for visual analysis.

15
Experiments

• Example plots

Directional information (x-axis)
Numel
Center of mass (x-axis)
Center of mass (y-axis)
16
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Experiments
• Feature analysis
• Results
• Conclusions

17
Feature analysis

• The analysis-stage is triggered by the
  Numel-feature.
• One feature by itself is not conclusive enough to
  determine an object dropping.
• A combination of the features has to be taken
  into consideration.
• Some features need to be examined over a time
  period.

18
Feature analysis

• Object dropping classifier

input video search window 2 x framerate(video)
for each frame if (numel increase by 1 and
numel gt 2) Area true when
Significant drop in size of first object at
current frame No significant size
increase in search window Second
objects size equal to first object size drop
Center of mass true when Distance
between first and second object is less then 2 x
Minoraxis Ratio true when Increase for
the first object before drop within search
window Highest ratio near point of first
objects standstill Directional information
true when Find first objects approx.
standstill in search window Check first
objects translation history in search window from
point if it of standstill gradually increase
until drop is made if (all return
true) object drop has occurred else no
drop has occurred end end end
19
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Experiments
• Feature analysis
• Results
• Conclusions

20
Results

• Object dropping videos
• Table with results from analysis-stage.
• Video 2 shown below.

Input Frame of drop Area Center of mass Ratio Directional information
Video 1 178 178 178 178 178
Video 2 87 87 87 87 87
Video 3 152 152 152 152 152
Video 4 145 145 145 145 145
21
Outline

• Introduction
• Abnormal events
• Proposed system
• Event classification
• Experiments
• Feature analysis
• Results
• Conclusions

22
Conclusions

• We were able to model object dropping, by
• Subjective analysis of video data.
• Making a general description of the event.
• Define a set of criterias.
• Extracting simple features from object.
• Based on the event classification the system
  managed to detect the points of the object
  dropping.

23

• THE END