Monitoring Camera Video Analysis

1
Monitoring Camera Video Analysis
Vera Kettnaker
2
Motivation
Cameras are everywhere...
but how can we make sense of all this
information ?
NYT 3/19/98 gt 80 cams on short route through
Manhattan
3
Current use of monitoring cameras (1)
Video from most cameras is analyzed by users On
the web Traffic cams, weather cams,
Seismo-cam.
4
Current use of monitoring cameras (2)
Computer Vision event detection Recording and
preventing law violations running red lights,
accidents, theft robbery, planting bombs in
airports
5
Current use of monitoring cameras (3)
Single camera content analysis systems Commercial
example compute traffic statistics for traffic
planning Volume/counts ( of vehicles)
Lane Occupancy ( time lane is occupied)
Speed (average speed in mph or kph) Density
(average density volume/speed) Headway
(average headway in seconds) Length (average
vehicle length in ft/meters) Vehicle
Classification by user-selectable lengths (5
bins) Delay (average delay in seconds)
Queue Length
6
Current use of monitoring cameras (4)
Systems of multiple, overlapping
cameras Construct multi-view mosaics or project
into common coordinate system
large area can be monitored even if there
are obstacles
7
System in this talk
Several non-overlapping cameras monitor
environment in which objects move along pathways,
e.g. corridors, streets, highways
Open environment objects enter and leave, it is
not known how many
different people were visible.
8
Task
All single camera observations are collected into
global database.
Task reconstruct the paths of all objects
through the environment
link successive occurrences of the same
object Application trajectory statistics are
useful for traffic planning
9
Preprocessing
Single-camera tracker extracts moving object info
that is stored in database.
data base
10
Single-camera information extraction
1. Use combination of background subtraction and
Boykov-Veksler-Zabih motion algorithm to
segment moving objects.

11
Single-camera information extraction (2)

• 2. Represent people by the color of their
  clothing
• How much area of the person is covered in
  which color ?
• coarse histograms choice of color space
  important !
• Varies over time use probabilistic model
• Main problems
• shadows
• different lighting conditions
• different viewing angles
• clothing colors difficult beige, grey, jeans
  
• inclusion of background
• Visual matching alone wont be sufficient
  to solve the task !

12
Exploiting additional information
Observation the pathway structure imposes
constraints on when and where
moving objects can reappear.
13
Input
1. Database of observation intervals

• 2.Traffic statistics
• transition probabilities
• transition time probabilities
• frequency of new objects.

14
Output
Solution set of chains

• Requirements
• most likely according to
• - usual traffic behavior
• - similarity of visual
• appearance
• chains must be mutually
• exclusive

Quality measure
Difficult to efficiently compute a solution that
satisfies both requirements ! If we only have a
black-box quality measure, we cant do much
better than brute force search. (Bad !! Huge
search space.)
15
Efficient computation
Key for efficient computation decompose quality
measure so that we can make decisions more
independent of each other.
1. Step of decomposition quality of set of
chains combination of quality measure of
each chain separately
16
Decomposing the quality measure - Step 2
Transformation such that quality of a chain
combination of quality measure of each link
17
What does the decomposition mean?
First decomposition step (chains) Main
obstacle chains is a global property of a
solution Relation between chains
quality is not simple Second
decomposition step (links) Requires
additional assumptions -- Markov
transition probabilities -- transitivity
of visual similarity
Adding a chain can increase or decrease the
quality of the solution !
18
Transformation into a Linear Program
Decomposition, Transformation
max quality
exclusivity constraints
Weighted Assignment Problem
Special case of a Linear Program
19
Weighted Assignment Problem
Two roles for each observation interval predecess
or and successor
...
...
N

• 1. Mutual exclusiveness of chains
• in-degree of successor 1
• (predecessor real o.i. or NEW)
• out-degree of predecessor lt1
• 2. Minimize total link costs

1
1
N
2
2
NEW
N
3
3
Can be solved in cubic time via Munkres
algorithm. see OR courses...
20
Matrix representation
Matrix elements link terms
comparison pairs grows very big with time
21
Pruning comparisons Focus Sets
Observation links that are less likely than
will not be part of the optimal
solution Conservative estimate which links
are plausible candidates ? Use upper bound on
visual match quality relative time window
for each pair of cameras
NEW
Focus set set of plausible match candidates
22
Online Algorithm

• So far batch algorithm
• Not appropriate for monitoring systems !
• problem size becomes very big
• continuous output of answers desirable
• Problem we can prove that there is no online
  algorithm
• that returns the correct result for
  all inputs.
• Proposal Approximate online algorithm that uses
• limited, semantically meaningful
  lookahead.

23
Online Algorithm
1. Determine possible match candidates for A.
V
W
Focus set A
X
A
4. Fix As assignment
24
Evaluation
Evaluation with 8 minute 4-camera experiment
25
Results

• 2 out of 28 observation intervals are linked to
  wrong
• predecessor
• Additionally 2 links are missing for
  reoccurences
• after unnaturally long
  disappearance time
• small focus sets

26
Summary

• Introduced Multi-camera Path Reconstruction Task
• visual appearance matching aided by
• knowledge of traffic statistics
• Transformation into a Weighted Assignment
  Problem
• efficient computation
• Focus sets, online version
• Preliminary experimental results are promising

27
(No Transcript)
28
Performance of online algorithm
Fraction of runtime as compared to the batch
version
For size 100 10x faster
Error fraction 2
Problem size in observations