RateAccuracy Tradeoff in Automated, Distributed Video Surveillance Systems

1
Rate-Accuracy Tradeoff in Automated, Distributed
Video Surveillance Systems

• Pavel Korshunov
• National University of Singapore

2
Introduction
The total number of cameras in the UK is around
4,000,000 London alone has 500,000
3
Introduction

• Large scale video surveillance systems deployed
  on top of IP-network
• Systems of scattered low cost video sensors

Due to bandwidth constraints, video streaming is
a challenging problem
4
Introduction
Suspicious events are rare
Limited human participation
Attention from the guard is rarely needed
Replace the guard with video analysis algorithms
Can we reduce the video bit rate for video
analysis algorithms?
5
Outline

• System Architecture
• Experiments
• Face Detection
• Face Tracking
• Rate-Accuracy Function
• Conclusion

6
System Architecture
Bandwidth can be reduced at the link between
camera and proxy
7
Main Question

• How much of the video bit rate can we reduce?

Video Surveillance System

• Example operations
• Face Detection
• Face Tracking

8
Experiments Detection, Tracking

• Face detection
• Viola-Jones algorithm (OpenCV lib)
• Compression, spatial qualities(IJG software,
  Bicubic algorithm)
• Test MIT/CMU dataset (507 faces with ground
  truth)
• Face tracking algorithm
• CAMSHIFT algorithm (OpenCV lib)
• Temporal, compression qualities(Microsoft Video
  1 codec)
• Test videos with moving faces

9
Face Detection Compression Quality
Index Value
JPEG Compression Quality
No significant changes in accuracy until compress
to 6 of original image sizes
10
Face Detection Spatial Quality
Index Value
Pre-scale to 70
Scale back
JPEG Compression Quality
Pre-scaling to 70 reduces average image size to
50
11
Face Tracking Dropping Pattern
Drop i out of ij
drop 1 out of 2 15 fps
15 frames, 30 fps
i1
j1
drop 2 out of 4 15 fps
drop 1 out of 5 24 fps
i1
j4
i2
j2
12
Face Tracking Temporal quality
Average Face Distance Ratio
i
Drop Gap i is crucial measurement for temporal
quality
13
Face Tracking Critical Drop Gap
Critical Drop Gap
JPEG Compression Quality
Compression quality does not affect accuracy
significantly
14
Prototype Results

• Viola-Jones Face Detection (quality 20, 5 fps)
• 29 times reduction for MJPEG
• CAMSHIFT Face Tracking (quality 50, 6 fps)
• 16 times reduction for MJPEG

15
Rate-Accuracy Function
Given a desired accuracy for a video analysis
algorithm,

• Detection index
• Face distance ratio

find video quality

• Compression
• Temporal
• Spatial

that minimizes the video rate

• Bit rate

subject to the environment conditions
16
How to find Rate-Accuracy Function

• Run experiments
• Change
• Video quality
• Video bit rate
• Observe the change in accuracy

Many video adaptations
Many video analysis algorithms
Too many experiments. Can we use analysis instead?
17
Example
Threshold on the speed of object

• Object tracking
• Dropping frames

Increases the speed of object
Speed Speed0/n
Rate Rate0/n
Accuracy (Speed
Sometimes we can derive rate-accuracy function
analytically
18
Basketball analogy
basketball
video
Ballsproperties
Video Features
Bouncinessof the ball
Accuracy of thevideo analysis algorithm
19
Rate-Accuracy function analysis

• Identify video features used by a given video
  analysis algorithm
• Estimate
• Combine estimations for all features

Changein feature
Changein accuracy
20
Rate-Accuracy function experiments

• Apply video adaptation and measure changes in
• Video feature
• Video bit rate

by
Changein feature
Changein accuracy
analysis
Video adaptation
Changein bit rate
21
Conclusion
?
by
?
Changein feature
Changein accuracy
?
analysis
Video adaptation
Changein bit rate

• Can we always identify video features used by a
  given algorithm?
• Can we always measure changes in a video feature
  caused by an adaptation?
• Can we always estimate the accuracy of a given
  algorithm using video features?