Critical Video Quality for Distributed Automated Video Surveillance

1
Critical Video Quality for Distributed Automated
Video Surveillance

• Pavel Korshunov
• Ooi Wei Tsang
• National University of Singapore

2
Introduction
978 cameras at 51MRT stations
3
Introduction

• Large scale video surveillance systems deployed
  on top of IP-network
• System of scattered low cost video sensors

Constraints on available bandwidth make video
streaming a challenging problem
4
Existing Systems

• Existing surveillance systems VSAM,KNIGHT, SfinX,
  etc.
• Focus on the design of computer vision algorithms
• PC is attached to each video source
• Stream full quality video or just images

The problem of bandwidth reduction is not
addressed
5
Introduction

• Suspicious
• events are rare

• Limited human
• participation

Attention from the guard is needed for very few
occasions
The guard is replaced by vision algorithms
Can we reduce the video quality for vision
algorithms?
6
Outline

• System Architecture
• Experiments
• Face Detection
• Face Tracking
• System Prototype
• Rate-Accuracy Function
• Conclusion

7
System Architecture
Bandwidth can be reduced at the link from camera
to proxy
8
Main Question

• How much can we reduce video quality if, instead
  of a human, the observer is a computer vision
  operation?

• Common operations
• Face Detection
• Face Tracking

Video Surveillance System
9
Experiments

• Study
• How does the performance of a vision algorithm
  decrease if video quality is reduced?

10
Experiments Face Detection

• Face detection algorithm
• Viola-Jones algorithm (OpenCV lib)
• Compression quality (IJG software)
• Test MIT/CMU still pictures dataset (507 faces
  with ground truth)

11
Index Value
JPEG Compression Quality (bpp)
No significant changes in performance until
compression quality decreases to 0.3 bpp (6 of
original image sizes)
12
Face Detection Critical Quality

• Compression quality 20 that gives 0.5 bpp is
  sufficient for Face Detection

13
Face Detection Lab Experiments

• 22.000 frames
• 237 faces 138 frontal

Consistent with MIT/CMU dataset results
14
Experiments Face Tracking

• Face tracking algorithm
• CAMSHIFT algorithm (OpenCV lib)
• Temporal and compression qualities (Microsoft
  Video 1 codec)
• Test several videos of a moving face

15
Face Tracking Dropping Pattern
Dropping Pattern Drop i out of ij
15 frames, 30 fps
i1, j1, i.e. 1 out of 2 gt 15 fps
i2, j2, i.e. 2 out of 4 gt 15 fps
i1, j4, i.e. 1 out of 5 gt 24 fps
16
Compression quality 100
Face Distance Ratio
i, drop gap
Drop Gap i is crucial measurement for temporal
quality
17
Face Tracking Critical Quality

• Compression quality 50 and temporal quality 6 fps
  is sufficient for Face Tracking

18
Prototype

• One network camera Canon VCC4 installed in the
  research lab
• One PC used as a proxy with running face
  detection or face tracking algorithm
• Another PC used as a monitor station

19
Alert and Observe Mode
Observe compression 20, frame rate 30 fps
Alert compression 90, frame rate 30 fps
Observe q 20
Alert q 90
Alert q 90
Observe q 20
20
Bit-rate Measurements
MJPEG video bit-rate
H.261 video bit-rate
Bit Rate (kbps)
Time (s)
Time (s)
21
Prototype Results

• Viola-Jones Face Detection (q 20, 5 fps)
• 29 times reduction for MJPEG
• CAMSHIFT Face Tracking (q 50, 6 fps)
• 16 times reduction for MJPEG

22
Rate-Accuracy Function
Given a desired accuracy for a vision algorithm,

• Detection index
• Face distance ratio

find video quality

• Compression
• Temporal
• Spatial

that minimizes the video rate

• Bit rate

subject to the environment conditions
23
Conclusion

• Video quality can be significantly reduced
• Reduction
• 29 times for Face Detection
• 16 times for Face Tracking
• Rate-accuracy function

24
Future Work

• Study spatial video quality
• Identify classes of vision algorithms that
  exhibit the same behavior
• Identify types of features used by vision
  algorithm that tolerate high compression

25
Future Work

• Can we design a new vision algorithm, which is
  more robust to low quality video?
• Can we design a new compression schema, which is
  optimized for vision algorithms?

26
The end!

• QA?