Context Enhancement of Nighttime Surveillance by Image Fusion

1
Context Enhancement of Nighttime Surveillance by
Image Fusion
Yinghao Cai Kaiqi Huang, Tieniu Tan and Yunhong
Wang
Center for Biometrics Research and
Testing National Laboratory of Pattern
Recognition Institute of Automation, Chinese
Academy of Sciences 2006-8-21
2
Outline

• Motivation
• Proposed Method
• Conclusions

3
(a) Nighttime Image
(c) Result of context enhancement
(b) Daytime background
4
Motivation

• Few work has been done on nighttime surveillance.
• Difficulties
• Low contrast
• Low signal to noise ratio
• Limited environmental information( context
  information)

5
Solution

• The camera is fixed.
• Capture scenes of day and night at the same
  viewpoint.
• Make use of the high quality background of the
  day to help enhance the context of nighttime
  images.

6
Questions

• What information should be preserved in the fused
  image?
• Moving objects
• Illumination effects
• Daytime background

Moving objects and illumination effects preserve
the fidelity of the important information of the
nighttime video
7
Framework
Nighttime image
Enhanced image
Nighttime image
Final image
Illumination segmentation
Daytime background
8
Enhancement of nighttime video

• A tone mapping function is employed to nighttime
  video enhancement Bennett, 2005.

Where is the pixel of the original nighttime
video, is the value of the enhanced video,
is a parameter.
9
Comparison with Gamma Correction
(a) Original video
(b) By Bennett, 2005 s method
(c) By Gamma correction
10
Motion Detection

• Gaussian mixture models Stauffer, 99.
• Real time motion detection.
• Robust to variations in lighting, moving scene
  clutter, multiple moving objects.

11
(b) Enhanced video
(a) Original nighttime video
(c) Motion detection of original video
(d) Motion detection of enhanced video
C. Stauffer and W.E.L.Grimson, adaptive
background mixture models for real-time tracking
, CVPR 99
12
Estimation of illumination characteristics

• The image I can be represented by the product of
  reflectance of
• the scene R, and illumination coming from the
  light source L.
• In Retinex theory, the illumination can be
  considered
• as the low frequency of image.
• In this paper, we represent the illumination
  characteristics of the
• nighttime image as the smoothed version of the
  original image.

Reflectance
Image
Illumination
13
Image fusion

• Where F is the final image, N is the nighttime
  image, D is the daytime background. W is the
  weight

Where M is the result of motion detection, L
is illumination characteristic. They are both in
the range 0,1.
14
Experimental Results
(b) Result of context enhancement
(a) Original nighttime image
15
Experimental Results
(a) Original nighttime video
(b) Result of context enhancement
16
Comparison with related work
Li , 2005s method
Our method
17
Conclusions

• Simple but effective.
• Provides a real time and robust solution to
  front-end image pre-processing in nighttime
  surveillance.
• The resultant image contains a more accurate and
  comprehensive description of the scene which is
  more useful for human visual and machine
  perception, especially in surveillance.

18
Thank you -)
yhcai_at_nlpr.ia.ac.cn