When Does a Camera See Rain?

1
When Does a Camera See Rain?
Kshitiz Garg Shree K. Nayar
Department of Computer Science Columbia
University
ICCV Conference October 2005, Beijing,
China Sponsors NSF, DARPA
2
When is Rain Visible?
Garg and Nayar, ICCV 05
Rain
Visibility of Rain Depends Strongly on Camera
Parameters ( Exposure time, Aperture size, Focus
distance )
3
Defining the Visibility of Rain
Intensity
Intensity
Pixel Intensity
Pixel Intensity
Time (Frames)
Time (Frames)
Visibility Variance due to a Volume of Rain
4
Intensity of a Rain Streak
Garg and Nayar, ICCV 05
Rain Region
Fog like
Background
background
Dependence on Exposure Time
zm
Rzm
Dependence on Defocus
Change in Intensity
Streak痴 width
Noise Level
Radius of blur circle
zm
Rzm
Distance
Dependence on Distance
5
Visibility and Volume of Rain
Garg and Nayar, ICCV 05
Background
Layer of Rain
Number of drops
Streak Width
Intensity Change
Variance
Scene Properties
Volume of Rain
Rain Properties
Camera Properties
6
Experimental Verification of Rain Visibility
Theoretical Visibility
Measured Visibility (with error bars)
Visibility
Visibility
Visibility
8
2
3
4
2
1
1
1
Standard deviation
Standard Deviation
0
2
4
6
8
10
0
10
20
30
40
50
0
40
80
120
(ms)
Distance of Focus Plane z0 (m)
F-number (N)
Exposure time
Camera Parametes
7
Optimal Settings for Rain Removal
Slow Motionlt 15 pixels/sec
Close z lt Rain Visible Region
Small Dz lt Rain Visible Region
8
Scene with Fast Motion
Default camera settings F-number 12 Exposure
time 8 ms Focus plane 100 m
9
Scene with Fast Motion
F-number decreased from 12 2.4 F-number
2.4 Exposure time 8 ms Focus plane 100 m
10
Comparison Scene with Fast Motion
Default camera settings
F-number 12 2.4
11
Scene with Large Depth
Default camera settings F-number 14 Exposure
time 16 ms Focus plane10 m
12
Scene with Large Depth
Exposure time Increased from 16 66
ms F-number 14 Exposure time 66 ms Focus
plane10 m
13
Comparison Scene with Large Depth
Default camera settings
Exposure time 16 66ms
14
Panning Video in Rain
Default camera settings Exposure time 16 ms
F-num14
Optimal camera settings Exposure time 33 ms
F-num6
15
Implications for Vision Algorithms
KLT feature tracker code from Stan Birchfield
Default camera settings Exposure time 16 ms
F-num14
Optimal camera settings Exposure time 33 ms
F-num6
16
Enhancing the Visibility of Rain Rain Gauge
Moderate Rain Video
Heavy Rain Video
Type of Rainfall Camera parameters (f, N, z0, T) Measured rain rate mm/hr Reported rain rate mm/hr
(a) Light (7095, 3.8, 3, 16) 1.34 0.763
(b) Moderate (5148, 4.4, 3, 4) 3.215 2.28
(c) Heavy (3300, 1.8, 3, 4) 14.28 11.17
17
Summary
Contributions
Analysis of Visibility of Rain
Camera Parameters for Removal of Rain
Camera Based Rain Gauge
Limitations
Not as effective in very heavy rain and for
scenes with close by and fast-moving objects
Future Work
A Camera-Lens system that automatically detects
rain in videos and sets optimal camera parameters
for reducing rain during image acquisition