Noise Estimation from a Single Image

1
Noise Estimation from a Single Image
Ce Liu William T. Freeman
Richard Szeliski Sing Bing Kang
2
Parameter Tweaking in Computer Vision

• Computer vision algorithms suffer from hand
  tuning parameters for particular images or image
  sequences
• We want vision algorithms that behave properly
  under varying lighting conditions, blur levels
  and noise levels
• Our work is one step in that direction
• Given an image, estimate the noise level
• Modify vision algorithms to be independent of
  noise

3
Image Noise Is Important in Vision

• In image denoising the noise is assumed to be
  known as Additive Gaussian White Noise (AWGN)
• However, in real applications the noise is
  unknown and non-additive
• Many other computer vision algorithms also
  explicitly or implicitly assume the type and
  level of image noise
• Hard to make vision algorithms fully automatic
  without knowing noise

4
Noise Level Function (NLF)

• The standard deviation of noise s is a function
  of image brightness I
• Measurable by fixing the camera and taking
  multiple shots of a static scene
• For each pixel
• Mean I
• Standard deviation s
• NLF depends on camera, ISO, shutter speed,
  aperture
• Our goal is to estimate NLF from a single image
• How to estimate noise without separating noise
  and signal?

5
An Example Image
6
Piecewise Smooth Image Prior
Affine model
Patch
Standard deviation s
For each RGB channel
Brightness mean I
7
Piecewise Smooth Image Prior
Patch
8
Piecewise Smooth Image Prior
Patch
9
Segmentation-based Approach
Observed image
10
Segmentation-based Approach
Over-segmentation
11
Segmentation-based Approach
Signal
12
Segmentation-based Approach
Residual noise unmodelled image variation
13
Estimate NLFs

• Assume brightness mean I is accurate estimate
• Standard deviation s is an over-estimate (may
  contain signal)
• The lower envelope is the upper bound of NLF

14
Issues

• Should the curve be strictly and tightly below
  the points?

15
Issues

• Should the curve be strictly and tightly below
  the points?
• How to handle the missing data?

16
Issues

• Should the curve be strictly and tightly below
  the points?
• How to handle the missing data?
• Correlation between RGB channels?

17
Solutions

• Formulate the inference problem in a
  probabilistic framework
• Learn the prior of noise level functions

18
Outline

• Over-segmentation and per-segment variance
  analysis
• Learning the priors of noise level functions
  (NLF)
• Synthesize CCD noise
• Sample noise level functions
• Learn the prior of noise level functions
• Inference estimate the upper bound of NLF
• Bayesian MAP to estimate NLFs for RGB channels
• Applications
• Adaptive bilateral filtering
• Canny edge detection

19
Camera Noise
Shot
Dark Current
Camera
Noise
Noise
Irradiance
Scene
Lens
/
Radiance
L
Atmospheric
CCD Imaging
/
Fixed Pattern
geometric
Attenuation
Bayer Pattern
Noise
Distortion
Quantization
Thermal
Noise
Noise
Digital
Image
I
Interpolation
/
White
Gamma
A
/
D
t
Demosaic
Balancing
Correction
Converter

• Noise model
• Camera response function (CRF) f download from
  Columbia camera response function database (used
  196 typical CRFs)

Tsin et. al. Statistical calibration of CCD image
process. ICCV, 2001
20
Synthesize CCD Noise
I
21
Sample NLFs by Varying the Parameters
Camera response function (CRF) f
22
The Prior of NLFs
23
Likelihood Function

• The estimated standard deviation should be
  probabilistically bigger than and close to the
  true value
• Bayesian MAP inference

24
Validation (1) Synthetic Noise

• Add synthetic CCD noise, estimate, compare to the
  ground truth

ground truth estimated

25
Validation (2) Measure NLF of a Real Camera

• 29 images were taken under the same settings (the
  camera is not in the database for training)
• The real NLF is obtained by computing mean and
  variance per pixel

26
Validation (3) Robustness Test

• Verify that different images from the same camera
  give the same estimated NLF (camera not in the
  database for training)

27
Application (1) Adaptive Bilateral Filtering

• Bilateral filter is an edge-preserving low-pass
  filter
• Spatial sigma and range sigma
• Adaptive bilateral filter
• Down-weigh RGB values by signal and noise
  covariance matrices
• The range sigma is set to be a function of the
  estimated standard deviation of the noise

Input noisy image
Smoothing kernel
Denoised image
From Durand and Dorsey, SIGGRAPH 02
28
Test on Low and High Noise
29
ResultsAdaptive Bilateral Filtering
Standard bilateral filtering
Adaptive bilateral filtering
low noise
high noise
30
ResultsAdaptive Bilateral Filtering
Standard bilateral filtering
Adaptive bilateral filtering
Zoom in
high noise
31
Application (2) Canny Edge Detection
low noise
high noise
32
ResultsCanny Edge Detection
Parameters adapted in MATLAB
Parameters adapted by estimated noise
low noise
high noise
33
Conclusion

• Piecewise-smooth image prior model to estimate
  the upper bound of noise level function (NLF)
• Estimate the space of NLF by simulating CCD
  camera on the existing CRF database
• Upper bounds are verified by both synthetic and
  real experiments
• An important step to automate vision algorithms
  independent of noise

34
Thank you!
Noise Estimation from a Single Image
Ce Liu William T. Freeman CSAIL MIT
Rick Szeliski Sing Bing Kang Microsoft
Research