Radiometric Self Calibration

1
Radiometric Self Calibration

• Tomoo Mitsunaga
  Shree K. Nayar
• Hashimoto Signal Processing Lab. Dept.
  of Computer Science
• Sony Corporation
  Columbia University

CVPR Conference Ft. Collins, Colorado June 1999
2
Problem Statement

• How well does the image represent the real world?

3
Scene Radiance and Image Irradiance
Radiance
Irradiance
Image irradiance
Ideal camera response
Aperture area
Exposure
4
Scene Radiance and Measured Brightness
Video
Image Formation
Image Exposure
Camera Electronics
Digitization
CCD
Measured brightness M
Scaled radiance I
Scene radiance L
linear
Photo
Image Formation
Image Exposure
Film Development
Scanning
Film
f (M) The radiometric response function
5
Calibration with Reference Objects

• The scene must be controlled
• The reflectance of the objects must be known
• The illumination must be controlled

6
Calibration without Reference Objects

• Differently exposed images from an arbitrary
  scene
• Recover the response function from the images
• Calibrate the images with the response function

7
Previous Works

• Mann and Picard (95)
• Take two images with known exposure ratio R
• Restrictive model for f
• Find parameters a, b, g by regression
• Debevec and Malik (97)
• General model for f only smoothness constraint
• Take several (say, 10) high quality images
• At precisely measured exposures (shutter speed)

8
Obtaining Exposure Information

• We have only rough estimates
• Mechanical error
• Reading error (ex. F-stop number)

9
Radiometric Self-Calibration

• Works with roughly estimated exposures
• Inputs
• Differently exposed images
• Rough estimates of exposure values
• ex. F-stop reading
• Outputs
• Estimated response function
• Corrected exposure values

10
A Flexible Parametric Model
High order polynomial model
f (M)

• Parameters to be recovered
• Coefficients cn
• Order N

M
f(M) of some popular imaging products
11
Response Function and Exposure Ratio
Images q 1,2,.Q , Pixels p 1, 2, ..P
Exposure ratio
Using polynomial model
12
An Iterative Scheme for Optimization
Rough estimates Rq,q1(0)
Rq,q1(i)
Optimize for Rq,q1
Optimize for f
f (i)
Optimized f and Rq,q1
13
Evaluation Noisy Synthetic Images
f (M)
M
Solid Computed response function Dots
Actual response function
14
Evaluation Noisy Synthetic Images (contd)
Percentage Error in Computed Response Function
Trial Number
Maximum Error 2.7
15
Computing a High Dynamic Range Image

• Calibrating by the response function
• Normalizing by corrected exposure values
• Averaging with SNR-based weighting

16
Results Low Library (video)
Captured images
I
Calibration chart
M
Computed response function
17
Results Low Library (video)
Captured images
Computed radiance image
18
Results Adobe Room (photograph)
Captured images
I
M
Computed radiance image
Computed response function
19
Results Taos Clay Oven (photograph)
Captured images
I
M
Computed radiance image
Computed response function
20
Conclusions

• A Practical Radiometric Self-calibration Method
• Works with
• Arbitrary still scene
• Rough estimates of exposure
• Recovers
• Response function of the imaging system
• High dynamic range image of the scene

Software and Demo http//www.cs.columbia.edu/CAVE/
21
Obtaining Quality Measurements

• Automatic noise reducing pre-processing
• For random noise within a pixel
• Temporal averaging
• For object movement and risky object edges
• Selecting pixels from spatially static area
• For vignetting
• Preferring the center part of the image

Object edges are sensitive to noise