Compressing and Companding High Dynamic Range Images with Subband Architectures

1
Compressing and Companding High Dynamic Range
Images with Subband Architectures

• Yuanzhen Li, Lavanya Sharan,
• Edward Adelson
• Massachusetts Institute of Technology

2
Dynamic Range Problem
Source Shree Nayar
3
Part I Range Compression (Tone Mapping)
Display HDR image on 8 bit monitor
Too bright
Too dark
range compression (tone mapping)
an HDR image
after tone mapping
A multiscale subband/wavelet approach.
4
Halo Artifacts
Halos are widely believed to be inherent with
multiscale methods.
5
We fixed the halos
A subband/wavelet method that minimizes halos.
6
Halo Artifacts
Halos are widely believed to be inherent with
multiscale methods.
7
Part IICompanding (Compress and Expand)

• Compress HDR to LDR and expand to HDR

Expand
Compress
LDR, stored as 8-bit JPEG
Source Spheron
HDR, original
HDR, retrieved
8
High Dynamic Range Imaging

• Image acquisition
• Mann and Picard 95
• Debevec and Malik 97
• Nayar Mitsunaga 00
• HDR sensors
• Storage
• Ward and Simmons 04
• Various standards
• Display (hardware)
• Seetzen et al. 04

• Display (Tone Mapping)
• Tumblin Rushmeier 93, Tumblin Turk 99
• Chiu et al., 93
• Ward-Larson et al. 97
• Jobson et al. 97
• Pattanaik et al. 98, 00
• Socolinsky 00
• DiCarlo Wandell 01
• Durand Dorsey 02
• Fattal et al. 02
• Reinhard et al. 02, Reinhard Devlin 05
• Ashikhmin 02
• Johnson Fairchild 03

9

• Part I
• Range Compression
• (Tone Mapping)

10
Range Compression
Method Gamma or log on intensities. Problem
loss of detail.
Solution filtering. Problem halos.
Halos!!
lowpass

highpass

11
Multiscale Techniques

• Graphics community
• Jobson et al. 1997
• Pattanaik et al. 1998
• Tumblin and Turk 1999
• Dicarlo and Wandell 2001
• Patents
• Labaere and Vuylsteke 1998
• Vuylsteke and Schoeters 1998
• Lee 2001

12
How did we fix the halos?

• Analysis-synthesis subband architecture, e.g.,
  wavelets
• Smooth gain control on subbands

13
A Primitive Multiscale Architecture
Analysis filters
Non- linearity
Nonlinearity
Original signal
Reconstructed signal

14
Analysis-Synthesis Architecture
Synthesis filters
Analysis filters
Non- linearity
Nonlinearity
Original signal
Reconstructed signal

Why? They filter away trans-band distortions.
15
Multiscale Subband Decomposition
spatial frequency
orientation
lowpass residue
Choice of filters Wavelets, QMFs, Laplacian,
etc. They all worked.
16
Analysis-Synthesis Architecture
Synthesis filters
Analysis filters
Non- linearity
Nonlinearity
Original signal
Reconstructed signal

17
Analysis-Synthesis Architecture
Synthesis filters
Analysis filters
Non- linearity
Nonlinearity
Original signal
Reconstructed signal

18
Point Nonlinearity on Subbands
Original subband
Modified subband
point nonlinearity limits range
flattened peak
Problem Nonlinear distortion.
19
Smooth Gain Control
gain(x) b(x) / b(x)
smooth
x

20
Smooth Gain Control Reduces Distortion
Smooth gain control
Point nonlinearity
Distorted.
Distortion reduced.
21
Gain Control in Human Visual System

• Retina approximate by taking the log of
  intensities
• Contrast normalization in the visual cortex
  (V1)

A model of contrast normalization in V1
- Heeger 92
22
Smooth Gain Control on Subbands
rectify
blur
activity map
gain map
23
Fattal et al. 2002
Ours
Reinhard et al. 2002
Durand Dorsey 2002
24
Fattal et al. 2002
25
Ours
26
Reinhard et al. 2002
27
Ours
28
Durand Dorsey 2002
29
Ours
30
Fattal et al. 2002
Ours
Reinhard et al. 2002
Durand Dorsey 2002
31
Fattal et al. 2002
Ours
Reinhard et al. 2002
Durand Dorsey 2002
32
Fattal et al. 2002
Ours
Reinhard et al. 2002
Durand Dorsey 2002
33
Summary Range Compression

• Despite the common belief that multiscale methods
  are finicky and cause halos, weve demonstrated
  they CAN work quite well.
• How we did it
• Analysis-synthesis architecture, e.g., wavelets
• Smooth gain control on subbands

34

• Part II
• HDR Companding

35
Companding (Compress and Expand)

• Compress HDR to LDR and expand to HDR

Expand
Compress
8 bit LDR
HDR, original
HDR, retrieved
36
One Solution
Point nonlinearity
Intermediate LDR
log(xc)
exp(y)
HDR, original
HDR, retrieved
compress
expand

• Problems
• The intermediate LDR image doesnt look so good.
• Quantization artifacts.
• 
  

contouring
37
With contouring
38
Without contouring
39
Another Solution

• Ward Simmons 2004 save auxiliary information
  in a side channel
• But if we dont want any auxiliary information,
  can we still do it?
• Answer yes we can.

40
Review Range Compression
LDR subband
HDR subband
gain map
41
Inverting it?
LDR subband
HDR subband
42
Inverting it?
LDR subband
HDR subband
Problem Gain map unknown.
gain map
43
Inverting it?
LDR subband
HDR subband
Problem Gain map unknown.
Make a guess.
gain map
44
A decent guess, but not perfect
HDR subband
compute gain map
Error
LDR subband
-
compute gain map
Estimated HDR subband
45
HDR subband
compute gain map
LDR subband
-
compute gain map
Estimated HDR subband
decoding
46
encoding
HDR subband
compute gain map
LDR subband
-
Estimated HDR subband
47
Iterative Search
Iteration 2
Iteration 1
Iteration 3
Iteration 4
Range compressed LDR
Expansion error (log domain)
48
8bit to 3bit, then back to 8bit
Range compressed, 3bit per channel
Range expanded, From the 3bpc image
Original, 8bit per channel
49
12bit to 8bit, then back to 12bit
Close-ups Original Log
Ours
Compressed LDR
Expanded HDR from the 8-bit LDR image
50
Close-ups
Original Log
Ours
51
Companding with JPEG
Expanded from 4.0 bpp JPEG
Expanded from 1.7 bpp JPEG
Original 36 bpp
52
Summary Companding

• Companding (COMpress and exPAND)
• Using a multiscale (wavelet) architecture
• The LDR can be stored, transmitted, and viewed on
  LDR displays.
• It can be further compressed into JPEG.

exPAND
COMpress
HDR LDR HDR
53
Conclusions

• Range compression a multiscale method that gets
  halos under control.
• Companding compress HDR into LDR and expand it
  back into HDR.

54
Acknowledgements

• Fredo Durand, Erik Reinhard, Ranaan Fattal,
  Francois Sillion, Ramesh Raskar, Jack Tumblin
• Siggraph reviewers
• Supported by
• MURI, NSF, NTT, SHELL, NGA.

55

• Thanks!