Perception-motivated High Dynamic Range Video Encoding

1
(No Transcript)
2
Perception-motivated High Dynamic Range Video
Encoding
Rafal Mantiuk, Grzegorz Krawczyk,Karol
Myszkowski, Hans-Peter Seidel
3
High Dynamic Range
4
High vs Low Dynamic Range Video

• LDR Video
• Intended for
• existing displays
• Relative pixel brightness

• HDR Video
• Intended for
• the human eye
• Photometric or radiometric units cd/m2,
  Watt/m2sr

5
High Dynamic Range Video

• Goal Efficient encoding of full dynamic range of
  luminance perceived by the human observer

• 1st demo

6
Overview

• HDR Pipeline
• HDR Video Encoding
• Luminance Quantization
• Edge Coding
• Results
• vs. MPEG-4
• vs. OpenEXR
• Demo Applications

7
Related Work

• HDR Pipeline

Acquisition ? Storage ? Display
8
Related Work

• HDR Pipeline

Acquisition ? Storage ? Display

• Global Illumination
• HDR Cameras
• HDRC (IMS Chips)
• Lars III (Silicon Vision)
• Autobrite (SMal Camera Technologies)
• LM9628 (National)
• Digital Pixel System (Pixim)
• Technology overview Nayar2003

HDRC IMS Chips
9
Related Work

• HDR Pipeline

Acquisition ? Storage ? Display

• Still images
• Radiance RGBE Ward91
• OpenEXR Bogart2003
• logLuv TIFF Ward98
• HDR JPEG Ward2004
• Video
• No video format

10
Related Work

• HDR Pipeline

Acquisition ? Storage ? Display

• LDR Displays
• But Tone Mapping necessary
• HDR displays start to appear
• University of British Columbia Seetzen2004

11
HDR Encoding Framework

• Detail level 1 Input Output

bitstream
LDR
Video encoder
HDR
White MPEG Orange HDR Encoder
12
HDR Encoding Framework

• Detail level 2 Color Transform

LDR
bitstream
YCrCb
Color
Video
Transform
Encoder
L u'v'
HDR
p
White MPEG Orange HDR Encoder
13
HDR Encoding Framework

• Detail level 3 Edge Coding

DCT
Variable
Coding
length
LDR
bitstream
Color
Motion
Tran.
Comp.
HDR
Run-
Edge
length
Coding
White MPEG Orange HDR Encoder
14
Encoding of Color
15
Encoding of Color

• How to represent color data?
• Floating Points ineffective compression
• Integers ok, but require quantization
• How to quantize color data?
• Quantization errors lt threshold of perception
• Use uniform color space (Luv, Lab) Ward98
• Find minimum number of bits
• Color (uv) 8 bits are enough

16
Encoding of Luminance

• How to quantize luminance?
• Gamma correction?
• Logarithm?

8
6
log(Y)?
4
log Luminance Y
2
0
-2
-4
Integer representation
17
Threshold Versus Intensity

• Psychophysical measurements
• The smallest perceivable difference ?Y for a
  certain adaptation level YA
• tvi Ferwerda96, CIE 12/2.1

?Y
log Threshold ?Y
YA - Adaptation Luminance
log Adaptation Luminance YA
18
Luminance Quantization
Just below threshold of perception
Maximum quantization error
log Luminance Y
Integers Lp
19
Luminance Quantization
Just below threshold of perception
Maximum quantization error
log Luminance Y
10 11 bits are enough

• Capacity function Ashkihmin02
• Grayscale Standard Display Function DICOM03

Integers Lp
20
Luminance QuantizationsComparison
2
cvi
11-bit percep. quant.
32-bit LogLuv
0
RGBE
log Contrast Threshold
-2
-4
-2
0
2
4
-4
6
8
log Adapting Luminance
21
Edge Coding
22
Edge Coding Motivation

• HDR video can contain sharp contrast edges
• Light sources, shadows
• DCT coding of sharp contrast may cause high
  frequency artifacts

DCT coding
Edge coding
23
Edge Coding Solution

• Solution Encode sharp edges in spatial domain,
  the rest in frequency domain

?Run-length encoding
?DCT encoding
24
Edge Coding Algorithm
original
I
horizontal decomposition
edge block horiz. edges
II
horizontal DCT
III
vertical decomposition
edge block vert. edges
IV
vertical DCT
25
Edge Coding Algorithm
original
I
horizontal decomposition
edge block horiz. edges
II
horizontal DCT
III
vertical decomposition
edge block vert. edges
IV
vertical DCT
26
Edge Coding Algorithm
original
I
horizontal decomposition
edge block horiz. edges
II
horizontal DCT
III
vertical decomposition
edge block vert. edges
IV
vertical DCT
27
Edge Coding Algorithm
original
I
horizontal decomposition
edge block horiz. edges
II
horizontal DCT
III
vertical decomposition
edge block vert. edges
IV
vertical DCT
28
Edge Coding Algorithm
original
I
horizontal decomposition
edge block horiz. edges
II
horizontal DCT
III
vertical decomposition
edge block vert. edges
IV
vertical DCT
29
Results

• 2x size of tone-mapped MPEG-4 video
• 20-30x saving compared to intra-frame compression
  (OpenEXR)

Bit-stream Size
30
Demo Applications

• Display dependent rendering
• Choice of tone-mapping
• Extended postprocessing

31
Conclusions

• HDR video compression
• Modest changes to MPEG-4
• Lpuv color space
• Luminance quantization (10-11 bits)
• Edge coding
• Applications
• On-the-fly tone mapping
• Blooming, motion blur, night vision
• Tuned for display
• LDR / HDR Display

32
Acknowledgments

• HDR Images and Sequences
• Paul Debevec
• SpheronVR
• Jozef Zajac
• Christian Fuchs
• Patrick Reuter
• HDR Camera
• HDRC(R) VGAx
• courtesy of IMS CHIPSwww.hdrc.com

• Comments and help
• Volker Blanz
• Scott Daly
• Michael Goesele
• Jeffrey Schoner

33
Thank you
http//www.mpi-sb.mpg.de/resources/hdrvideo/