MotionCompensated Lifted Wavelet Video Coding

1
Motion-Compensated Lifted Wavelet Video Coding
2
Outline

• PART-1
• New Update Step for reduction of PSNR fluctuation
• Heuristics for implementing New Update Step
• Results
• PART-2
• Highly accurate distortion modeling
• Includes correlation between noise introduced in
  different temporal subbands
• Cursory results of modeling distortion in
  reconstructed video
• PART-3
• Spatial Intra Prediction incorporated in the
  lifting steps

3
PART 1 New Update Step
Minimizes total reconstruction distortion Girod,
Han 2004 i.e.
Additionally minimizes difference in distortion
in and i.e.
One needs to find appropriate for every
pair (X,Y)
4
Heuristic approximations for implementation
Tremendous amount of computation and storage
required
Heuristic 1)
are pretty close (at least for simple motion)
and always all positive

• Heuristic 2) Modified Barbell update
  incorporating
• Given value of
• Appropriate attenuation for every pixel update

5
Recall Optimal Update for Full Pel MC
Heuristic Rules for

• M-connected
• Weight
• Unconnected
• No update

Girod, Han 2004
For the rule becomes instead of

6
The idea of Barbell Update
A simple inversion of a many-to-many mapping
Prediction Update
Problem If a pixel is M-connected, it might get
inappropriate amounts of energy during update
7
Modified Barbell Update
Incorporate the weight in Barbell Update.
Prediction Update
For the case of Full-pel MC, implements
exactly
8
Results Quarter Pel MC
9
Results Full Pel MC
10
Full pel MC, GOP size 16
11
Full pel MC, GOP size 16, first few frames
12
PART 2 Highly accurate distortion modeling

• Original
• sequence

1
2
3
4
5
6
7
8
L1
L2
L3
L4
H2
H1
H3
H4
LL1
LL2
Distortion propagation through multiple levels
LH1
LH2
LLL1
LLH1
13
Previous work

• Additive models Distortion in any frame Linear
  combination of distortions in different temporal
  subbands
• e.g.
• Mavlankar 04, Chang 05

Can be at the most as accurate as the actual
additive distortion propagation
14
Distortion propagation from y0 in 16-GOF
Chang 05
15
Distortion propagation from y1 in 16-GOF
Chang 05
16
Distortion propagation from y8 in 16-GOF
Chang 05
17
Distortion Prediction in 16-GOF
Chang 05
18
MC-Lifted Haar Wavelet
Error propagation from temporal subbands to
video frames
Distortion propagation considering correlation
between noise introduced in different temporal
subbands
19
MC-Lifted Haar Wavelet (contd.)
i.e.
Signomial
where
always positive and significant in magnitude
20
How to determine the signomial coefficients?

• Problem
• After a single inverse decomposition step, the
  noise process becomes non-stationary.
• (Since during MC-P and MC-U different blocks are
  filtered using different filter kernels.)
• There are multiple (3 to 4) levels of inverse
  decomposition before reconstructing frames of the
  video.
• Is there really a way to determine approx.
  values of terms like
• without making any
  measurements involving
• ?

21
How to determine the signomial coefficients?

• Choose the approx. rate region
• Choose the approx. rate allocation
• Get for every inverse
  decomposition step in the hierarchy
• Calculate signomial coefficients for every
  inverse decomposition step in the hierarchy
• Assume that these signomial coefficients hold for
  the approx. rate region

22
To evaluate proposal on previous slide

• Determine signomial coefficients for one rate
  allocation and use these to predict the
  distortion for different rate allocations
• Compare performance against
• Actual observed MSE
• The theoretical limit to which any additive model
  can perform. i.e. actual additive MSE

23
GOP size 2 Frames
Signomial coefficients calculated using same rate
allocation
24
GOP size 2 Frames
Signomial coefficients from slide 23 Rates
changed Rate Allocation 1
25
GOP size 2 Frames
Signomial coefficients from slide 23 Rates
changed Rate Allocation 2
26
GOP size 2 Frames
Signomial coefficients from slide 23 Rates
changed Rate Allocation 3
27
GOP size 4 Frames
Signomial coefficients calculated using same rate
allocation
28
GOP size 4 Frames
Signomial coefficients from slide 27 Rates
changed Rate Allocation 1
29
GOP size 4 Frames
Signomial coefficients from slide 27 Rates
changed Rate Allocation 2
30
GOP size 4 Frames
Signomial coefficients from slide 27 Rates
changed Rate Allocation 3
31
GOP size 8 Frames
Signomial coefficients calculated using same rate
allocation
32
GOP size 8 Frames
Signomial coefficients from slide 31 Rates
changed Rate Allocation 1
33
GOP size 8 Frames
Signomial coefficients from slide 31 Rates
changed Rate Allocation 2
34
GOP size 8 Frames
Signomial coefficients from slide 31 Rates
changed Rate Allocation 3
35
GOP size 16 Frames
Signomial coefficients calculated using same rate
allocation
36
GOP size 16 Frames
Signomial coefficients from slide 35 Rates
changed Rate Allocation 1
37
GOP size 16 Frames
Signomial coefficients from slide 35 Rates
changed Rate Allocation 2
38
GOP size 16 Frames
Signomial coefficients from slide 35 Rates
changed Rate Allocation 3
39
Reduction of PSNR fluctuations

• We now have 2 tools
• Modified update step
• Fluctuation aware rate allocation following the
  temporal decomposition. Either
• a) Distortion in every frame described by a
  huge signomial. Is this a convex
  optimization problem ? Or
• b) Follow a simple algorithm described on next
  slide

40
Reduction of PSNR fluctuations

• Choose approx. for every decomposition
  step depending on unconnected pixels
• Allocate rate to LLL1. Equate the 2 distortions

and calculate the required . This
fixes the rate for LLH1. 3) Repeat same procedure
up the hierarchy for every single inverse
decomposition step.
41
PART 3 Spatial Intra Prediction
Intra Prediction Intra Update
Aim Make and more compressible !
42
Spatial Intra Prediction Modes

• Same modes implemented for 8x8 blocks
• Intra Prediction preferred only if reduction in
  SSD compared to best Inter-MV is above certain
  threshold

43
L and H are indeed more compressible
44
Spatial Intra Prediction

• More serious distortion propagation than
  Inter-Prediction.
• Gains at low rates reported due to savings on
  MV-rate.
• Wu, Woods 2004
• (Intra-Prediction has only 9 modes)
• Build this into criterion for ME and check R-D
  performance.
• - Wu, Woods 2004 does not have Intra-Update.

45
Summary

• New Update Step plus heuristics
• Highly accurate distortion modeling
• Future Research
• Above can be used now to get results with actual
  spatial encoding
• Possible gains due to spatial Intra Prediction