MPEG4 Fine Grained Scalable Multi-Resolution Layered Video Encoding

1
MPEG4 Fine Grained Scalable Multi-Resolution
Layered Video Encoding

• Authors from University of Georgia
• Speaker Chang-Kuan Lin

2
Reference

• S. Chattopadhyay, S. M. Bhandarkar, K. Li,
  FGS-MR MPEG4 Fine Grained Scalable
  Multi-Resolution Layered Video Encoding, ACM
  NOSSDAV 2006.
• W. Li, Overview of Fine Granularity Scalability
  in MPEG-4 Video Standard, IEEE Trans. on
  Circuits and Systems for Video Technology, Vol.
  11, No. 3, pp. 301-317, Mar. 2001.
• H. Radha, M. van der Schaar, and Y. Chen, The
  MPEG-4 fine-grained scalable video coding method
  for multimedia streaming over IP, IEEE Trans. on
  Multimedia, vol.3, pp. 5368, Mar. 2001.

3
Outline

• Introduction
• MPEG-4 Fine Grained Scalability
• Motivation
• FGS-AQ vs. FGS-MR
• Experimental Results
• Conclusion

4
Introduction

• MPEG4 Fine Grained Scalability (FGS) profile for
  streaming video
• Base Layer Bit Stream
• must exist at the decoder
• has coarsely quantized DCT coefficients
• provides the minimum video quality
• Enhancement Layer Bit Stream
• can be absent at the decoder
• contains encoded DCT coefficient differences
• provides higher quality
• can be truncated to fit the target bit rate

5
FGS Encoding Block Diagram
6
Motivation

• Base Layer video quality is usually not
  satisfactory
• in order to provide a wide range of bit rate
  adaptation
• MPEG4 FGS Adaptive Quantization (FGS-AQ) for Base
  Layer video does not provide good rate-distortion
  (R-D) performance
• parameter overhead at the decoder
• Proposed FGS-MR
• no parameter overhead to transmit
• transparent the codec
• better rate-distortion performance

7
Outline

• Introduction
• MPEG-4 Fine Grained Scalability
• Motivation
• FGS-AQ vs. FGS-MR
• FGS-AQ
• FGS-MR
• MR-Mask Creation
• MR-Frame
• Experimental Results
• Conclusion

8
FGS Adaptive Quantization (AQ)

• Goals
• To improve visual quality
• To better utilize the available bandwidth
• Method
• Define different quantization step sizes for
  different transform coefficients
• within a macro-block (low freq. DCT coeff. gt
  small step size)
• for different macro-blocks (different
  quantization factors)
• Disadvantages
• R-D performance degrades due to FGS-AQ parameter
  overhead

9
Proposed Multi-Resolution FGS (FGS-MR)

• Goal
• To improve the visual quality
• To better utilize the available bandwidth
• No transmission overhead and hence maintaining
  the R-D performance
• Method
• Apply a low-pass filter on visually unimportant
  portion of the original video frame before
  encoding.

10
Two Equivalent Operations

• Apply a low-pass filter on the spatial domain of
  an image
• Truncate DCT coefficients in the corresponding
  transform domain of an image

11
FGS-MR Process (Step 1)

• MR-Mask creation
• Use Canny edge detector to detect edges
• Weight Mask
• an weight parameter wi, j for each pixel p(i, j)
  of an image, 0 ? wi, j ?1
• wi, j 1, if p(i, j) is on the edge
• 0 lt wi, j ?1, if p(i, j) is near edge
• wi, j 0, if p(i, j) is in non-edge region

12
Original (5.12Mbps)
13
MR-Mask
14
FGS-MR Process (Step 2)

• MR-Frame Creation
• VI (I-W) VL W VH
• VF Iteration( VI, G(sI))
• Note
• VI contains abrupt changes in resolution
• VF is a smooth version of VI

• Parameters
• Vo original video frame
• VL low resolution frame from the convolution of
  Vo and G(sL)
• VH high resolution frame from the convolution
  of Vo and G(sH)
• VI intermediate video frame
• VF final multi-resolution frame
• I matrix with all entries as 1
• W MR-mask weight matrix
• G(s) Gaussian filter with standard deviation of
  sas LPF
• sL gtsH

15
Original (5.12Mbps)
16
FGS-AQ (0.17Mbps, PSNR 22.77dB)
17
FGS-MR (0.17Mbps, PSNR 26.5dB)
18
Determine Parameters

• sL, sH, and sI
• to control the bit rate
• W (weight matrix)
• to control the quality of the encoded video frame
• Figure of merit function dQ/C
• Q 2( PSNR(sL, sH, sI)/10 )
• or PSNR 10log(Q)
• C compression ratio
• The authors empirically determine the parameters
• sL 15, sL 3, and varying sI

19
Outline

• Introduction
• MPEG-4 Fine Grained Scalability
• Motivation
• FGS-AQ vs. FGS-MR
• FGS-AQ
• FGS-MR
• Experimental Results
• Rate Distortion
• Resource Consumption
• Conclusion

20
Experiments

• Video 1
• 320x240, fps 30
• A single person walking in a well lighted room
• Video 2
• 176x144, fps 30
• A panning view across a poorly lighted room.
• No moving object

21
Rate Distortion Performance

• Vary sI from 3 to 25 to adjust the target bit rate

22
Power Consumption

• Energy used and hence power consumed by wireless
  network interface card (WNIC)

T time duration S data size b the bit rate of
streaming video B available BW ER energy used
by WNIC during data reception Es energy used by
WNIC when sleeping
23
Power Consumption Comparison
24
Conclusion

• The rate distortion performance of FGS-MR is
  better than FGS-AQ.
• FGS-MR can be seamlessly integrated into existing
  MPEG4 codec.
• My comment
• Processing time issue of FGS-MR
• Empirical determined filter parameters