MPEG2 FGS Implementation

1
MPEG2 FGS Implementation

• ECE 738
• Advanced Digital Image Processing
• Author Deshan Yang
• 05/01/2003

2
Introduction of FGS

• FGS fine granularity scalability
• For MPEG2 / MPEG4 and others
• Comparing to SNR, temporal, spatial scalability,
  FGS enhances video quality continuous instead of
  stepwise.
• Using bitplane coding for enhancement layer, base
  layer is still using regular MPEGx coding
• Enhancement layer bitstream can be truncated. The
  quality of video stream is proportional to the
  amount of enhancement bitstream data used.

3
Project outline

• Implement MPEG2 FGS enhancement into the current
  MPEG2 codec project from MPEG MSSG (MPEG Software
  Simulation Group).
• Implementation includes
• Updating the current MPEG2 codec source code to
  support FGS
• Implementing bitplane VL Codec and bit streaming
• Implementing a new FGS enabled MPEG2 player
• Challenges
• Understanding and using the MSSG MPEG2 codec
  source code (much more difficult than it seems to
  be)
• Bitplane coding / decoding
• MPEG2 player on Windows platform
• Media data to play with

4
Project outline (cont.)
5
Source code

• MPEG2 codec source code is fromMPEG Software
  Simulation Group (MSSG) at http//www.mpeg.org/MPE
  G/MSSG/
• MPEG2 player source code searching google.com
  for mpg2w11b.zip.
• My MPEG2 FGS player is combined from the the
  MPEG2 player with my MPEG2 FGS decoder MPEG2
  player source.

6
MPEG2 Encoder

• More configurable options to control FGS
  encoding.
• Integrating FGS enhancement layer bitplane
  encoder into the original MPEG2 encoder
• Encoding base layer in very low bit rate
• Calculate the DCT coefficients residues before
  and after the quantization step.
• Encoding the DCT coefficients residues with the
  bitplane VLC
• Encoded bitplane VLC output data are stored (as
  binary files) as the enhancement layer bitstream
• Supporting I-frame and P-frame, not B-frame

7
MPEG2 FGS Encoder Architecture
8
MPEG2 Decoder

• More configurable options to control FGS decoding
• Integrating the FGS bitplane VLD into the
  original MPEG2 decoder
• Accessing the enhancement layer bit stream (from
  binary files on disk)
• Decoding DCT residues from the enhancement bit
  stream
• Control the degree of enhancement
• Generating the enhanced DCT coefficients
• Decoding the enhanced DCT coefficients to get
  better video over base layer
• Supporting I-frame and P-frame, not B-frame

9
MPEG2 FGS Decoder Architecture
10
MPEG2 Player

• Combination of original MPEG2 player with the new
  MPEG2 FGS decoder
• New features
• Supporting FGS
• Replay
• Stop
• Stretching to window
• Supporting wide screen format
• FGS scale change on the fly
• Turning FGS on/off
• Player window resizing
• Supporting new command line options

11
DCT Coefficients

• Before quantization88,-14,0,0,-3,0,0,0,-40,-2,-2,
  0,0,0,0,0,-7,8,3,0,0,0,0,0,-5,0,0,0,0,0,0,0,0,3,0,
  0,0,0,0,0,0,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,
  0,0,0,2
• After base layer quantization88,0,0,0,0,0,0,0,-32
  ,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,-0,0,0,0,0,0,0,0,0,
  0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,
  0,0,0,0,0,1
• Difference0,-14,0,0,-3,0,0,0,-8,-2,-2,0,0,0,0,0,-
  7,8,3,0,0,0,0,0,-5,0,0,0,0,0,0,0,0,3,0,0,0,0,0,0,0
  ,3,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,4,0,0,0,0,1
• Sign bitsx,1,x,x,1,x,x,x,1,1,1,x,x,x,x,x,1,0,0,x,
  x,x,x,x,1,x,x,x,x,x,x,x,x,0,x,x,x,x,x,x,x,0,x,x,x,
  x,x,x,x,x,x,x,x,x,x,x,x,x,0,x,x,x,x,0
• Compressed sign bits1,1,1,1,1,1,0,0,1,0,0,0,0

12
Bitplane coding

• DCT coefficient differences0, -14, 0, 0, -3, 0,
  0, 0, -8, -2, -2, 0, 0, 0, 0, 0, -7, 8, 3, 0, 0,
  0, 0, 0, -5, 0, 0, 0, 0, 0, 0, 0, 0, 3, 0, 0, 0,
  0, 0, 0, 0, 3, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,
  0, 0, 0, 0, 0, 4, 0, 0, 0, 0, 0
• Bit planes
• 0 00000000000000000000000000000000000000000000000
  00000000000000000
• 1 00000000000000000000000000000000000000000000000
  00000000000000000
• 2 00000000000000000000000000000000000000000000000
  00000000000000000
• 3 00000000000000000000000000000000000000000000000
  00000000000000000
• 4 01000000100000000100000000000000000000000000000
  00000000000000000
• 5 01000000000000001000000010000000000000000000000
  00000000000100000
• 6 01001000011000001010000000000000010000000100000
  00000000000000000
• 7 00001000000000001010000010000000010000000100000
  00000000000000000
• 8 02002000222000002110000020000000010000000100000
  00000000000100000 (sign bits)
• Compressed sign bits 1,1,1,1,1,1,0,0,1,0,0,0,0
• Bitplane VLC encoding
• Planes 0,1,2,3 are all zero
• Plane 4 (1, 0, 0), (6, 0, 0), (8, 1, 0)
• Plane 5 (1, 0, 0), (14, 0, 0), (7, 0, 0), (33,
  1, 0)
• Plane 6 (1, 0, 0), (2, 0, 0), (4, 0, 0), (0, 0,
  0), (5, 0, 0), (1, 0, 0), (14, 0, 0), (7, 1, 0)

13
Bitplane coding (cont.)
Bit plane VLC coding Binary code
0 All zero 3F
1 All zero 3F
2 All zero 3F
3 All zero 3F
4 (1, 0, 0), (6, 0, 0), (8, 1, 0) 01, 06, 48
5 (1, 0, 0), (14, 0, 0), (7, 0, 0), (33, 1, 0) 01, 0E, 07, 61
6 (1, 0, 0), (2, 0, 0), (4, 0, 0), (0, 0, 0), (5, 0, 0), (1, 0, 0), (14, 0, 0), (7, 1, 0) 01, 02, 04, 00, 05, 01, 0E, 47
7 (4, 0, 0), (11, 0, 0), (1, 0, 0), (5, 0, 0), (8, 0, 0), (7, 1, 1) 04, 0B, 01, 05, 08, C7
Code format BPnnnnnn B End of block flag, P
End of plane flag, nnnnnn number of 0 before
1 3F00111111All zero plane,
BF0x10111111All planes are zero)
14
Bitplane VL Codec

• VLC
• Splitting DCT coefficients to bitplanes
• Converting each bitplane to VL codes
• Converting VL codes to binary codes (Huffman
  codes)
• Storing binary codes as enhancement layer
  bitstream (binary file)
• VLD
• Loading binary codes from enhancement layer
  bitstream (binary file)
• Converting binary codes (Huffman codes) to VL
  codes
• Converting VL codes to bitplanes
• Merging bitplanes to DCT coefficients

15
Enhancement Layer Bitstream
16
No FGS
15 FGS
65 FGS
30 FGS
30 FGS
17
Demo

• Run fgs.bat

18
Issues

• Enhancement layer bitstream size is much bigger
  than the base layer bitstream size.
• Motion prediction can not be applied to
  enhancement layer
• Current implementation only supports I-frame and
  P-frame, not B-frame yet
• Current implementation does not support Huffman
  VLC coding in the bitplane VLC. I dont have a
  good Huffman VLC table, and Huffman VLC will only
  help to reduce the enhancement bitstream size ¼
  anyway.