MPEG Video (Part 2)

1
MPEG Video (Part 2)
2
MPEG Encoding Revisit

• Overall MPEG bitstream organization.
• I-Frames
• Examples of many encoding techniques
• Subsampling (chrominance planes)
• Transform Coding (DCT, zig-zag)
• Run-length Encoding (AC coeffs)
• Predictive Encoding (DC coeffs)
• Entropy Encoding (Huffman encoding)
• Quantization (All coefficients)

3
P and B Frame Encoding

• P and B frames
• Motion compensation.
• Search techniques
• The problem with error measurements
• Skipped macroblocks
• Quantization control
• Variable bitrate vs. Constant bitrate
• DCT Artifacts
• Spider noise
• Blockiness

4
DCT Artifacts

• Spiders
• High frequency noise.
• Shows up most with diagonal edges.
• Blockiness
• Quantization noise in lower frequencies between
  macroblocks creates an artificial edge.
• Shows up a lot with smooth gradients.

5
P-Frames

• Two types of macroblocks in P-Frames
• I-Macroblocks.
• Just like macroblocks in a I-Frame
• DC term is differentially encoded from DC
  predictor
• AC terms
• RLEd as (run,value) pairs. Huffman encoded.
• P-Macroblocks

6
P-Macroblocks
Macroblock Type determines if Q Scale, Motion
Vector, or Block Pattern exist. One or all of the
blocks may be absent in a P-Macroblock.
7
Address Increment

• Each macroblock has an address.
• MB_WIDTH width of luminance / 16
• MB_ROW row of upper left pixel / 16
• MB_COL col. of upper left pixel / 16
• MB_ADDR MB_ROW MB_WIDTH MB_COL
• Decoder maintains PREV_MBADDR.
• MB address increment added to PREV_MBADDR
  provides current macroblock address.

8
MB Type

• Huffman coded.
• 7 possible codes (1 - 6 bits)
• Determine the following
• Intra or non-intra.
• Q scale specified or not.
• Motion vector exists or not.
• Block pattern exists or not.
• Not all combinations are possible.
• Not all possible combinations are feasible.

9
Quantization Scale

• 5 bits.
• Zero is illegal.
• Encoded as 1-31 which results in q-scale values
  of (2-62).
• Odd values impossible to encode.
• Decoder maintains current q-scale.
• If not specified, current q-scale used.
• If specified, current q-scale replaced.

10
Motion Vector

• Two components
• Horizontal and vertical offsets.
• Offset is from upper left pixel of macroblock.
• Positive values indicate right and down.
• Negative values indicate left and up.
• Offsets are specified in half pixels.
• Motion vector is used to define a predictive base
  for the current macroblock from the reference
  picture.

11
Motion Vector Illustrated
P-Frame
Previously Decoded I- or P- Frame
Prediction base does not have to be macroblock
aligned. If predictive base is half-pixel
aligned, bilinear interpolation is used. Whatever
luminance pixels are picked out, corresponding
chrominance pixels used to form chrominance
prediction.
12
P-Frame Encoding
13
Motion Vector Encoding

• If no motion vector is present, then motion
  vector is understood to be (0,0).
• Horiz. component followed by vertical.
• Decoder maintains motion vector predictor.
• Set to 0,0 at beginning of picture or slice or
  whenever an I-macroblock is encountered.
• Difference between predictor and value is Huffman
  encoded.

14
Predictive Base

• P-Macroblocks always specify a predictive base
• Either motion vector picks out an area, or
• No motion vector implicitly implies 0,0 (i.e.,
  predictive base is same macroblock in reference
  frame.)

15
Block Pattern

• The goal of motion compensation is to find
  predictive base that matches most closely with
  macroblock.
• If match is really good, then no appreciable
  difference will need to be encoded at all.
• Block pattern indicates which blocks have enough
  error to warrant coding.
• Absence of block pattern indicates no blocks
  needed coding.

16
Block Pattern Encoding

• Huffman code
• 3-9 bits
• Encodes values 1-63
• 0 not needed because then pattern wouldnt be
  encoded.
• Each bit in the block pattern value corresponds
  to a particular block

5
4
3
2
1
0
V
U
Lower right luminance
Lower left luminance
Upper right luminance
Upper left luminance
17
Block

• Difference between pixels in prediction and
  macroblock is encoded as block
• 9-bit input values
• Still produces 12-bit coefficients
• Sometimes called error blocks.

18
P-Frame Review

• Macroblocks are either I-macroblocks or
  P-macroblocks.
• I-macroblocks just like macroblocks in
  I-frame.
• P-macroblocks define predictive base and encode
  the difference.

19
Skipped Macroblocks

• If P-macroblock has (0,0) motion vector and no
  appreciable difference to encode, then can be
  skipped altogether.
• Skipped macroblock detected when address
  increment for next coded macroblock is detected.

20
Decoder State Updates

• DC predictors are reset whenever a
  P-macroblock or skipped macroblock is
  encountered.
• Motion vector predictors reset whenever
  I-macroblock is encountered.

21
B-Frames

• B-frames have 4 macroblock types
• I-macroblocks
• P-macroblocks
• Predictive base specified from previous reference
  frame.
• B-macroblocks
• Predictve base specified from subsequent
  reference frame.
• Bi-macroblocks.
• Predictive base specified from both reference
  frames.

22
B-Frame Macroblock Structure
Macroblock Type determines if Q Scale, Motion
Vectors, or Block Pattern exist. One or all of
the blocks may be absent in a P-, B- or
Bi-macroblock.
23
Two Motion Vectors

• Separate predictors are maintained for both
  components of both the forward and backward
  motion vectors.
• Reset whenever it doesnt get used.
• If both are specified, then predictive base is
  formed by bilinear interpolation of both
  prediction areas.
• Error encoded in blocks as per P-macroblocks.

24
(No Transcript)
25
Putting it all together
Input Buffer
VLC Decoder
ZZ and DeQuant
IDCT
Forward MC
Prev. Frame Store
Adder
Display
Bi- directional MC
Future Frame Store
Backward MC
26
Motion Compensation

• Provides most of MPEGs compression.
• Relies on temporal coherence.
• Finding a good motion vector essentially a search
  problem.
• Evaluating how goodness of a motion vector can be
  a bit tricky.
• MC is what makes MPEG asymmetric.
• Harder to encode than to decode.

27
Error Measurements

• Regardless of search algorithm, need to determine
  which motion vector is best.
• Simple measures
• Mean Squared Error
• Mean Absolute Error
• Minimum Difference Variance
• Fundamental problem is no good correlation
  between any simple metric and perceptual quality.

28
VBR vs. CBR

• Two ways to handle bitrate
• Variable Bit Rate (VBR)
• Allows compressed bitrate to vary
• Constant Bit Rate (CBR)
• Bitrate constant over some averaging window.
• MPEG buffer model.
• Optional (dont have to use it).
• Provides in the sequence header parameters to a
  buffer model that can describe bitrate behavior.

29
VBR Q-scale adjustments

• In general, VBR used to maintain quality.
• Q scale is adjusted to provide maximum
  compression given quality limit.
• Need some metric for quality.
• Same issue for judging perceptual quality crop up
  here.
• Common solution q scale statically set for I-,
  P-, and B-frames.
• A variation on this is differentiating among
  macroblock types.

30
CBR Q-scale adjustments

• To achieve CBR, q-scale used to control bitrate.
• Higher q-scale provides better compression at the
  expense of quality.
• Lower q-scale provides better quality at the
  expense of compression.
• Algorithms for controlling how q-scale is
  adjusted can get pretty complicated.
• Common solution is to have target I, P, and B
  frame sizes and then adjust q-scale as
  macroblocks are encoded to hit the target.