Improving Scene Cut Quality for Real-Time Video Decoding

1
Improving Scene Cut Quality for Real-Time Video
Decoding

• Giovanni Motta, Brandeis University
• James A. Storer, Brandeis University
• Bruno Carpentieri, Universita di Salerno

2
Outline

• Introduction
• H.263 and TMN-8 Rate Control
• Problem Description
• Optimal Algorithm based on Dynamic Programming
• Experimental Results
• Conclusions and Future Research

3
Introduction

• High variability in video sequences may cause the
  encoder to skip frames
• Frame skipping occurs after a scene cut (i.e.
  when MC-prediction model fails)
• If the encoder has some look-ahead capability it
  is possible to improve quality in proximity of
  scene cuts

4
H.263 Video Encoding

• State of the art Video Coding
• MC-prediction and DCT coding
• I and P macroblocks
• Rate control

5
TMN-8 Rate Control

• I/P Frame and MB decisions
• Target bit rate for each frame
• RD optimized bit allocation for MBs
• Buffer control

6
Problem Description

• Bits per frame (std100.qcif)

7
Problem Description

• PSNR and Bits per frame across a scene cut

8
Problem Description

• Frame n has several I macroblocks
• Encoder is forced to skip n1, n2, n3
• Frame n-1 frozen on receivers display
• Frame n4 has a large prediction error
• Encoder forced to skip frame n5

9
Basic Idea

• Avoid extra skipping and improve quality by
  selecting which frame should be encoded after a
  scene cut
• Assumption Encoder has look-ahead capability

10
Simplified approach
TMN-8 behavior
Last frame of the skipped sequence encoded
11
Simplified approach

• PSNR and Bits per frame across a scene cut

12
Optimal Algorithm

• Minimizes the number of skipped frames
• Generalization of the text-paragraphing algorithm
• Assumptions
• When the quality of Fi-j is fixed to Q, the
  cost Pi, j of predicting Fi from Fi-j, is
  independent of how Fi-j is encoded
• Pi, j ? Pi, j1 ? Pi, 0, 1 ? j ? d

13
Optimal Algorithm

• Compute Pi, 0 for each frame
• Compute Pi, j for 1 ? j ? d
• Build (right to left) two matrices
• Ri, j maximum residual capacity when
  Fi, , Fn are encoded so that the first frame
  that is not skipped is predicted by Fi-j
• Si,j number of skipped frames corresponding to
  residual capacity Ri, j
• Time is O(d2n) O(n) (constant d ? 7)

14
Test Sequences
Std and Std100 concatenation of standard test
sequences
Commercials Sampled TV commercials
15
Experimental Results

• Gain in Bit/PSNR in proximity of scene cuts
  (simplified method)

16
Experimental Results

• Gain on whole sequence (simplified method)

17
Conclusions

• Simple yet effective method to improve quality in
  proximity of scene cuts
• Experiments with simplified method show
  improvements of 14-30 (in Bit/PSNR)
• Suitable for encoders of the MPEG family,
  provided that encoder has look-ahead capability
• Decoding is unaffected

18
Future Research

• Assess quality improvement when using optimal
  algorithm
• Experiment with progressive transmission to
  eliminate frozen frame displayed by the decoder