The Role of Perceptual Quality Measures in Video Compression

1
The Role of Perceptual Quality Measures in Video
Compression

• Jing Hu 06/15/04

2
Why Are Perceptual Quality Measures Important in
Video Compression?

• Goal of video compression algorithms
• - optimize the quality of compressed video
• - at a given bit rate
• - for a certain quality metric
• What is wrong with the popular metric Mean Square
  Error (MSE) ?
• MSE(Y) E (Y - Y)2
• - does not correlate well with perceived
  quality measurement
• - fails when one compares different kinds of
  artifacts, e.g., artifacts of block-based versus
  subband or wavelet coders

3
Current Measurement Methods for Video Quality

• Subjective test methods
• - require human viewers
• - costly, time-consuming and not real-time
• - accurate for any evaluation
• Conducted by Video Quality Experts Group (VQEG)
• Objective test methods
• - based on knowledge of Human Vision System
  (HVS)
• - tradeoff between latency and computational
  simplicity
• - may replace subjective test methods in some
  situations

4
Computational Models of Early Human Vision

• Optics Point-spread function
• Sampling in the retina
• - rods and cones
• - retinal ganglion cells
• Lateral geniculate nucleus
• Optic radiations
• - spatiotemporal filtering
• - parallel pathways
• Visual cortex

5
Psychophysics of HVS

• Amplitude Nonlinearity
• - uniform image and background
• - Webers Law ?I / I k 0.33
• Contrast Sensitivity Function (CSF)
• - sine-waves on uniform background
• - viewing distance matters
• Contrast Masking
• - image components with similar spatial
  locations and frequency contents
• Temporal Masking
• - scene change
• - temporal CSF

6
Image Quality Metrics ? General Models
Front end
Frequency analysis
Contrast sensitivity
Masking model
Error Pooling

• Frequency analysis decomposes the images into
  subbands with different spatial frequencies and
  orientations ?
• Baseline contrast sensitivity determines the
  amount of energy in each subband in order to be
  detected ?
• masking model luminance masking and contrast
  masking ?
• Error pooling
• - Minkowskis function

7
Image Quality Metrics ? Comparison of General
Models
8
Image Quality Metrics ? Coder-Specific Models
Front end
Frequency analysis
Entropy encoder
quantizer
Contrast sensitivity
Masking model

• Roles of the perceptual models
• - used independently to evaluate the performance
  of compression
• - incorporated to minimize the visibility of the
  quantization errors
• Safranek Johnstons perceptual image coder
  (PIC) and metric

9
Image Quality Metrics ? Coder-Specific Models(2)

• Watsons DCT-based metric
• - luminance masking threshold
• - contrast masking adjustment
• - Overall threshold
• - Error pooling ? quantization
• Other Image Quality Metrics
• - metrics designed for specific types of
  artifacts, e.g., blocking

10
Video Quality Metrics Set Up in the Last Decade

• General video quality metric model
• - By Van den Branden Lambrecht and Verscheure,
  1996
• - 5 spatial frequency bands, 4 orientation bands
  and 2 temporal bands
• - Metrics for image features
• Just Noticeable Difference (JNDmetrix) model
• - By Sarnoff Corporation, 1999
• Perceptual Distortion Metric (PDM) model
• - By Winkler, 1999
• - Compared three no-reference metrics on
  blocking effect

11
Video Quality Metrics Set Up in the Last Decade
(2)

• Digital Video Quality (DVQ) model
• - By Watson, 1998
• - Subject experiments to get thresholds for DCT
  coefficients
• - Subject experiments to simulate contrast
  masking
• In-service video quality monitoring
• - By Institute for Telecommunication Sciences
  (ITS), 1998
• - transmitting reduced-reference image quality
  information between coder and decoder

12
Our Research Plans

• Objective quality measurement of H.264 videos
• - Integer transform combined with quantization
• - Blocking effect
• General video quality measurement algorithms
• - Starting from unexplored image quality
  metrics
• Enhancing performance of H.264 standard
• - Inter/intra mode switching and macroblock
  (MB) prediction
• - Quantization step sizes
• - Entropy Coding Variable-Length Coding (VLC)
  and Context-based
• Adaptive Binary Arithmetic Coding (CABAC)