Evaluation of Video Quality

1
Evaluation of Video Quality
Video Quality Evaluation Techniques
Nitin Suresh and Dr. Nikil Jayant Multimedia
Communications Lab School of Electrical and
Computer Engineering Georgia Institute of
Technology, Atlanta, GA
Objective Testing Schemes
Comparisons of VQ Metrics PSNR and JND are
Full-Reference Metrics STJM is a
Partial-reference Metric AVQ is a No-Reference
Metric
Effectiveness of JND over PSNR

• Subjective
• Ideal approach but time consuming
• Involves people sitting through many tests
• Mean Time Between Failures an intuitive score of
  subjective quality that can be measured with ease
• Objective
• Automation of evaluation is desirable
• Many objective metrics have been researched
• Objective metrics are evaluated by correlating
  with subjective scores

• Examples of objective metrics
• PSNR uses just the difference between the
  processed and the original video
• JND, VQM incorporate some HVS properties into
  the metric

AVQ Metric
VQEG clips used
Testing Scheme

• The AVQ scores are expressed in terms of
  meaningful values like 'Mean Time Between
  Failures'. It can be scaled to different types of
  video content and quality appropriately
• Some of the desirable characteristics of AVQ are
  that it does not need the original signal as a
  reference, and it can be implemented in real time
  with low complexity
• The algorithm is to be filed as a utility patent
  by May 2007

Sample application of the AVQ meter
Significant Artifacts vs. Time
AVQ Metric

• The no-reference AVQ metric has been designed
  with new innovative algorithms based on the
  nature of the various kinds of artifacts observed
  in video transmission systems, and the knowledge
  of HVS characteristics
• A comprehensive understanding of current
  objective metrics and an understanding of the
  various strengths and weaknesses of these metrics
  has also helped in making AVQ very efficient