Adaptive Content-Aware Scaling for Improved Video Streaming.

1
Adaptive Content-Aware Scaling for Improved Video
Streaming.

• Avanish Tripathi
• Advisor Mark Claypool
• Reader Bob Kinicki

2
Outline

• Introduction
• Motivation
• Related Work
• Methodology
• Experiments
• Results
• Conclusions and Future Work

3
Motivation

• Internet disseminates enormous amounts of
  information
• TCP is the de facto standard but
• TCP is not ideal for multimedia and
• 77 of all Web traffic is Multimedia, of this
  about 33 is streaming content.
• Chandra, Ellis 99

4
Multimedia Flows

• tend to use UDP with no congestion control
• Other network protocols are being developed
• TFRC smooth reduction in rates as against abrupt
  drops in TCP Floyd et. al. 00
• RAP Architecture for delivery of layered
  encoded streams. Rejaie et. al. 99
• MPEG-TFRCP Mapping MPEG to TFRC Protocol
  Miyabayashi et. al. 00
• Idea-rate based with smooth increase and decrease

5
Multimedia issues

• Generally very high bandwidth requirements
• Random packet drop by routers during congestion
  is detrimental to perceptual quality due to
  interdependencies between packets
• Need application level solution

6
Media Scaling

• Need media scaling Application level data-rate
  reduction
• Scaling types
• Temporal
• Quality
• Spatial
• Content of the stream should influence the
  choice of scaling mechanism
• To the best of our knowledge this idea has not
  yet been employed

7
Related Work

• Quality Scaling Receiver-driven Layered
  Multicast McCanne 96
• Temporal Scaling
• Player for adaptive MPEG Streaming Walpole et.
  al. 97
• Better Behaved Better Performing MM networking
  Chung, Claypool 00
• Content based forwarding for differentiated
  networks use priorities based on MPEG
  characteristics Shin et. al. 00
• Filtering System used for media scaling of MPEG
  streams. Yeadon 96

8
Outline

• Introduction
• Motivation
• Related Work
• Methodology
• Experiments
• Results
• Conclusions and Future Work

9
Methodology Content-Aware Scaling

• Develop and verify motion measurement mechanism
• Define temporal and quality scaling levels
• Evaluate the potential impact of content-aware
  scaling
• Build system to do content-aware scaling
  adaptively
• Evaluate the practical impact of the full system

10
MPEG Overview

• Three kinds of pictures
• I- Intra encoding
• P- Predictive encoding
• B- Bi-directional predictive encoding
• Subdivided into Macroblocks
• Intra, predictive, interpolated macroblocks
• Motion vectors are used for motion compensation

11
Motion Measurement

• Higher percentage of interpolated macroblocks
  means low motion
• Lower percentage of interpolated macroblocks
  means high motion
• Conducted a pilot study to verify our hypothesis
• Divide frame into 16 sub-blocks
• Count the number of blocks that have motion
• Correlate that with the percentage of
  Interpolated macroblocks.

12
Pilot Study Result Motion Measurement
13
Motion Computation

• Keep latency low so that the system is
  sufficiently reactive

14
Methodology Content-Aware Scaling

• Develop and verify motion measurement mechanism
• Define temporal and quality scaling levels
• Evaluate the potential impact of content-aware
  scaling
• Build system to do content-aware scaling
  adaptively
• Evaluate the practical impact of the full system

15
Filtering

• We extend the system developed at Lancaster
  university
• Frame dropping filter (Temporal Scaling)
• Requantization filter (Quality Scaling)

16
User Study Details

• 22 graduate and undergraduate students in the
  department
• Platform
• 3 Pentium III machines with 128MB RAM running
  Linux
• Clips were on local hard drives
• Four 10 second clips (2 high motion, 2 low
  motion)
• Users rated the clips with numbers from 0 -100

17
User Study Details

• Five versions of each clip
• Perfect, Temporal Level 1, Temporal Level 2,
  Quality Level 1, Quality Level 2

18
Methodology Content-Aware Scaling

• Develop and verify motion measurement mechanism
• Define temporal and quality scaling levels
• Evaluate the potential impact of content-aware
  scaling
• Build system to do content-aware scaling
  adaptively
• Evaluate the practical impact of the full system

19
Results

• Four men sitting at a bar
• Low Motion ( 70 interpolated macroblocks)

20
Results

• A girl walks across a room while talking on the
  phone
• Low Motion (57 interpolated Macroblocks)

21
Results

• Rodeo scene where a man on horseback tries to
  rope a bull
• High Motion (27 interpolated macroblocks)

22
Results

• Car commerical
• High Motion (20 interpolated macroblocks)

23
Methodology Content-Aware Scaling

• Develop and verify motion measurement mechanism
• Define temporal and quality scaling levels
• Evaluate the potential impact of content-aware
  scaling
• Build system to do content-aware scaling
  adaptively
• Evaluate the practical impact of the full system

24
Full System Architecture
Quality Filter
Internet
High
Motion Measurement
MPEG
Feedback Generator
Server
Client
Input
Low
Temporal Filter
25
System Functionality

• Server is capable of quantifying motion as the
  movie plays
• The filtering system has five scale levels for
  finer granularity
• The system is adaptive and scales movies in
  real-time depending on the loss pattern as
  received from the feedback module

26
User Study

• Four clips (2 or more scene) 30 seconds
• Four versions of each
• Perfect Quality
• Temporal scaling
• Quality scaling
• Adaptive scaling
• Bandwidth distribution functions how often the
  rate changes
• Every 3 seconds
• Every 200ms
• Fit the scale values(1 through 5) on a normal
  curve Floyd 00

27
Future Work

• Accurately determine the threshold below which
  temporal scaling is unacceptable
• More accurate bandwidth distribution function
• Hybrid scaling methods (Quality Temporal)
• Audio Scaling

28
Conclusions

• Application level solution to the problem of
  congestion due to unresponsive video streams
• Developed a mechanism to quantify the amount of
  change in a video stream
• Shown that content aware scaling can improve user
  perceived quality by as much as 50
• Developed a system to do adaptive content-aware
  scaling and are in the process of determining it
  impact on user perceived quality