Video Streaming

1
Video Streaming !!!

• By Mohsen Shaaban

2
Case Study
3
Use Case
4
OUTLINE

• Overview of Video Streaming
• Fundamental Challenges in video streaming
• Bandwidth
• Delay
• Losses
• Approaches to limit them.
• Video Transcoding
• Bit Rate Reduction
• Temporal Resolution Reduction
• Spatial Resolution Reduction
• Conclusion

5
Video Communication Environment

• Communication type
• Broadcast,
• Point-to-Point or
• Multicast.

6

• Real-time encoding/Pre-encoded (stored) video

Real-time encoding Interactive applications
(e.g. videophone, video conferencing). Also for
non-interactive applications Pre-encoded video
locally (DVD, VCD) or remotely (video server e.g.
VOD)

• Does not real-time encoding constraints
• Not flexible.

• Channel Characteristics (Bandwidth, Delay and
  Loss)
• Static ISDN
• Dynamic (time varying) wireless channels,
  Internet
• Packet-Switched/Circuit-Switched Network
• Internet PSTN/ISDN
• Quality of Service (QoS)/ Best Effort (BE)
• Internet

7
Video Streaming over Internet
8
Which Standard are Relevant for Video Streaming?
9
Use Case - Revisited
10
Use Case - Revisited
11
Video Streaming Concept

• Video Delivery
• Video delivery via file download
• Long time before playback
• Large storage space
• Video delivery via video streaming

1. Partition the compressed video into packets 2.
Start delivery of these packets 3. Begin decoding
and playback at the receiver while the video is
still being delivered
Packets
Encoder
Decoder
12
Challenges/Basic problems

• Bandwidth
• Delay jitter
• Loss

13
Time-varying Bandwidth

• Rate Control
• To match the video stream bit rate to the
  available Network transmission rate

14
Time-varying Bandwidth

• Rate Control
• To match the video stream bit rate to the
  available Network transmission rate

15
Bandwidth

• Transmission rate is lower than the media rate
• Short-term drop client buffering
• e.g. long 200 kbps bitstream through a 100 kbps
  channel
• Transcoding decoding the media then re-encoding
  with the desired bite rate.

16
Bandwidth

• Scalable/layered Compression

Rate Control at the Receiver side
Base Layer
- Coded with low bit rate
Quality degradation
- Coded with high bit rate
Inefficient
not flexible
- since some coding standards does not support
more than 3 layers.
Multiple Description Video Coding (MDC)
17
Bandwidth

• Multiple file switching

Video Server
Multiple copies of the same media coded at
different bit rates (e.g. common connection
speeds)

• Very little computation
• need to store multiple copies
• no adaptability to varying rates

18
Delay Jitter

• The end-to-end delay for a packet is fluctuating
  from packet to another.

Introducing jerks in video
Receiver
Playout Buffer
? Extends the presentation deadlines for all
packets. ? This allows retransmission when
packets are lost. ? Improve throughput
Additional storage at client Delay before
playback
19
Error Control for channel losses

• Channel Coding
• Retransmission
• Forward Error Correction (FEC)
• Source Coding
• Error Concealment
• Error Resilient

20
Channel Error Control

• Retransmission
• FEC

? Adapt to changes in channel conditions
Additional Delay (RTT) Require a back channel.
? No back channel / lower delay. Increase
transmission rate. Overhead even no losses.
21
Source Error Control

• Error Concealment
• Error control methods applied when an error
  already occurred (e.g. lost packet)

- Assume zero amplitude / black square in the
frame
- Spatial Interpolation / Intracoded frames
16x16 macroblock
- Temporal Interpolation / good with little motion

• Motion compensated temporal interpolation
• using the coded motion vector of that block.
• using neighboring motion vectors as an estimate.

22
Source Error Control

• Error Resilient
• Loss of bitstream resynchronization
• Decoder is not able to decode the bitstream due
  to a bit error (Huffman codes, VLCs)

• RVLCs
• Data partitioning (MPEG-4)

23
Video Transcoding

• Purpose of Transcoder
• Bit rate reduction
• SDTV 6Mbps ? 3Mbps, HDTV 19.2Mbps ? 11Mbps
• Frame rate reduction
• 30 frame/sec ? 10 frame/sec
• Spatial resolution reduction
• 720x480i, 30Hz ? 352x240p, 10Hz
• Syntax conversion
• MPEG-2 ? MPEG-4, Mobile devices

24
Use Case - Revisited
25
Transcoding Techniques
26
Transcoding Techniques
27
Bit Rate Reduction

• A straightforward way to reduce bit rate is
• Requantization
• Discarding the high frequencies DCT.
• Open Loop Architecture

new Bit Rate
VLD
VLC
IQ1
Q2
Drift Problem Where there is a mismatch between
the prediction/residual components in both the
Encoder/Decoder.
28
Bit Rate Reduction
29
Bit Rate Reduction
I
I
P
P
P
P
The error will drift from one P frame to
another The error will be terminated with an
Intracoded frame (I frame)
More I frames would make that architectures
efficient, but this will increase the bit rate
30
Bit Rate Reduction
Closed Loop Architecture
31
Temporal Resolution Reduction
Frame Dropping
32
Temporal Resolution Reduction
33
Spatial Resolution Reduction
Downsampling
(21) spatial resolution reduction
a. Pixel Averaging
- average of 2x2 pixels.
b. Filtering
- drop every alternate pixel in the horizontal
and vertical direction
16x16
8x8
C. DCT Decimation
16x16
4x4
8x8
DCT of 4 8x8 blocks
IDCT
4x4
34
Spatial Resolution Reduction
Motion Vector Mapping
one 16x16 MV
4 16x16 MB
41 mapping
11 mapping
four 8x8 MVs
a. Median
b. Moving with majority c. Average/mean d.
Weighted average

• 0.5 pixel MV refinement is carried out.
• final selected MV is scaled to desired reduced
  resolution.

35
Spatial Resolution Reduction
36
Spatial Resolution Reduction
Drift Error Analysis
reference
open-loop
Drift error
Quantization Error
Downsampling Error
37
Spatial Resolution Reduction
Closed-loop Architectures
With the Increase in the GOP size closed-loop
architectures undergo drift degradation. Such a
large GOP is used for Video Streaming and
wireless video that demand high coding
efficiency.
38
Spatial Resolution Reduction
Large drift error always correlates to
inter-coded blocks with large residual energy or
motion activity!!
39
Conclusion

• Video Streaming requires adaptation of content
  and robustness of transmission
• Video Streaming obstacles is in many sense video
  transmission problems.
• Low bit rate
• Rate Control
• R-D optimization
• Techniques that change the structure of the video
  data might be very effective for video streaming.
• Multiple layer/Scalable video coding
• Multiple Description video coding
• Video Streaming add more constraint to the
  network QoS issues which is the Delay Jitter.