Topics in Multimedia Computing and Networking

1
Introduction

• Topics in Multimedia Computing and Networking
• Ketan Mayer-Patel

2
The medium is the message.

• Marshall McLuhan (1967)
• Themes of the course
• In multimedia communication, how the message
  arrives is often as important as what the message
  is.
• The two are entangled.
• How the message arrives Networking
• What the message is Multimedia Encoding

3
Characterizing media data

• Continuous media is a bit more accurate.
• Periodic
• Robust
• Often large
• Correlated
• Endless?
• Video, audio, stock ticker?, depth?
• Compare with web pages.
• Web pages are definitely multimedia, but dont
  fit the characteristics of continuous media.

4
Video Taxonomy
Compressed
Uncompressed
Analog
Digital
Tape
Streaming
5
Audio Taxonomy
Compressed
Uncompressed
Speech-based
General
6
Network Basics
Throughput Amount of data arriving per unit
time.
Delay
Source
Dest
Interpacket Arrival Time
Loss
Jitter Variance of interpacket arrival times
7
Problems of Delay

• Interactive applications
• Video conferencing
• Human communication starts to break down at
  around 500 msec delays.
• Feedback loops
• Haptic interfaces
• Quality of Service
• Best effort and end-to-end mechanisms can do
  little to control delay.

8
Problems of Jitter

• Buffer management
• Buffers are used to smooth jitter, but how big
  should the buffer be?
• Unexpected jitter can lead to starvation.
• Over-provisioning buffers creates delay.
• Jitter is hard to bound even with QoS.
• Media jitter vs. packet jitter
• Variable bitrate media (where number of packets
  for unit of media varies) can make this
  relationship hard to model.

9
Problems of Throughput

• Estimating throughput is problematic.
• Media coding may be tunable, but generally
  changes in quality are perceptually annoying.
• Throughput will often vary faster than feedback
  delay which makes estimation hard.
• Multicast makes this worse.
• Scalable encoding is one solution path.
• QoS mechanisms are another.

10
Problems of Loss

• Effect of loss is very media and encoding
  specific.
• Loss of one packet in a video stream can result
  in the loss of multiple frames.
• Other packets received can be rendered useless.
• Goodput vs. Throughput
• Retransmission often not feasible.
• Encoding may be designed to alleviate packet loss
  effects.
• Forward Error Correction

11
Administrivia

• Who am I?
• kmp_at_cs.unc.edu
• On campus Mon. and Wed. (629 Soda)
• Who are you?
• Class web page
• http//inst.eecs.berkeley.edu/cs294-9
• Schedule of topics and readings posted here.
• Need to sign up for presenting papers.
• More on this next week.
• Reading for next week.
• Jim Blinns article on NTSC

12
TCP/UDP Review

• Transport protocols on top of IP
• TCP ordered, byte-stream, reliable
• UDP datagram, unreliable, unordered
• Programming API sockets
• Originally from UNIX, but available in Windows as
  WinSock and Java
• Network byte-order is big endian .

13
UDP Packet Format
32 bits
Src Port
Dest Port
Checksum
Length
Data
14
UDP Review

• Connectionless
• No communication necessary other than the
  datagram itself.
• Unreliable
• No guarantee or feedback whether data got ther or
  not.
• Unordered
• Packets may arrive in any order
• Limit on delay 1 minute (comes from IP)

15
TCP Packet Format
32 bits
Src Port
Dest Port
Sequence
Acknowledgement
Hdr Len.
Un-used
Flags
Window Size
Checksum
Urgent Ptr
Options
Data
16
TCP Lifecycle

• 3-way handshake
• Establishes connection.
• Data transfer
• Congestion controlled
• Reliable
• In order.
• FIN
• Connection teardown.

17
3-way handshake

• One side must be listening for a connection.
• Whoever is listening is the server.
• Client sends a SYN packet.
• SYN flag is set.
• Seq. is randomly chosen starting byte number.
• Server sends back SYN-ACK packet.
• SYN flag is set, ACK flag is set.
• Seq. is set and Ack reflects clients Seq.
• Client sends back ACK

18
Data Transfer

• Full duplex connection.
• Both sides can send data simultaneously.
• Seq. set to byte number of first byte.
• Ack set to next byte expected.
• Receiver generates an ACK even if it doesnt have
  data to send back.
• Some implementations delay sending ACK to
  optimize piggyback case.
• In general, implementations expected to generate
  one ACK for each data packet received.

19
Reliability

• Sender maintains data until acknowledged.
• This is done at the OS. The application is free
  from this responsibility.
• Sender estimates RTT from ACKs.
• Timeout values dynamically calculated from RTT
  and RTT variance.
• If timeout occurs, data retransmitted.
• Holes in the byte stream are possible.
• Receiver still generates ACK, but ACK reflects
  next byte needed in seq.
• Multiple ACKs used to retransmit quickly.

20
Transfer Example
Seq. 0
Seq. 10
Seq. 20
Seq. 30
Ack 10
Ack 10
Ack 10
Timeout
Seq. 10
Ack 40
21
Flow Control
Sender
Receiver
Application
Application
OS
OS
TCP
Last ACKd byte
TCP
First un-ACKd byte
Recv Buffer
Send Buffer
Last un-ACKd byte
Free Space
Recv. Window
Recv. Window is advertised to the sender in ACK.
22
Congestion Control

• Basic question how fast to send?
• Too slow underutilize network
• Too fast no one gets anything useful through
• Not responsive congestion collapse
• TCP congestion control is window-based.
• TCP maintains a window of data in the network.
• Successful transmissions allow window to
  increase.
• Packet drops cause window to shrink.
• Reliability feedback and congestion control
  coupled together (may not have been the best
  idea).

23
Slow Start

• Slow Start
• When starting, dont know what the window should
  be, so probe for capacity.
• Initial window 1 segment size
• A segment is a set number of bytes (i.e.,
  packet size).
• For every sucessful ACK, increase window by 1
  segement size.
• Doubles window size every RTT.

24
Slow Start Illustrated
Seq. 0
Ack 10
Seq. 10, 20
Ack 20, 30
Seq. 30, 40, 50, 60
Ack 40, 50, 60, 70
25
Congestion Avoidance

• Eventually slow-start results in a packet drop.
  This establishes ssthresh.
• Ssthresh set at 1/2 of current window.
• Current window shutdown to 1 segment.
• Data retransmitted.
• Slow-start begins again.
• When window gets back to ssthresh, every ACK
  results in window 1/window.
• Additive increase, multiplicative decrease.

26
AIMD Illustrated
Window Size
ssthresh
ssthresh
ssthresh
Time
27
Misc. TCP Oddities

• Window is actually minimum of receiver advertised
  window and congestion window.
• Duplicate ACKs used to detect congestion before
  actual timeout. (Fast retransmission)
• Special-case calculation of congestion window
  after fast retransmission to allow new data to be
  sent. (Fast recovery)
• Lots of tweaks and options (SACK, T/TCP)

28
Multimedia and TCP/UDP

• Streaming multimedia rarely uses TCP. Why?
• Retransmission is pointless
• By the time data is recovered, too late.
• AIMD congestion makes jitter and throughput
  changes unbearable.
• So, why did we just review TCP?
• Everything else in the world is TCP.
• How non-TCP traffic affects TCP traffic is an
  important question.
• In general, streaming media protocols built on
  top of UDP.