CIS679: Multimedia Basics

1
CIS679 Multimedia Basics

• Multimedia data type
• Basic compression techniques

2
Multimedia Data Type

• Audio
• Image
• Video

3
Audio

• Digitization
• Sampling
• Quantization
• Coding
• Higher sampling rate -gt higher quality
• Nyquist sampling theorem for lossless
  digitization, the sampling rate should be at
  least twice the maximum frequency responses
• Higher bits per sample -gt higher quality
• Sampling at 8 KHz, 8 bit samples -gt 64kbits/sec
• CD-quality audio
• Sampling at 44.1KHz, 16 bit samples -gt 705.6
  kbits/sec

4
Image/Video

• Digitization
• Scan a picture frame
• Digitize every pixel
• Color represented by RGB
• Normally converted to Y (black and white TV), U
  and V
• Luminance Y 0.30R 0.59G 0.11 R
• Chrominance U (B-Y) 0.493
• V (R-Y) 0.877

5
Video Transmission Standards

• NTSC
• Y 0.30R 0.59G 0.14B
• I 0.60R 0.28G 0.32B
• Q 0.21R 0.52G 0.21B
• PAL

6
Studio-quality TV

• NTSC
• 525 lines at 30 frames/second
• Y sampled at 13.5 MHz, Chrominance values at 6.75
  MHz
• With 8-bit samples,
• Data rate (13.5 6.75 6.75) 8 216 Mbps

7
Summary of Multimedia Data Types

• Audio data rate 64kbps, and 705.6kbps
• Video date rate 216 Mbps
• Compression is required!

8
Can Multimedia Data Be Compressed?

• Redundancy can be exploited to do compression!
• Spatial redundancy
• correlation between neighboring pixels in
  image/video
• Spectral redundancy
• correlation among colors
• Psycho-visual redundancy
• Perceptual properties of human visual system

9
Categories of Compression

• Lossless
• No distortion of the original content
• Used for computer data, medical images, etc.
• Lossy
• Some distortion
• Suited for audio and video

10
Compression Techniques
Run-length Coding Run-length Coding
Entropy Encoding Huflfman Coding Huflfman Coding
Arithmetic Coding Arithmetic Coding
DPCM
Prediction DM
FFT
Transformation DCT
Source Coding Bit Position
Layered Coding Subsampling
Sub-band Coding Sub-band Coding
Vector Quantization Vector Quantization
J PEG J PEG
MPEG MPEG
Hybrid Coding H.261 H.261
DVI RTV, DVI PLV DVI RTV, DVI PLV
11
Entropy Encoding Techniques

• Lossless compression
• Run-length encoding
• Represent stream as (c1, l1), (c2, l2),, (ck,
  lk)
• 1111111111333332222444444 (1, 10) (3, 5) (2,4)
  (4, 5)
• Or ABCCCCCCCCDEFGGG ABC!8DEFGGG
• Pattern Substitution
• Substitute smaller symbols for frequently used
  patterns

12
Huffman Coding

• Use variable length codes
• Most frequently used symbols coded with fewest
  bits
• Codes are stored in a codebook
• Codebook transferred with the compressed stream

13
Source Encoding Techniques

• Transformation encoding
• Transform the bit-stream into another domain
• Data in the new domain more amenable to
  compression
• Type of transformation depends on data
• Image/video transformed from time domain into
  frequency domain (DCT)

14
Differential/Predictive Encoding

• Encoding the difference between actual value and
  a prediction of that value
• Number of Techniques
• Differential Pulse Code Modulation (DPCM)
• Delta Modulation (DM)
• Adaptive Pulse Code Modulation (APCM)
• How they work?
• When consecutive change little
• Suited for audio and video

15
Vector Quantization

• Divide the data stream into blocks or vectors
• One or two dimensional blocks
• Use codebooks
• Find the closest symbol in codebook for a given
  sample
• Transmit the reference to that symbol
• Codebook present at sender/receiver
• When no exact match, could send the error
• Lossy or lossless
• Useful with known signal characteristics
• Construct codebooks that can match a wide range
  of symbols

16
Major Steps of Compression

• Preparation
• Uncompressed analog signal -gt sampled digital
  form
• Processing
• Source coding
• DCT typically used Transform from time domain -gt
  frequency domain
• Quantization
• Quantize weights into integer codes
• Could use different number of bits per
  coefficient
• Entropy encoding
• Lossless encoding for further compression

17
Conclusion

• Multimedia data types
• Why multimedia can be compressed?
• Categories of compression
• Compression techniques
• Entropy encoding
• Source encoding
• Hybrid coding
• Major steps of compression
• Whats next?
• JPEG
• MPEG