Image and Video Coding

1
(No Transcript)
2
Image and Video Coding

• Compressing data to a smaller volume without
  losing (too much) information

3
Why Code Data?

• To reduce storage volume
• To reduce transmission time
• One colour image
• 760 by 580 pixels
• 3 channels, each 8 bits
• 1.3 Mbyte
• Video data
• Same resolution
• 25 frames per second
• 33 Mbyte/second

4
Redundancy

• Spatial
• Correlation between adjacent pixels
• Chromatic
• Correlation between colour channels
• Temporal
• Correlation between adjacent frames
• Perceptual
• Unnoticed losses

5
Example
6
Redundancy - Consequences

• Data exceeds information
• More data than content justifies
• Can lose data without losing information

7
Compression Ratio
8
Lossy vs Lossless Compression

• Can lose data without losing information
• Lossily compressed images can look similar to the
  original
• Lossy compression has greater C

9
Quality of Decoded Images

• Measure differences between
• Original
• Coded/decoded images
• Options
• Maximum difference
• Average difference
• Subjective scales

10
Example
11
Subjective Quality Measurement Scales - Example

• 0 Unusable
• 1 Annoying degradation
• 2 Adequate images
• 3 Barely perceptible degradation
• 4 No observable degradation

12
Difference coding

• Adjacent pixels are similar
• Difference is small
• Uncompressed
• Compressed

13
Coding

• Assign 4 bits to a difference
• Can code 7, , 8
• Overflow?
• Use 7 and 8 to show larger differences
• Code 6, , 7 directly
• Use overflow codes to indicate shift of codes

14
(No Transcript)
15
Predictive Coding

• If signals are well sampled
• Adjacent samples are correlated
• They have similar values
• Differences are small
• Can guess next sample from value of current

16

• Constants are correlation coefficient and mean
  grey value
• Difference between real and predicted values are
  smaller
• Code as for difference coding

17
Run Length Coding

• Replace runs of equal brightness values by
• (length of run, value)
• 1 2 2 3 3 4 4 4 5 6 5
• (1 1) (2 2) (2 3) (3 4) (1 5) (1 6) (1 5)
• More use when few brightness values
• e.g. fax

18
Huffman Coding

• Uses variable length codes
• Most frequently occurring grey value has shortest
  code
• Least frequently occurring values have longest
  codes

19
Example
Symbol
Probability
2
1
3
4
A B C D E F
0.4 0.3 0.1 0.1 0.06 0.04
0.4 0.3 0.1 0.1 0.1
0.4 0.3 0.2 0.1
0.4 0.3 0.3
0.6 0.4
20
(No Transcript)
21
GIF

• Applicable to images with 256 colours
• Replace sequences of bytes with shorter codes
• Most common sequences use shortest codes

22
Wavelet coders

• Wavelet transform organises image content
  efficiently
• Can easily select data to be discarded

23
Wavelet coders
24
JPEG standard

• A subcommittee of ISO
• Optimised for a range of image subject matter
• Compression rates can be defined
• Quality inversely proportional to C

25
Block Transform Coding
26
Block Transform Encoding
27
Block Encoding
Original image
1260 -1 -12 -5 -23 -17 -6 -3
-11 -9 -2 2 -7 -2 0 1
139 144 149 153 144 151 153 156 150 155 160
163 159 161 162 160
DCT
quantise
79 0 -1 0 -2 -1 0 0 -1 -1 0 0 0 0 0
0
Zig-zag
79 0 -2 -1 -1 -1 0 0 -1 0 0 0 0 0 0 0
0 79 1 -2 0 -1 0 -1 0 -1 2 -1 0 0
run length code
10011011100011.
Huffman code
28
Block Transform Decoding
DCT
Zig-zag
quantise
run length code
010111000111..
entropy code
29
Result of Coding and Decoding
139 144 149 153 144 151 153 156 150 155 160
163 159 161 162 160
144 146 149 152 148 150 152 154 155 156 157
158 160 161 161 162
Original block
Reconstructed block
-5 -2 0 1 -4 1 1 2 -5 -1 3 5 -1
0 1 -2
errors
30
Discrete Cosine Transform
with
C(w)
31
Discussion

• Where are lossy steps?
• Quatisation and subsampling before coding
• How is quantisation matrix chosen?
• Its predefined by the standard after much
  experimentation

32
Video coding

• Many specific standards
• AVS, MOV, QT,
• One universal standard
• MPEG

33
MPEG Standards

• Standard specifies audio, video and system layers
• MPEG-1 low data rates, poor quality VHS quality
  at 1.5Mbits-1
• MPEG-2 high quality hence high data rates
  studio quality, 15Mbits-1
• MPEG-4 low data rates, small images, 64 kbits-1

34
MPEG-1

• Audio and video designed to work at CD ROM
  speeds 1.5Mbits-1
• Video 1.150Mbits-1
• Audio 0.256Mbits-1
• System 0.094Mbits-1

35
MPEG-2

• Released in 1994
• Aimed at digital TV, ATM.
• Additions for
• Interlaced video
• Scalable video coding
• Graceful degradation with noise
• Implementation of full standard impractical
• Varying profiles/levels of conformity

36
MPEG-4

• Coding specifically for multimedia objects

37
Coding Algorithms

• Frame sequence
• Motion compensation
• Frame coding

38
Frame Sequence

• I frames (Intraframes)
• Coded independently of any other frame
• P frames (Predicted frames)
• Derived from previous I frame by motion
  prediction
• B frames (Bidirectionally interpolated)
• Interpolate motion compensated blocks between I
  and P frames

39
Motion Compensation

• Image is divided into macroblocks (16 x 16
  pixels)
• Matching macroblocks are found by minimising
  differences
• Code differences and macroblock displacement

40
Frame Coding

• Use JPEG algorithms

41
Summary

• Why code data?
• Redundancy
• Assessment of compression
• Lossy vs. lossless compression
• Algorithms
• JPEG, MPEG

42

• But what is it good for?
• Engineer at the Advanced Computing Systems
  Division of IBM, commenting on the microchip in
  1968