ECEC 453 Image Processing Architecture

1
ECEC 453Image Processing Architecture

• Lecture 8, February 5, 2004
• JPEG A Standard for all Seasons
• Oleh Tretiak
• Drexel University

2
Course Overview

• Image Compression Fundamentals
• Lossless
• Lossy
• Image Compression Standards
• Image
• Video
• Image Compression Architectures

Past
Present
Future
3
Review Decorrelation and Compression

• Tradeoff between distortion and bits per picture
• Correlation allows for greater compression
• More complex prediction gt greater compression

Review
4
Video as Sequence of Images

• Previous frame predicts current frame
• I(x, y, t) I(x, y, t-1) e(x, y, t)

x
x
time
Review
5
Predictive Coding of Video

• E(x, y, t) I(x, y, t) - P(x, y, t)
• I image, P prediction, E error
• P(x, y, t1) P(x, y, t) Code(E(x, y, t))
• At receiver, Ie(x, y, t) P(x, y, t1)
• Ie(x, y, t) estimate of image at time t

Review
6
Generic Encoder - simplified
Review
7
Generic Decoder
Review
8
Predicting sequential images
f(t-1)
f(t)
Review
f(t)f(t1)
9
Motion Compensation

• Macroblock size
• MxN
• Matching criterion
• MAE (mean absolute error)
• Search window
• p pixel locations
• Search algorithm
• Full search
• Logarithmic search
• Parallel Hierarchical One-Dimensional Search
• Pixel subsampling and projection
• Hierarchical downsampling

Review
10
Motion Estimation Methods
No compensation
Full search
logarithmic search
3 level hierarchical
Review
11
Comparison
Review
12
This Lecture
JPEG Classic
13
JPEG standard, AKA

• ISO/IEC international standard 10918-1
• ITU-T Recommendation T.81
• JPEG the Joint Photographic Experts Group
• Goals of JPEG
• General method, independent of picture source
• State-of-the art (as of 12 years ago)
• Allow tradeoff between compression and quality
• Low computational load, allow software
  implementation
• Both sequential and progressive
• Hierarchical

14
JPEG at a Glance

• Two decorrelation technologies
• DCT (lossy)
• Predictive coding (lossless)
• Two coder structures
• Huffman
• Arithmetic
• Four modes
• Sequential DCT
• Progressive DCT
• Lossless
• Hierarchical

15
JPEG in practice

• Lossy DCT coding used most often
• Some use of sequential DCT (Web images)
• Lossless? Hierarchical?
• JPEG DCT allows many varieties of parameters
• Choice of quantizer tables
• Choice of Huffman codes
• Arithmetic coding (ever?)
• Different encoding of color and luminance
• There are baseline (default) combinations of
  quantizers, Huffman codes, and luminance/chrominan
  ce samples

16
Block Diagram JPEG DCT
For color images, each component is coded
separately
17
Block Diagram JPEG DCT
For color images, each component is coded
separately
18
JPEG DCT Decoder

• The JPEG DCT coder is (almost) symmetric

19
JPEG DCT Decoder

• The JPEG DCT coder is (almost) symmetric

20
Features of JPEG coder

• Standard specifies compressed bitstream structure
  only
• File format not part of JPEG standard
• Many features of JPEG compression (e. g. coding
  tables) are optional
• Standard specifies input-output characteristics,
  not details of algorithms
• Either Huffman or arithmetic coding may be used
• Huffman coding almost universal many
  implementations do not include arithmetic coding
• Standard specifies baseline implementation. This
  is optional, but often followed.

21
DCT JPEG Major Components

• Color
• Data interleaving
• Chrominance subsampling
• Quantization
• Entropy coding
• DC coefficients
• AC coefficients

22
Color and JPEG

• Coding of color images is an essential part of
  JPEG
• Details of color coding are not part of JPEG
• Many color representation
• RGB
• YUV
• YCrCb
• CIELAB
• Need to represent multispcectral images
• A JPEG image consists of up to 255 color
  components
• Each color component can have different numbers
  of samples and different numbers of bits per
  sample
• Subsampling of color components is specified by
  the JPEG interchange standard, not the
  compression standard

23
Data Interleaving

• Image components can be stored (transmitted) with
  or without interleaving
• Without interleaving, each color component is
  stored as a unit
• First send all luminance blocks in image
• Then send all Cr blocks
• Finally send all Cb blocks
• Interleaving terminology
• Data unit (DU) smallest unit of data that can be
  processed, 8x8 pixels for DCT
• Minimum coded unit (MCU) smallest group of
  interleaved data
• For example if three color components have the
  same samping factor, an MCU consists of thee data
  units, one of each color.

24
Data Interleaving with Subsampling

• Example a color image with Y (intensity), Cb,
  Cr, (color) components is subsampled so that one
  color block corresponds to four Y blocks
• MCU1 Y00 Y01 Y10 Y11 Cr00 Cb00, MCU2 Y02 Y03
  Y12 Y13 Cr01 Cb01

25
The DCT

• 8x8 DCT only (no options)
• For 8 bit image data, subtract 128 before
  performing DCT
• This makes control of over/underflow easier.
• DCT DC coefficient in range of 1024
• AC coefficients in range 512
• Standard allows up to 12 bit per pixel

26
Quantization and Tables

• Quantization equationy DCT coefficient, q
  quantization step, z coded value
• All DCT blocks in a component have the same
  quantization table
• Different color components can have different (or
  same) quantization tables
• Typically, luminance and color components have
  different quantization tables
• qij 1 leads to (near) lossless coding
• Image quality can be varied wit an overall
  quality factor qij QFbij, bij baseline
  quantization steps, QF quality factor
• QF 1, high quality, larger QF -gt lower quality,
  higher compression

27
JPEG Baseline Quantization Tables
DC

• DC

Luminance
Chrominance
CCITT rec. T.81, Tables K.1 and K.2, p 143 147
28
Entropy Coding

• DCT coefficients are pre-processed before entropy
  coding
• DC coefficient is coded differently from other
  coefficients
• Non-zero coefficients are run-length coded
• Either Huffman coding or arithmetic coding may be
  used
• Arithmetic coding produces better performance
• Improvement is only a few percent over run-length
  coding
• Arithmetic coding algorithms are patented
  license fees!
• Coding tables must be included in bit stream
• Typically, baseline coding tables are used
• Restrictions
• No code may be longer than 16 bits
• 0xFF may not be a code word

29
Coding DC Coefficients

• DC coefficients are sent with differential coding
• zi DCi - DCi-1 (difference between quantized DC
  coefficients in two blocks)
• Baseline same coding as for lossless predictive
  (CCITT rec. T.82, Table K.3, page 149 153)
• Example Difference 195, range 8, value
  195-128 67 10000112. Assume that range is
  coded with 11110. The difference code is then
  11110 1000011
• For negative differences, value is represented in
  1-s complement. Difference -195, code
  111100111100

30
(No Transcript)
31
(No Transcript)
32
Coding AC Coefficients

• AC coefficients are coded in zig-zag (called ZZ
  in standard) order to maximize possible runs of
  zeros.
• Code unit consist of run lengthfollowed by
  coefficient size.
• Baseline coding of size category is the same as
  for DC differences (Table 2.9)
• Example run of 6 zeros, size -18. In the
  table, -18is in category 5. Code is(6/5,
  01101). If the Huffmancode for 6/5 is 1101,
  codeword 110101101

33
Coding AC special conditions

• Maximum run size is 15. A longer run is
  represented by 5/0 followed by remainder of run.
• Example a run of 24 zeros is followed by a 7.
  Symbols are 15/0 9/7
• After all nonzero coefficients have been
  processed, the special pair 0/0 is coded (EOB
  end of block).

34
Huffman Coding - Block Diagram
35
JPEG Coding Example
DC coding Assume DC(i-1) 39 DC(i) - DC(i-1)
42-40 3 Size 2, value 3 Assume Huffman code
for size 2 is 011 DC code is 01111

• AC coding
• Zig-zag scan
• 16 3 0 -2 -21 10 0 0 0 0 0 2 2 -15 0 -3 -1 0 0
  ...
• Run length codes (length/value)
• 0/16, 0/3, 1/-2, 0/-21, 0/10, 5/2, 0/2, 0/-15,
  1/-3, 0/-1, ....

36
JPEG Entropy Coding - more

• Run length codes (length/value)
• 0/16, 0/3, 1/-2, 0/-21, 0/10, 5/2, 0/2, 0/-15,
  1/-3, 0/-1, ....
• Run/size/amplitude (binary)
• 0/5/10000, 0/2/11, 1/2/01, 0/5/01010, ....
• Assume Huffman code for 0/5 is 11010 (JPEG
  baseline table)
• First code word is 1101010000
• Assume Huffman code for 0/2 is 01
• Second code word is 0111
• Assume Huffman code for 1/2 is 11010
• What is the third codeword?

37
Remarkable DCT Entropy Coding

• DC and AC coefficients treated differently
• DC is coded through inter-block differences
• Size/Amplitude coding (like lossless JPEG
  Category/Residual coding, Table 2.9)
• Zig-zag scan of AC, followed by run-value
  encoding
• Value converted to Size/Amplitude
• Combination of run length and size is
  Huffman-coded
• Unusual coding system (not off the shelf!)
• Responsible for much of JPEG DCT compression

38
Example of JPEG compression
Very high quality compression 2.33 Photoshop
Image
Very low quality compression 115 Produced by
MATLAB with Quality 0
39
More about JPEG

• JPEG Bible
• William B. Pennbaker and Joan L. Mitchell, JPEG
  Still Image Data Compression Standard, Van
  Nostrand Reinhold, 1993
• Contains ISO DIS 10918-1International Standards
  Organization Draft International Standard - not
  available on www - available on our web site
• Includes block diagrams of coding algorithms
• JPEG read and write capability included in
  MATLAB, Photoshop
• JPEG programs (sources, in C) are available on
  the web.
• JPEG 2000 is a new system. Includes wavelet
  compression