Multidimensional signal processing

1
Multidimensional signal processing

• Lecturers
• Thomas Moeslund
• Hans Jørgen Andersen (mm3)
• Course information
• Schedule and changes, see course web-site
• Literature Masters online or at the secretarys
  office
• Rikke Dyg Klemmesen A5-210
• Exercises Mini project
• Evaluation Mini project (free study activity)
• Comments and questions tbm_at_cvmt.dk

2
Purpose of this course

• Multidimensional signal processing
• Signals 2D and/or large amount of data
  (similar)
• To learn about
• Compression of large amount of data
• Reduce the dimensionality of high dimensional
  data sets
• Signal processing (pattern recognition) in 2D
• Different backgrounds

3
Mini module 1

• Purpose Compression of (image) data
• Data 300x300x8 720Kbit
• Color 720x3 2.16Mbit
• Standard modem 56.6Kbit/s
• Raw download time 12.7 second and 38.2 second
• How can you do it in 1 second?
• Describe different compression methods

4
Mini module 2

• Purpose Describe one of the core technologies in
  (image) data compression transform coding,
  especially the KLT method
• KLT is seldom applied even though it is an
  optimal method in (image) compression
• Applied allot to reduce the dimensionality of
  data sets (gt100 dimensional) by removing noise
  KLT gt PCA

5
Mini module 3

• Orthogonal Regression
• Estimation of several lines in incomplete
  datasets using expectation-maximization EM
• Fisher Transformation of data contra PCA

6
Mini module 4

• Purpose Video compression
• Video 2.16Mbit/image x 30 image/s 64.8Mbit/s
• Motion picture 90min 64.8Mbit/s x 60 x 90
  349.92Gbit
• 56.6K modem gt Raw download time (excl. sound)
  1717 hours or 72 days!!!
• The ultimate compression task?
• Describe different compression methods
• Incl. MPEG (popular due to DVD and DTV)

7
Mini module 5

• Student presentations of miniprojects

8
Exercise

• One mini (micro) project
• To be presented by each group at the last mini
  module
• Task design and implement an algorithm
• that can compress images with at least a factor
  10 without losing significant image quality
• Topic of project mainly related to the first 2
  mini modules

9
Mini module 1Compression of (image) data

• Use images as case lots of data and visually
  appealing
• Methods are general and
• therefore also suitable for
• other signal types
• Agenda before the break
• Redundancies, block diagram, differential coding,
  quantization, entropy coding
• Agenda after the break
• JPEG
• Exercise

10
Concepts

• Lossless compression
• Astronomy, medicine gt details
• Lossy compression
• normal images gt overall
• An image 2D array of pixels (rows, columns)
• A pixel value
• Greyscale/intensity 8 bits
• Color 3 x 8 bits Red, Green, Blue R,G,B
• Data
• Greyscale 300x300 90.000 pixels 90Kbyte
  720Kbit
• Color x3 270Kbyte 2.16Mbit

11
Why is it possible to compress images?

• Data ! information/knowledge
• Data gtgt information
• Key idea in compression only keep the info.
• But why is data ! info? Answer Redundancy
• Statistical redundancy
• Spatial redundancy and coding redundancy
• Phychovisual redundancy
• Greyscale redundancy and frequency redundancy

12
Spatial redundancy

• Pixel values are not spatially independent
• High correlation among neighbour pixels

13
Coding redundancy

• Redundancy when mapping from the pixels (symbols)
• to the final compressed binary code (Information
  theory)
• Example
• Lavg,1 3 bits/symbol
• Lavg,2 4x0.12x0.20.54x0.053x0.15 1.95
  bits/sym.
• Code 2 is also unique and shorter

14
Phychovisual redundancy

• The end-user is a human gt only represent the
  info. which can be preceived by the Human Visual
  System (HSV)
• From a datas point of view gt lossy
• From the HSVs point of view gt lossless
• Differential sensitivity

15
Intensity redundancy
I1
I2

• Webers law DI / I1 constant
• DI I1 I2, where just recognizable
• The high (bright) values need
• a less accurate representation
• compared to the low (dark) values
• Webers law holds for all
• human senses!

Sound
I1
DI
I2
Noise level
I1
DI
I2
16
Frequency redundancy

• Human eye functions as a lowpass filter gt
• High frequencies in an image can be ignored
  without the HVS noticing
• Key issue in lossy image compression
• Now we know why image compression is possible
  Redundancies
• Investigate how to implement these redundancies
• in algorithms

17
Block diagram of (visual) coding

• Purpose communication or storage system
• Coder (en)coder decoder codec
• Source encoder removes redundancy
• Channel encoder adds redundancy
• A/D, D/A, en/decryption optional
• Only deal with the source coder

18
Source coder block diagram
Coded bit-string

• Transformation new representation of data
• Differential coding, transform coding (MM2)
• Quantization In-reversible process gt lossy
  coding
• Codeword assignment (entropy coding) Info.
  Theory
• Huffman-, run length-, arithmetic-, dictionary
  coding

19
Differential coding

• AKA predictive coding
• A transformation based on
• spatial redundancy (vis)
• Difference signal
• signal predicted signal gt
• d(i) z(i) pz(i)
• Simple version
• pz(i) z(i-1) gt d(i) z(i) z(i-1)
• Low variance on the differential
• signal gt compact representation
• -1, 0, 1 gt 42

20
Quantization

• Reduce the number of different values in the
• transformed signal
• Well known from A/D-converters
• Round off to nearest allowed value
• For example differential coder with advanced
• prediction gt value -1.32 etc. Lots of
  different values
• Step size larger step gt fewer values but bigger
  error
• Uniform quantization (underlying assumption)
• When input is not uniformly distributed (e.g.
  differential signal) gt non-uniform quantization

21
Codeword assignment (entropy coding)

• After transformation and quantization gt source
  symbols s1, s2, s3,, sn
• The symbols need to be represented by bits
• Remove the redundancy in the symbols (lossless)
• Methods Run length, Huffman, arithmetic,
  modifications, dictionary (LZW zip, gif, tiff,
  pdf,..)
• Quick introduction to run length and
• Huffman coding

22
Run length coding

• Code 7,7,7,7,7,13,90,9,9,9,2,1,1,0,5, 15 Byte
• RLE 5,7,13,90,3,9,2,2,1,0,5, 11 Byte
• How to distinguish between values and counts?
• One value of a byte to indicate a count, e.g. 0
  or 255
• 255 255,5,7,13,90,255,3,9,2,255,2,1,0,5, 14
  Byte
• One byte to indicate count 1 and value 0 for
  8 values gt
• 10001001,5,7,13,90,3,9,2,2,0001,0,5.. 12,5
  Byte

23
Huffman coding

• Arrange symbols p(s1) gt p(s2) gt gt p(sn)
• li length in bits of codeword si
• Key idea use fewer bits to code the most likely
  symbols l1 lt l2 lt l3 lt lt l n
• Algorithm
• Combine the two symbols with lowest probabilities
  into a new symbol
• Assign one bit
• Re-arrange symbols

24
Huffman coding

• Algorithm
• Combine the two symbols with lowest probabilities
  into a new symbol
• Assign one bit
• Re-arrange symbols

25
The rest of today
Coded bit-string

• More on transformation MM2
• JPEG
• Exercise

26
JPEG

• 1987 ITU ISO gt international standard for
  still image compression, due to grows in the PC
  market JPEG Joint Photographic Expert Group
• Goal non-binary images keeping a good to
  excellent image quality
• First standard in 1992
• JPEG is NOT an algorithm but rather a framework
• with severel algorithms and user-settings

27
JPEGs 4 modes

• Sequential 10-20 (with good quality)
• Lossless 2 (No quantization, differential)
• Progressive
• Hierarchical
• Multi resolution

28
JPEG Sequential
29
Quantization examples
30
Huffman tables
EOB 1010 ZRL 11111111001
31
What to remember (1)

• Lossless compression vs. Lossy compression
• Data ! info. Data gtgt info. Redundancy
• Statistical redundancy spatial, coding
• Phychovisual redundancy
• Human visual system (HVS)
• Differential sensitivity
• Low-pass filter no high frequencies

32
What to remember (2)

• Source coder block diagram
• (encoding decoding codec)

Coded bit-string

• Transformation a new representation of data
• Differential coding, transform coding, e.g., DCT
  (MM2)
• Quantization In-reversible process gt lossy
  coding
• Codeword assign. (entropy coding) Huffman, RLE

33
What to remember (3)

• JPEG Joint Photographic Expert Group
• NOT an algorithm, a framework with severel
  algorithms and user-settings
• Standard JPEG
• 8x8 blocks, DCT, zig-zag scan
• Quantization. Data is lost in this process!
• Entropy coding (Huffman, RLE)

34
Exercise

• One mini (micro) project
• To be presented by each group at the last mini
  module
• Task design and implement an algorithm
• that can compress images with at least a factor
  10 without losing significant image quality
• Follow the JPEG framework and/or be creative e.g.
  the optimal KLT instead of the DCT
• Mail me (tbm_at_cvmt.dk) your location and group
  email
• Questions?
• The end of the lecture

35
Implementation

• No quantization gt lossless coding gt z(i)
  zest(i)
• Problem Accumulation of quantization errors

36
Solution

• Include the decoder loop in the encoder gt
• Predicted value delayed estimated value gt
• pz(i) zest(i-1)

37
General differential coder

• More advanced prediction based on a number of
  previous estimated values
• 1D example
• pz(i) Sj1 aj zest(i-j)
• aj is the jth weight factor

z(i)
n
.
i