Image Processing in the block DCT Space

1
Image Processing in the block DCT Space

• Jayanta Mukhopadhyay
• Department of Computer Science Engineering
• Indian Institute of Technology, Kharagpur,
  721302, India
• jay_at_cse.iitkgp.ernet.in

2
Processing in the spatial domain
3
Processing in the compressed domain
4
Motivations

• Computation with reduced storage.
• Avoid overhead of inverse and forward transform.
• Exploit spectral factorization for improving the
  quality of result and speed of computation.

5
Typical Applications

• Resizing.
• Transcoding.
• Enhancement.
• Retrieval.

6
Typical Applications

• Compressed image streams with varying resolutions
  may be transmitted over channels of varying
  bandwidth
• Browsing of images in different resolutions over
  internet, eg. Initially at low resolution and
  later, if interested, browsed with higher
  resolution.

7
DCT Definitions
8
Different types of DCTs

• Let x (n), n0,1,2,...N1 be a sequence of
  length N1. Then N-point type-I DCT is defined
  as

Note- Type-I N-point DCT is defined over N1
data points.
9
Different types of DCTs

• Similarly, x (n), n0,1,2,...N, be a sequence
  of length N. Its N-point type-II DCT is defined
  as

Note- Type-II N-point DCT is defined over N
data points.
10
where is given by the
following equation


11
Sub-band Relationship
12
Subband DCT

• DCT of a 2D images x(m,n) of size N x N

Jung, Mitra and Mukherjee , IEEE CSVT (1996)?
13
Low-low subband
Sub-band approximation
14
Truncated approximation
Generalized Truncated approximation
C' (k,l) v(L.M).C(k,l), k,l0,1,....,
(N-1) 0
otherwise
Mukherjee and Mitra (2005), IEE VISP
15
Spatial Relationship
16
Need for Analysis of the Relation between DCT
blocks and sub-blocks

• Direct conversion of DCT blocks of one size into
  another possible in the compressed domain itself
• Saving of computation complexity and memory
  required
• Decompression and re-compression steps reduced
• DCT matrices being sparse number of
  multiplications and additions reduced
• DCT blocks of different sizes required for
  different standards

Jiang and Feng (2002), IEEE SP
17
Spatial Relationship of DCT coefficients
18
Standard N-point DCT of Bx(i) and of its pth
section Tp
19
Basis function b(k,t) constructed from b1(k,t)?
20
Equations to find the transform matrices for
composition and decomposition
21
EXAMPLES OF TRANSFORM MATRICES

• A(2,4)
• 0.7071 0 0 0
  0.7071 0 0 0
• 0.6407 0.2040 0.0528 0.0182 -0.6407
  0.2040 0.0528 0.0182
• 0 0.7071 0 0
  0 0.7071 0 0
• -0.2250 0.5594 0.3629 0.0690 0.225
  0.5594 -0.3629 0.0690
• 0 0 0.7071 0
  0 0 0.7071 0
• 0.1503 0.2492 0.5432 0.3468 0.1503
  -0.2492 -0.5432 0.3468
• 0 0 0 0.7071
  0 0 0 -0.7071
• -0.1274 0.1964 0.2654 0.6122 0.1274
  0.1964 0.2654 0.6122

A(2,2) 0.7071 0 0.7071
0 0.6533 0.2706 0.6533 0.2706
0 0.7071 0 -0.7071
-0.2706 0.6533 0.2706 0.6533
22
Block Composition and Decomposition in 2D
Composition of L x M adjacent N x N DCT blocks to
a single LN x MN DCT block
Decomposition of LN x MN size DCT block to L x M
adjacent N x N DCT blocks
23
Example of the conversion matrix

• Note-
• The conversion matrices and their inverses are
  sparse.
• Requires less number of multiplications and
  additions than those of usual matrix
  multiplications.

24
DCT domain Filtering
25

Type-I Symmetric Extension

• For type-I DCT symmetric extension of N1 sample
  take place in the following

26

Type-II Symmetric Extension

• For type-II DCT the symmetric extension of input
  sequence is done in the following manner

27
Symmetric Convolution
Let type II symmetric extension of x(n) be
denoted by and type I symmetric
extension of h(n) be denoted as
. Symmetric convolution of x(n) and h(n) defined
as follows
28
CONVOLUTION-MULTIPLICATION PROPERTY
Martucci (1994), IEEE SP
29
Filtering Approaches

1. Based on Symmetrical Convolution
2. Linear filtering with DCT of filter matrices

30

1. Filtering with Symmetric Convolution
Note- JPEG compression scheme uses type-II DCT,
transform domain filtering directly gives the
type-II DCT.
Martucci and Mersereau, IEEE ASSSP
1993, Martucci, IEEE SP 1994, Mukherjee and
Mitra LNCS 2006
31
(No Transcript)
32
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
33
( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
2. Linear Filtering in the block DCT domain
Here, both filter matrices and input both in
Type-II DCT space.
Spatial Domain
DCT Domain
Merhav and Bhaskaran, HPL-95-56, 1996, Kresch
and Merhav, IEEE IP 1999, Yim, IEEE CSVT 20004
40
Filtering in the block DCT domain

• DCT convolution-multiplication property reduces
  the cost of filtering.
• Type-I DCT of the filter and the Type-II DCT data
  are used.
• To avoid boundary effect, Filtering is performed
  on the larger DCT block.
• The filtering technique by Mukherjee and Mitra
• Three separate steps composition, filtering and
  decomposition of the DCT blocks.
• Cost of filtering is high.
• In this work, three operations are computed in a
  single step.

( Mukherjee and Mitra, LNCS-4338, ICVGIP 2006)
41
CMP Filtering an improvement
B_1
B0
B1
Input 1. Compose 2. Filter 3.
Decompose
Filter response
Point wise Multiply
Type-II DCT, B
Type-I DCT of Filter
Type-II DCT, Bf
Bf-1
Bf 0
Bf 1
Steps 1-3 combined step
42
Filtering in the block DCT space
1. 2. 3. 4.
5.
43
Filtering in the block DCT space
In 2-D
44
Gaussian Filtering
45
Results Gaussian filter
Close to 300 dB
4x4
Note- Sparse DCT block
46
Performance Gaussian filtering
Variation in Number of blocks
No variation in PSNR and cost
Cost varies with L and M
47
Threshold variation
48
Performance of proposed filtering for noisy images
49
ResultsThreshold variation
Original Images
Threshold 10-3
Threshold 10-4
50
Blocking artifacts removal
51
Image Sharpening
Sharpened Images
52
Observations

• The proposed technique provides equivalent
  results as obtained by Linear convolution.
• Complexity can be reduced by varying the
  threshold value.
• The technique is comparable with existing
  techniques.
• The technique is based on simple linear
  operations.

53
Thanks
54
(No Transcript)
55

Type II 2D-DCT
Type-II 2-D DCT of a block
?
?
?
?
,
k
X
II
?
?
?
?
?
?
1
1
N
N
2
?
?
?
?
?
?
?
?
?
?
?
?
?
1
2
1
2
n
k
m
?
?
?

,
cos
cos
,
)?
(
)?
(

n
m
x
k
2
2
N
N
N
where,
?
?
0
0
m
n
?
?
?
?
1
,
0





N
k