Adaptive%20Image%20Filtering%20Using%20Run-Time%20Reconfiguration

1
Adaptive Image Filtering Using Run-Time
Reconfiguration

• Nitin Srivastava
• Jerry L. Trahan
• Ramachandran Vaidyanathan
• Suresh Rai
• Department of Electrical and Computer Engineering
• Louisiana State University

2
The Problem Adaptive Image Filtering

• A filtering window moves over the image pixel by
  pixel. Window size is usually 3?3, 5?5, or 7?7.
• The filter multiplies the intensity values of
  pixels that the window overlaps with its
  coefficients and sums the products to produce the
  new value of the pixel at the center of the
  window.

3
Working of a 3?3 size filter
4

• Spatially invariant filter does not change
  values of its coefficients with the position of
  the filtering window over the image.
• Adaptive filter adjusts values of its
  coefficients according to the nature of the
  image. For instance, handles uniform regions
  differently than edges.

5
Use of Run-Time Reconfiguration

• For fixed filter coefficients, could use constant
  coefficient multipliers (KCMs) configured for
  coefficients.
• For adaptive coefficients, we use KCMs configured
  for pixel values. Flow of data regular, but more
  involved.
• Inspired by 1D adaptive filtering technique of
  Wojko and ElGindy (RAW99).

6
Use of FPGAs

• Circuit design tailored to the problem
  Filtering exhibits regular, repeated operations,
  taking an inner product among the same number of
  elements at each pixel position.
• Problem size-specific components and datapaths.
• Advantages for this problem even without
  reconfiguration.

7
Image Filtering Details
8
Solution Approach

• Gray scale image of size 256?256, using a
  filtering window of size 3?3.
• Can tailor to different image size changes some
  register sizes and memory requirements.
• Can tailor to different window size changes
  memory requirements.
• Can extend to video window is 3D across frames.

9
Solution Approach, cont.

• Basic component is a module.
• Sixteen pipelined modules act on 16 contiguous
  pixels at a time from the same row.
• Three sets of three steps each, corresponding to
  the three rows in a 3?3 window and the three
  positions in each row.
• For each of these nine steps, a module
  contributes to one of the nine window
  computations in which its pixel participates.

10
Module Algorithm

• Procedure THREEPIX(r, in, out)
• Step 0 Adder(r) ? KCM(r) in
• Step 1 Adder(r) ? KCM(r) Adder(r?1)
• Step 2 Adder(r) ? KCM(r) Adder(r?1)
• out ? Adder(r)
• Contributes to three pixel values on the same row.

11
Overall Algorithm

• for i ? 0 to 255
• for k ? 0 to 255 in steps of 16
• for all j, where k ? j ? k15
• r j mod 16
• / module r has v(i, j) /
• THREEPIX(r, 0, memory)
• THREEPIX(r, memory, memory)
• THREEPIX(r, memory, I/O pins)

12
Module
13
First vantage point one module
14
5,22
5,26
5,27
5,28
5,29
5,23
5,24
5,25
6,22
6,23
6,24
6,25
6,26
6,27
6,28
6,29
7,22
7,28
7,29
8,22
8,23
8,24
8,25
8,26
8,27
8,28
8,29
9,22
9,23
9,24
9,25
9,26
9,27
9,28
9,29
15
(No Transcript)
16
(No Transcript)
17
Nitin Slide added
18
v7,23
v7,24
v7,25
v7,26
v7,27
0
0
0
0
0
pd8,27,-1,-1
pd8,25,-1,-1
pd8,26,-1,-1
pd8,28,-1,-1
pd8,24,-1,-1
pd8,25,-1,0


pd8,24,-1,0
pd8,26,-1,0
pd8,27,-1,0
pd8,23,-1,0

pd8,25,-1,1

19
block memories
block memory 9
block memory
block memory
5,22
5,26
5,27
5,28
5,29
5,23
5,24
5,25
6,23
6,22
6,24
6,25
6,26
6,27
6,28
6,29
7,22
7,26
7,23
7,24
7,25
7,27
7,28
7,29
8,24
8,22
8,23
8,25
8,26
8,27
8,28
8,29
9,22
9,23
9,24
9,25
9,26
9,27
9,28
9,29
20
v7,23
v7,24
v7,25
v7,26
v7,27
0
0
0
0
0
pd8,27,-1,-1
pd8,25,-1,-1
pd8,26,-1,-1
pd8,28,-1,-1
pd8,24,-1,-1
pd8,25,-1,0


pd8,24,-1,0
pd8,26,-1,0
pd8,27,-1,0
pd8,23,-1,0

pd8,22,-1,1
pd8,24,-1,1
pd8,25,-1,1
pd8,26,-1,1
pd8,23,-1,1
rs(-1)8,24

21
block memories
block memory 9
block memory
block memory 11
rs(-1)8,24
rs(-1)7,26
rs(-1)7.24 rs(0)7,24
rs(-1)6,26 rs(0)6.26
5,22
5,26
5,27
5,28
5,29
5,23
5,24
5,25
6,27
6,23
6,22
6,24
6,25
6,26
6,28
6,29
7,22
7,26
7,23
7,24
7,25
7,27
7,28
7,29
8,24
8,22
8,23
8,25
8,26
8,27
8,28
8,29
9,22
9,23
9,24
9,25
9,26
9,27
9,28
9,29
22
v7,23
v7,24
v7,25
v7,26
v7,27
0
0
0
0
0
pd8,27,-1,-1
pd8,25,-1,-1
pd8,26,-1,-1
pd8,28,-1,-1
pd8,24,-1,-1
pd8,25,-1,0


pd8,24,-1,0
pd8,26,-1,0
pd8,27,-1,0
pd8,23,-1,0
pd8,22,-1,1 rs(-1)7,26

pd8,24,-1,1
pd8,25,-1,1
pd8,26,-1,1
pd8,23,-1,1
rs(-1)8,24
pd7,26,0,-1
pd7,28,0,-1
pd7,25,0,-1
pd7,27,0,-1
pd7,24,0,-1
pd7,24,0,0
pd7,25,0,0
pd7,26,0,0
pd7,27,0,0
pd7,23,0,0
pd7,23,0,1
pd7,25,0,1
pd7,26,0,1
pd7,24,0,1
rs(-1)6,26 rs(0)6,26
pd7,22,0,1
rs(-1)7,24rs(0)7,24
pd6,26,1,-1
pd6,27,1,-1
pd6,28,1,-1
pd6,25,1,-1
pd6,24,1,-1

pd6,25,1,0
pd6,24,1,0
pd6,26,1,0
pd6,27,1,0
pd6,23,1,0
pd6,24,1,1
pd6,25,1,1
pd6,26,1,1
pd6,23,1,1
pd6,22,1,1
newv6,24
23
Second vantage point one pixel
24
5,22
5,26
5,27
5,28
5,29
5,23
5,24
5,25
6,23
6,24
6,25
6,26
6,27
6,22
6,28
6,29
7,22
7,23
7,24
7,25
7,26
7,27
7,28
7,29
8,22
8,23
8,24
8,25
8,26
8,27
8,28
8,29
9,22
9,23
9,24
9,25
9,26
9,27
9,28
9,29
25
v6,23
v6,24
v6,25
v6,26
v6.27
0
0
0
0
0
pd7,25,-1,-1
pd7,26,-1,-1
pd7,27,-1,-1
pd7,24,-1,-1
pd7,28 1,-1



26
(No Transcript)
27
v6,23
v6,24
v6,25
v6,26
v6.27
0
0
0
0
0
pd7,27,-1,-1
pd7,25,-1,-1
pd7,26,-1,-1
pd7,24,-1,-1
pd7,28,-1,-1


pd7,24,-1,0
pd7,25,-1,0
pd7,26,-1,0
pd7,27,-1,0
pd7,23,-1,0

pd7,24,-1,1
pd7,23,-1,1
pd7,25,-1,1
pd7,26,-1,1
pd7,22,-1,1
rs(-1)7,25
28
(No Transcript)
29
Evaluation

• Xilinx Virtex-E FPGA XCV200E
• Number of CLB slices required 492
• Clock frequency 101.9 MHz
• Time spent in filtering a 256?256 image 642 ?s

30
Comparison