Walsh Transform

1
Walsh Transform

• ELE5430 Pattern Recognition
• W.K. Cham
• Professor
• Department of Electronic Engineering
• The Chinese University of Hong Kong

2
Walsh Transform (2)

• Historical Note
• Dyadic Symmetry
• Some properties of Dyadic Symmetry
• Generation of Walsh matricws
• Conversion between matrices of different ordering
• Dyadic Decomposition
• Fast Walsh Transform
• Ongoing Works

3
Historical Note

• We have explained how a vector can be represented
  as a weighted sum of orthogonal basis vectors.
• The representation of a signal as a weighted sum
  of a set of functions has many applications and a
  long history. Many people have contributed in
  this work and one of the most important
  contribution is by this man in 1807.

in 1922, Rademacher devised an incomplete set of
orthogonal functions. The Rademacher functions
are defined within the 0,1) and take the values
1 and -1. Reademacher, H., 'Einige Satze von
allgemeinen Orthogonal-funktionen," Math. Annalen
87, pp.122-138, 1922.
(3)
4
Rademacher functions
R0(t) 1, t?0,1) For i ? 1, Ri(½ t)
Ri-1(t) Ri(½ t ½) - Ri-1(t) , t?0,1)
Walsh functions In 1923, J.L. Walsh add more new
functions and formed a complete orthonormal set
of functions, now known as Walsh
functions. Walsh, J.L., 'A closed set of
orthogonal functions," American J. Of the
Mathematics, Vol.45, pp.5-24, 1923.
(4)
5
Walsh Transform
However, Walsh function did not receive much
attention. Fourier function which is the
eigenfunction of convolution attracted all
attention. It also lead to the development of FFT
and a complete theory for discrete system. In
1969, Pratt and others used the Walsh transform
(WT), developed from the Walsh functions, in
place of the FFT for image coding. Pratt, W.K.,
Kane, J., Andrews, H.C.,'Hadamard Transform Image
Coding,' Proc. IEEE, Vol.57, No.1, pp.58-68, Jan.
1969.
In the early 1970's, the simplicity of WT
resulted in a wide range of applications and
interest. Symposium on Applications of Walsh
Functions, Washington, D.C., 1970. Symposium on
Applications of Walsh Functions, Washington,
D.C., 1971. Symposium on Applications of Walsh
Functions, Washington, D.C., 1972. Symposium on
Applications of Walsh Functions, Washington,
D.C., 1973.
(5)
6
Walsh Transform
The effectiveness of most of these applications,
especially filtering and coding, depends on the
ability of the transform to pack signal energy
into a few transform coefficients. The WT is
inferior to DFT in that aspect. Later, people
discovered the Discrete Cosine Transform (DCT)
which has even better energy packing ability. The
interest in WT diminished quickly. Today Walsh
transform is mainly used in multiplexing which is
to send several data simultaneously. It does not
require high energy packing ability.
Multiplexing
To transmit data a, b, c, d, e, f, g, h at the
same time over a period, the following signal
will be sent.
To decode a, b, c, d, e, f, g and h , perform the
dot product between y(t) and the corresponding
Walsh function.
(6)
7
Nomenclature
During the development, different researchers
adopted different nomenclatures. In fact, it is
not unified even today.
Def Hadamard matrix is a square matrix of only
plus and minus one whose rows (and columns) are
orthogonal to one another.
Hadamard functions, the counterpart in the
continuous case, are also called Walsh-like
functions. Walsh functions and transform are
particular cases of Hadamard functions and
transforms.
(7)
8
Many methods were developed to generate Walsh
transforms. Depending on the row orders, they can
be grouped into the following 3
Natural-ordered Walsh transform Pratt, W.K.,
Kane, J., Andrews, H.C.,'Hadamard Transform Image
Coding,' Proc. IEEE, Vol.57, No.1, pp.58-68, Jan.
1969.
Dyadic-ordered Walsh transform Shanks, J.L.,
'Computation of the fast Walsh-fourier
Transform,' IEEE Trans. Vol.18, May 1969,
pp.457-459.
Sequency-ordered Walsh transform Harmuth, H.F.,'A
Generalized Concept of Frequency and some
Applications,' IEEE Trans. on Information Theory,
Vol.14, No.3, May 1968, pp.375-382.
(8)
9
Natural-ordered Walsh transform An order 2N
transform can generated using an order N
transform as follows.
(9)
10
Sequency-ordered Walsh transform Conversion from
natural-ordered Walsh transform e.g. Let F be
the vector of natural-ordered Walsh transform
coefficients and C be the vector of
sequency-ordered Walsh transform coefficients.
000 001 010 011 100 101 110 111
? 000 ? 111 ? 011 ? 100 ? 001 ? 110 ?
010 ? 011
Compute F (with coefficients in natural order)
which is then converts into C (with coefficients
in sequency order).
(10)
11
Dyadic Symmetry (11)
Def A particular type of even symmetry is said
to exist in a vector of n elements if and only if
the n elements can be divided into n/2 paris of
elements of the same value. Def A particular
type of odd symmetry is said to exist in a vector
of n elements if and only if the n elements can
be divided into n/2 paris of elements of the same
magnitude and opposite sign.
e.g. (a, b, c, d) has 3 ways to be paired up
and so 3 types of even symmetry. (a, b,
c, d), (a, b, c, d), (a, b, c, d)
e.g. (a, b, c, d, e, f, g, h) has 105 ways to
be paired up and so 105 types of even symmetry.
(a, b, c, d , e, f, g, h), (a, b, c, d ,
e, f, g, h), (a, b, c, d , e, f, g, h), ....
(11)
12
Dyadic Symmetry
Def A vector of 2m elements ( a(0), a(1), a(2),
..., a(2m-1) ) is said to have the i th dyadic
symmetry if a(j) s ? a(j ? i) where (i) ?
is the exclusive or operator, (ii) j ? 0,
2m-1 and i ? 0, 2m, (iii) s 1 when the
symmetry is even and s -1 when the symmetry is
odd.
e.g. (a, a, b, b, c, c, d, d ) has the 1st even
dyadic symmetry. (a, b, a, b, c, d, c, d ) has
the 2nd even dyadic symmetry. (a, b, b, a, c, d,
d, c ) has the 3rd even dyadic symmetry. (a, b,
c, d, a, b, c, d ) has the 4th even dyadic
symmetry. (a, b, c, d, b, a, d, c ) has the 5th
even dyadic symmetry. (a, b, c, d, c, d, a, b )
has the 6th even dyadic symmetry. (a, b, c, d, d,
c, b, a ) has the 7th even dyadic symmetry.
(12)
13
e.g. dyadic symmetry
Def A vector of 2m elements (a(0), a(1), a(2),
..., a(2m-1)) is said to have the ith dyadic
symmetry if a(j) s ? a(j ? i) where ? is the
exclusive or operator, j?0, 2m-1 and i?0,
2m, s1 for even symmetry and s-1 for odd
symmetry.
(13)
14
Some Properties of Dyadic Symmetry (14)
Thm 1 If a 2m vector has dyadic symmetry S1, S2,
..., Sr , this vector also has dyadic symmetry
Sk where Sk S1 ? S2 ? ... ? Sr .
e.g. If an order-8 vector (a, b, c, d, e, f, g,
h) has the 1st 2nd even dyadic symmetry, then
it also has the 3rd even dyadic symmetry.
It has the 1st even dyadic symmetry so (a, a, b,
b, e, e, g, g ). It has the 2nd even dyadic
symmetry so (a, b, a, b, e, f, e, f ). Hence, a
b and e f. The vector is (a, a, a, a, e, e, e,
e ) and so it has the 3rd even dyadic symmetry.
(14)
15
Independent and Dependent Dyadic Symmetry
Def The r dyadic symmetries S1, S2, ..., Sr are
said to be dependent if there exist r elements
k1, k2, ..., kr not all zero, such that k1?S1
? k2?S2 ? ... ? kr?Sr 0. Otherwise, the r
dyadic symmetries are said to be independent.
e.g. Dyadic symmetries ( 001 ), ( 010 ), (011)
are dependent because (001) ? (010) ? (011) 0
k1?S1 ? k2?S2 ? k3?S3 0
e.g. Dyadic symmetries ( 001 ), ( 010 ), (100)
are independent.
(15)
16
Independent and Dependent Dyadic Symmetry
Thm 2 If a vector has r independent dyadic
symmetries, this vector has also 2r-1 dyadic
symmetries.
e.g. Suppose a vector has dyadic symmetries ( 001
), ( 011 ), (111). By Theorem 1, it also has
dyadic symmetries which are linear combination of
these 3 dyadic symmetries.
Hence, it has dyadic symmetries (010) (001) ?
(011) (110) (001) ? (111) (100) (011) ?
(111) (101) (001) ? (011) ? (111)
(16)
17
Generation of Walsh Matrices (17)
Thm 3 Every basis vector of an order-2m Walsh
matrix has all the 2m-1 dyadic symmetries.
The following is not a row of the Walsh matrix.
What dyadic symmetries does it have? ( 1 -1
1 -1 1 -1 -1 1 )
1st odd DS?
(17)
18
Generation of Walsh matrix using independent
Dyadic Symmetry
We use dyadic symmetries (100) (010) and (100) to
generate an order-8 natural-ordered Walsh matrix.
(100) (010) (001) 0 0 0 0
0 1 0 1 0 0
1 1 1 0 0 1 0
1 1 1 0 1 1
1
(18)
19
Binary Walsh matrix
A binary 2m x 2m Walsh matrix was defined from a
Walsh matrix as follows
w(i,j) 1 ? b(i,j) 0 w(i,j) -1 ?
b(i,j) 1
(19)
20
b(i,j) j S-1 iT where i and j are m-vectors
and S is a matrix whose rows are m independent
dyadic symmetries.
e.g.
(100) (010) (001) i ( i(1) i(2) i(3)
) 0 0 0 0
0 1 0 1 0
0 1 1 1 0 0
1 0 1 1 1
0 1 1 1
j ( j(1) j(2) j(3) ) 000 001 010 011
100 101 110 111
The elements of i (i(1) i(2) ... i(m))
determines the type (odd or even) of dyadic
symmetry of the ith row.
(20)
21
The elements of i (i(1) i(2) ... i(m))
determines the type (odd or even) of dyadic
symmetry of the ith row.
b(i,j) j S-1 iT
j ( j(1) j(2) j(3) ) 000 001 010 011
100 101 110 111
(100) (010) (001) i ( i(1) i(2) i(3)
) 0 0 0 0
0 1 0 1 0
0 1 1 1 0 0
1 0 1 1 1
0 1 1 1
The sign of the jth element in the ith row
depends on the types of the m independent dyadic
symmetries.
The sign of the jth element in the ith row
depends on the types of the m independent dyadic
symmetries. The actual relationship can be found
by expressing j (j(1) j(2) ... j(m)) w.r.t. the
a basis formed by the m independent dyadic
symmetries.
(21)
22
The elements of i (i(1) i(2) ... i(m))
determines the type (odd or even) of dyadic
symmetry of the ith row.
b(i,j) j S-1 iT
j ( j(1) j(2) j(3) ) 000 001 010 011
100 101 110 111
(100) (010) (001) i ( i(1) i(2) i(3)
) 0 0 0 0
0 1 0 1 0
0 1 1 1 0 0
1 0 1 1 1
0 1 1 1
Let r (r(1) r(2) ... r(m)) s.t. j (r(1)?S1
? r(2)?S2 ? ... ? r(m)?Sm).
If r(m)1, then b(i,j) depends on Sm. If r(m)0,
then b(i,j) is independent of Sm
or r j S-1.
(22)
23
The elements of i (i(1) i(2) ... i(m))
determines the type (odd or even) of dyadic
symmetry of the ith row.
b(i,j) j S-1 iT
j ( j(1) j(2) j(3) ) 000 001 010 011
100 101 110 111
(100) (010) (001) i ( i(1) i(2) i(3)
) 0 0 0 0
0 1 0 1 0
0 1 1 1 0 0
1 0 1 1 1
0 1 1 1
r j S-1.
If r(m)1, then b(i,j) depends on Sm Otherwise
not.
j S-1 iT .
b(i,j) r(1)i(1) ? r(2)i(2) ? ... ? r(m)i(m) r
iT
(23)
24
b(i,j) j S-1 iT
Def The matrix S in the natural-ordered binary
Walsh matrix is
Def The matrix S in the dyadic-ordered binary
Walsh matrix is
Def The matrix S in the sequency-ordered binary
Walsh matrix is
(24)
25
e.g. We use dyadic symmetries (001) (011) and
(111) to generate an order-8 sequency-ordered
Walsh matrix.
b(i,j) j S-1 iT
Hence, b(0,j)0 b(1,j)j(1)
b(2,j) j(1)?j(2)
b(3,j) j(2)
b(4,j) j(2)?j(3)
b(7,j) j(3)
b(5,j) j(1)?j(2)?j(3)
b(6,j) j(1?j(3)
(25)
26
b(i,j) j S-1 iT
e.g. We use dyadic symmetries (001) (010) and
(001) to generate an order-8 natural-ordered
Walsh matrix.
i(1)j(1) ? i(2)j(2) ? i(3)j(3)
e.g. We use dyadic symmetries (100) (010) and
(100) to generate an order-8 dyadic-ordered Walsh
matrix.
i(1)j(3) ? i(2)j(2) ? i(3)j(1)
(26)
27
Conversion between matrices of different ordering
(27)
Sequency-ordered Walsh transform coefficients
from natural-ordered Walsh transform e.g. Let F
be the vector of natural-ordered Walsh transform
coefficients and C be the vector of
sequency-ordered Walsh transform coefficients.
000 001 010 011 100 101 110 111
? 000 ? 111 ? 011 ? 100 ? 001 ? 110 ?
010 ? 011
Compute F (with coefficients in natural order)
which is then converts into C (with coefficients
in sequency order).
(27)
28
natural-ordering dyadic-ordering
sequency-ordering
in
id
iz
Conversion between natural-ordering,
dyadic-ordering and sequency-ordering can be
obtained using the following equations where in ,
id ,and iz are the corresponding indices. izT
Z D-1 idT Z N-1 inT idT D Z-1 izT
D N-1 inT inT N Z-1 izT N D-1
idT
b(i,j) j S-1 iT
Proof b(i,j) j N-1 inT j Z-1 izT
j D-1 id for all j.
(28)
29
natural-ordering dyadic-ordering
sequency-ordering
in
id
iz
e.g. Conversion from dyadic-ordering to
sequency-ordering izT Z D-1 idT
(29)
30
natural-ordering dyadic-ordering
sequency-ordering
in
id
iz
e.g. Conversion from natural-ordering to
sequency-ordering izT Z N-1 inT
(30)