Fast PNN Using Partial Distortion Search

1
Fast PNN Using Partial Distortion Search

• Olli Virmajoki1, Pasi Fränti1 and Timo
  Kaukoranta2
• 1Department of Computer Science, University of
  Joensuu
• 2Turku Centre for Computer Science (TUCS),
  Department of Computer Science, University of
  Turku.

2
1. Introduction

• The pairwise nearest neighbor method (PNN) is a
  simple and well-known method for generation of
  codebook in VQ.
• The pairwise nearest neighbor method (PNN)
  provides good quality codebooks for VQ but at the
  cost of high run time.
• Partial Distortion Search (PDS) reduces the
  workload caused by distance calculations.
• PDS reduces the run time down to 50-60 .

3
2. Pairwise Nearest Neighbor Method

• The aim is to find a codebook C of M code vectors
  (ci) by minimizing
• (1)
• Cluster is defined as the set of training vectors
  that belong to the same partition a
• (2)

4
The basic structure of the PNN

• PNN(X, M) ? C, P
• si ?xi ? i?1,N
• m ? N
• REPEAT
• (sa, sb) ? NearestClusters()
• MergeClusters(sa, sb)
• m ? m-1
• UpdateDataStructures()
• UNTIL mM

5
Merging two clusters

• The cost of merging two clusters sa and sb can be
  calculated
• (3)
• where ca and cb are the cluster centroids and na
  and nb are the cluster sizes.
• All possible cluster pairs are considered and the
  one (a,b) increasing the distortion least is
  chosen.

6
3. Partial Distortion Search (PDS)

• Let dmin be the distance of the best candidate
  found so far.
• The distance is calculated cumulatively by
  summing up the squared differences in each
  dimension.
• The cumulative summation is non-decreasing, as
  the individual terms are non-negative.
• The calculation can therefore be terminated and
  the candidate rejected if the partial distance
  value exceeds the current minimum dmin.

7
PDS / Simple Variant

• MergeCost(sa, sj, dmin) ? d
• e ? 0
• k ? 0
• w ? na ? nj / (na nj)
• REPEAT
• k ? k 1
• e ? e (cak - cjk)2
• d ? w ? e
• UNTIL (d gt dmin) OR (k K)
• RETURN d

8
PDS / Simple Variant

• (4)
• (5)
• here cak and cjk refer to the kth component of
  the corresponding vector.
• After each summation, we calculate the partial
  distortion value (waj eaj) and compare it to
  the distance of the best candidate (dmin)
• (6)
• The distance calculation is terminated if this
  condition is found to be true.
• The calculation of the partial distortion require
  an additional multiplication operation and an
  extra comparison for checking the termination
  condition.

9
PDS / Optimized Variant

• The extra multiplication in (6) can be avoided by
  formulating the termination condition as
• (7)
• The right part of the equation can now be
  calculated in the beginning of the function, and
  only the comparison remains inside the summation
  loop.
• We refer this as the optimized variant.
• As a drawback, there are one extra division due
  to (7) and extra multiplication outside the loop.

10
The computational efficiencies of the two variants

• The simple variant is faster when the dimensions
  are very small and in cases when the termination
  happens earlier.
• The equation (6) is also less vulnerable to
  rounding errors than (7).
• The optimized variant produces significant
  improvement when the dimensions are very large.

11
Training sets
12
Remaining run time relative to full search PNN
13
Run time
14
Remaining run time relative to full search
15
5. Conclusions

• 50 speed-up.
• Works only when dimensions high enough (K gt 3).
• Speed-up increases with dimensions (e.g. 10
  left when K 256)