Data Visualisation using Topographic Mappings

1
Data Visualisation using Topographic Mappings

• Colin Fyfe
• The University of Paisley,
• Scotland

2
Outline

• Topographic clustering.
• Topographic Product of Experts, ToPoE
• Simulations
• Products and mixtures of experts.
• Harmonic Topographic Mapping, HaToM
• 2 Varieties of HaToM

3
(No Transcript)
4
The somatosensory homunculus

• Larger area of cortex for more sensitive body
  parts.

5
Kohonens Self-organizing Map

• camtasia\som.html

6
Orientation selectivity
7
Topology preservation

• Data space Feature Space
• Nearby Nearby
• Distant Distant (SOM)
• Nearby Nearby (SOM)
• Distant Distant

8
The Model

• K latent points in a latent space with some
  structure.
• Each mapped through M basis functions to feature
  space.
• Then mapped to data space to K points in data
  space using W matrix (M by D)
• Aim is to fit model to data to make data as
  likely as possible by adjusting W

9
Mental Model
latent space

• 1 2 3 4 5 6

Feature space
Data space
10
Details

• t1, t2, t3, ,tK (points in latent space)
• f1(), f2(), , fM() (basis functions creating
  feature space)
• Matrix F (K by M), where fkm fm(tk),
  projections of latent points to feature space.
• Matrix W (M by D) so that FW maps latent points
  to data space. tk mk

11
Products of Gaussian Experts
12
Maximise the likelihood of the data under the
model
13
Using Responsibilities
14
(No Transcript)
15
(No Transcript)
16
Comparison with GTM
17
Growing ToPoEs
18
Demonstration
19
(No Transcript)
20
Advantages

• Growing need only change F which goes from K by
  M to (K1) by M.
• W is approximately correct and just refines its
  learning.
• Pruning uses the responsibility if a latent
  point is never the most responsible point for any
  data point, remove it.
• Keep all other points at their positions in
  latent space and keep training.

21
(No Transcript)
22
Twinned ToPoEs

• Single underlying latent space
• Single F
• Two sets of weights
• Single responsibility distance between current
  projections and both data points.

23
Twinned ToPoEs - 2
24
Using fkmtanh(mtk)
25
Products of Experts
26
(No Transcript)
27
(No Transcript)
28
Responsibilities with Tanh()
29
(No Transcript)
30

31
A close up
32
(No Transcript)
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Harmonic Averages

• Walk d km at 5 km/h, then d km at 10 km/h
• Total time d/5 d/10
• Average Speed 2d/(d/5d/10)
• Harmonic Average

38
K-Harmonic Means beats K-Means and MoG using EM

• Perf

39
Growing Harmony Topology Preservation

• Initialise K to 2. Init W randomly.
• Init K latent points and M basis functions.
• Calculate mkfkW, k1,,K.
• Calculate dik, i1,,N, k1,,K
• Re-calculate mk, k1,,K. (Harmonic alg.)
• If more, go back to 1.
• Re-calculate W(FTF?I)-1FT?
• K K 1. If more, go back to 1.

40
Disadvantages-Advantages ?

• Dont have special rules for points for which no
  latent point takes responsibility.
• But must grow otherwise twists.
• Independent of initialisation ?
• Computational cost ?

41
Generalised K-Harmonic Meansfor Automatic
Boosting

• Perf

42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
(No Transcript)
48
(No Transcript)
49
(No Transcript)
50
Two versions of HaToM

• D-HaToM (Data driven HaToM)
• W and m change only when adding a new latent
  point
• Allows the data to influence more the clustering
• M-HaToM (Model driven HaToM)
• W and m change in every iteration
• The data is continually constrained by the model

51
Simulations(1) 1D dataset
K8
K4
K2
K20
M-HaToM
D-HaToM
52
Simulations(3) algae dataset
D-HaToM
M-HaToM
Wider rij
Narrower rij
53
Conclusion

• New forms of topographic mapping.
• Based on latent space concept but
• free from probabilistic constraints.
• Product Mixture of experts.
• automatic setting of local variances.
• Two types based on K-harmonic Means
• Very sensitive to data, or not, as required