Multidimensional Scaling

1
Multidimensional Scaling

• Vuokko Vuori
• 20.10.1999
• Based on
• Data Exploration Using Self-Organizing Maps,
  Samuel Kaski, Ph.D. Thesis, 1997
• Multivariate Statistical Analysis, A Conceptual
  Introduction, Kachigan
• Pattern Recognition and Neural Networks, B. D.
  Ripley

2
Contents

• Motivations
• Dissimilarity matrix
• Multidimensional scaling (MDS)
• Sammons mapping
• Self-Organizing maps
• Comparison between MDS, Sammons mapping, and SOM

3
Motivations

• MDS attempts to
• Identify abstract variables which have generated
  the inter-object similarity measures
• Reduce the dimension of the data in a non-linear
  fashion
• Reproduce non-linear higher-dimen-sional
  structures on a lower-dimen-sional display

4
Dissimilarity Matrix

• In MDS, the dissimilarities between every
• pair of observations are given
• Genuine distances (continuos data)
• Simple matching coefficients, Jaccard
  coefficients (categorical data)
• Scaled ranks (ordinal data)
• Gowers dissimilarity for mixed data
  

5
Multidimensional Scaling

• Metric MDS
• Distances between data items are given, a
  configuration of points which gives rise to those
  distances is sought
• Can be used for non-linear projection
• Objective function which is minimized

6

• Nonmetric MDS
• Only the rank order of the distances is important
• A monotonically increasing function that acts on
  the original distances is introduced the rank
  order can be better preserved
• Normalized objective function
• For given projection, is always chosen to
  minimize

7
Sammons Mapping

• Closely related to metric MDS
• Tries to preserve pairwise distances
• Errors in distance preservation are normalized
  with the original distance
• Objective function

8
Self-Organizing Maps

• Algorithm that performs clustering and non-linear
  projection onto lower dimen-sion at the same time
• Finds and orders a set of reference vectors
  located on a discrete lattice
• Learning rule
• Objective function
• (Discrete data, fixed neighbourhood kernel)

9
Comparison Between MDS, Sammons Mapping and SOM

• MDS tries to preserve the metric (ordering
  relations) of the original space, long distances
  dominate over the shorter ones
• SOM tries to preserve the topology (local
  neighbourhood relations), items projected to
  nearby locations are similar
• Sammons lies in the middle it is like MDS but
  puts more emphasis on small distances