Semi-supervised Affinity Propagation

1
Semi-supervised Affinity Propagation

• Inmar Givoni, Brendan Frey, Delbert Dueck
• PSI group
• University of Toronto

2
Affinity Propagation

• Clustering algorithm that works by finding a set
  of exemplars (prototypes) in the data and
  assigning other data points to the exemplars
  Frey07

• Input pair-wise similarities (negative squared
  error), data point preferences (larger more
  likely to be an exemplar)
• Approximate maximization of the sum of
  similarities to exemplars
• Mechanism message passing in a factor graph

3
Semi-supervised Learning

• Large amounts of unlabeled training data
• Some limited amounts of side information

Partial labels
Equivalence constraints
4
Some Motivating examples
5
AP with partial labels

• All points sharing the same label should be in
  the same cluster.
• Points with different labels should not be in the
  same cluster.
• Imposing constraints
• Via the similarity matrix
• Explicit function nodes

6
Same label constraints

• Set similarity among all similarly labeled data
  to be maximal.
• Propagate to other points (teleportation)
• Without teleportation, local neighborhoods do not
  move closer.
• e.g. Klein02

S(x1,x2)0
x1
x2
y2
y1
7
Different labels

• Can still do a similar trick and set similarity
  among all pair-wise differently labeled data to
  be minimal.
• But no equivalent notion of anti-teleportation.

8
Adding explicit constraints to account for
side-information
9
Adding explicit constraints to account for
side-information
10
Problems

• Lets call all the labeled points portals
• They induce the ability to teleport
• At test time, if we want to determine a label for
  some new point we need to evaluate its closest
  exemplar, possibly via all pairs of portals -
  expensive.
• Pair-wise not-in-class nodes for each pair of
  differently labeled points is expensive.
• Introducing

11
Meta-Portals

• An alternative way of propagating neighborhood
  information.
• Meta-portals are dummy points, constructed
  using the similarities of all portals of a
  certain label.
• We add N new entries to the similarity matrix,
  where N is the number of unique labels.

12
Meta-portals

• mtps can be exemplars.
• Unlike regular exemplars, mtps can be exemplars
  for other points but choose a different exemplars
  themselves

13
These function nodes force the MTPs to choose
other data points as their exemplars.
Similarities alone are not enough, since both MTP
can choose same exemplars and still have inf
similarities.
14
Some toy data results
15
Future work

• Investigate interplay between modifying
  similarities and incorporating explicit
  constraints.
• Possible tool for user-guided labeling