A Study of the Relationship between SVM and Gabriel Graph

1
A Study of the Relationship between SVM and
Gabriel Graph

• ZHANG Wan and Irwin King,
• Multimedia Information Processing
  Laboratory,
• Department of Computer Science
  Engineering ,
• The Chinese University of Hong Kong

2
Outline

• Introduction

• Related Background

• Support Vector Machine(SVM)

• Gabriel Graph

• Relative Neighborhood Graph

• Other Concepts

• Experiments

• Discussion

3
Data Classification

• Given training data in different classes(labels
  known)
• Predict test data(labels unknown)

• Examples

• Handwritten digits recognition

• Speech recognition

• Face recognition

• Methods

• Decision tree

• Neural network

• Nearest neighbor

4

• SVM(Support Vector Machine)
• ----- from Statistical learning theory
• introduced by Vapnik in 1990s
• become more and more popular

• Gabriel graph, Relative neighborhood graph
• ---- from Computational Geometry

5
Simple case of SVM
Maximize distance between two parallel separating
planes
6
SVM and Gabriel graph
SVM
Convex Hull
Gabriel Graph
Delaunay triangulation
Relative Neighborhood Graph
7
Gabriel graph

• Definition
• Decision boundary can be constructed from those
  Gabriel neighbors (p and q) such that p and q are
  of different classes.

8
Relative neighborhood graph

• Definition
• Let
• Denotes an open sphere centered at
  x with radius r, i.e.

9
Summary

• -Skeleton(Kirkpatrick,1985)
• --- a parameterized family of neighborhood
  graphs
• The neighborhood is defined,for any
  fixed ,( ) as the
  intersection of two spheres

And GG(V)G1(V), RNG(V)G2(V).
10
Summary-2

• -Skeleton of V, is neighborhood
  graph with the set of edges defined as follows

11
Gabriel editing Algorithm

• Compute the Gabriel graph for the training
  set.

• Visit each node, marking it if all its Gabriel
  neighbors are of the same class as the current
  node.

• Delete all marked nodes, exiting with the
  remaining ones as the edited training set.

12
Algorithm for SVM

• parameter C, the kernel function and any kernel
  parameters.

• Solve Dual Quadratic problem using an
  appropriate quadratic programming.

• Recover the primal threshold variable b using the
  support vectors

• Obtain the decision function

13
Comparison of time Complexity
Where n - No. of dataset, d - No. of
dimension, -- obtained through an
normalization of objective function ,which
depends on n.
SVM --- data-sensitive
Neighbor graph --- more dimension-sensitive
14
Experiments Observations

• Libsvm for SVM classification
• -skeleton algorithm implemented with C.
• Datasets include Iris dataset, Wine Cultivar
  dataset, Glass identification data set.
• The following is the parameters selected for
  SVM method to obtain an optimal solution.

Parameter Iris Data Wine Data Glass Data
Kernel Function RBF RBF RBF
Error Penalty(C) 212 27 211
Gamma for RBF 2-9 2-10 2-2
15
Experiments Observations(2)
16
Experiments Observations(3)
17
Conclusion

• According to the observations we could improve
  SVM with Gabriel graph algorithm as follows

• Map the data to some other higher, possibly
  infinite, dimension space and fit an optimal
  linear classifier in that space.

• Use the Gabriel graph algorithm to reduce the
  size of the training data.

• Using the SVM's optimization steps to obtain
  the solution to the quadratic problem and find
  the separating plane.

18
QA