Support Vector Machines Exercise solutions

1
Support Vector MachinesExercise solutions

• Ata Kaban
• The University of Birmingham

2
Exercise 1.

• What is the main idea behind linear Support
  Vector Machines (SVM)? Illustrate your
  explanation by drawing a figure.
• ANSWER
• The figure should show e.g. two linearly
  separable clusters of points, each cluster
  corresponding to a different class. Even though
  there are many possible separating lines, we pick
  the one that has maximal minimum distance from
  the closest points of each class.
• This choice is supported by a theorem in learning
  theory that bounds generalization error in terms
  of separation margins.

3

• b) Given K1 and K2 two proper kernels. Determine
  which of the following formulae define proper
  kernels and explain why.
•  

ANSWER K4 and K5 are proper kernels, since given
a proper kernel K, then aK, a gt 0, is a proper
kernel. Also given proper kernels K' and K'',
then K' K'' is a proper kernel. K3 is not a
proper kernel because it is negative definite for
any data sets.
4

• c) Consider the 2-dimensional inputs .
• Is the following a proper kernel? Explain why.
• ANSWER
• It is a proper kernel, since for any real valued
  function over the input space,
  is a proper kernel.

5
Remember to master the worked questions /
exercises

• How do we know if a kernel is proper?
• - - given a proper kernel K, then aK, a gt 0, is a
  proper kernel
• - - given proper kernels K' and K'', then K' K''
  is a proper kernel
• - - if K is a proper kernel, for any real valued
  function over the input space,
  is a proper kernel.