Support Vector Regression

1
Support Vector Regression
(Linear Case
)

• Motivated by SVM

• Some tiny error should be discard

2
-Insensitive Loss Function
3
(No Transcript)
4
(No Transcript)
5
SV Regression by Minimization Quadratic
-Insensitive Loss

• We minimize

at the same time

• Occams razor the simplest is the best

• We have the following problem

where
6
Primal Formulation of SVR for Quadratic
-Insensitive Loss
subject to

• Extremely important At the solution

7
Dual Formulation of SVR for Quadratic
-Insensitive Loss
subject to
8
KKT Complementarity Conditions

• Dont forget we have

• KKT conditions are

9
Simplify Dual Formulation of SVR
subject to

• The case , problem becomes to the
• least squares linear regression with a
  weight
• decay factor

10
Kernel in Dual Formulation for SVR
subject to

• Then the regression function is defined by

where
is chosen such that
with