Linear Classifiers (LC)

1
Linear Classifiers (LC)

• J.-S. Roger Jang (???)
• jang_at_mirlab.org
• http//mirlab.org/jang
• MIR Lab, CSIE Dept.
• National Taiwan University

2
Introduction to Linear Classifiers

• Linear classifiers (LC)
• The output is based on linear combination of
  features
• Types
• Linear perceptrons
• SVM (support vector machine)
• Logistic regression

3
Example Gender Classification

• Goal
• Determine a persons gender from his/her profile
  data
• Features collected
• Birthday
• Blood type
• Height and weight
• Density
• Three measures
• Hair length
• Voice pitch
• Chromosome

Training data
x2 (voice freq.)
x1 (hair length)
4
Perceptrons

• Proposed by Widrow Hoff in 1960
• AKA ADALINE (Adaptive Linear Neuron) or
  single-layer perceptron

Training data
x1
w0
w1
y
x2 (voice freq.)
w2
x2
Quiz!
x1 (hair length)
5
Characteristics of LC

• Characteristics of LC
• Guaranteed to converge to a set of weights that
  will perfectly classify all the data if such a
  solution exists
• Data rescaling is necessary to speed up
  convergence of the algorithm
• Stops whenever a solution with zero error rate is
  found
• Nonlinear decision boundaries can also be found
  by the adaptive technique
• For a k-class problem, it needs k(k-1)/2 decision
  boundaries to do complete classification

6
Demo of Perceptrons

• perceptronDemo.m

• lincTrain.m