1. Introduction to Pattern Recognition and Machine Learning.

1
1. Introduction to Pattern Recognition and
Machine Learning.

• Prof. A.L. Yuille.
• Dept. Statistics. UCLA.
• Stat 231. Fall 2004.

2
Structure.

• Examples of Patterns.
• Discriminate/Decisions about Patterns.
• Schools of Pattern Recognition.
• Learning Theory.

3
What are Patterns?

• Laws of Physics Chemistry generate patterns.

4
Patterns in Astronomy.

• Humans tend to see patterns everywhere.

5
Patterns in Biology.

• Applications Biometrics, Computational Anatomy,
  Brain Mapping.

6
Patterns of Brain Activity.

• Relations between brain activity, emotion,
  cognition, and behaviour.

7
Variations of Patterns.

• Patterns vary with expression, lighting,
  occlusions.

8
Speech Patterns.

• Acoustic signals.

9
Goal of Pattern Recognition.

• Recognize Patterns. Make decisions about
  patterns.
• Visual Example is this person happy or sad?
• Speech Example did the speaker say Yes or
  No?
• Physics Example is this an atom or a molecule?

10
Applications of Pattern Recognition.
Handwritten digit/letter recognition Biometrics
voice, iris, fingerprint, face, and gait
recognition Speech recognition Smell recognition
(e-nose, sensor networks) Defect detection in
chip manufacturing Interpreting DNA
sequences Fruit/vegetable recognition Medical
diagnosis Terrorist Detection Credit Fraud
Detection Credit Applications.
11
Two Extreme Approaches

• Generative Methods
• Determine models of how patterns are formed.
• Use these models to perform discrimination.
• Pattern Theory. Grenander.
• Discriminative Methods
• Dont model pattern formation.
• Instead extract features from patterns and
  make decision using these features.

12
Example Salmon versus Sea Bass.

• Generative methods attempt to model the full
  appearance of Salmon and Sea Bass.
• Discriminative methods extract features
  sufficient to make the decision (e.g. length and
  brightness).

13
Fish Features. Length.

• Salmon are usually shorter than Sea Bass.

14
Fish Features. Lightness.

• Sea Bass are usually brighter than Salmon.

15
Decision Boundaries.

• Classify fish as Salmon or Sea Bass based on a
  decision boundary in feature space.

16
Generative Models for Speech.

• Stochastic Grammars for Speech Natural
  Language. (Manning Schutze).

17
Bayes Decision Theory

• Bayes Decision Theory gives a framework for
  Generative and Discriminative approaches.
• Current Wisdom
• (i) Discriminative methods are simpler,
  computationally faster, and easier to apply.
• (ii) Generative methods are needed for most
  complex problems.
• Hybrid methods are increasingly popular.
• Stat 231 concentrates on Discriminative Methods
  and simple Generative Models.
• Other courses by Prof.s Zhu Yuille deal with
  complex Generative Models.

18
Learning Theory.

• Both Generative and Discriminative methods
  require training data to learn the
  models/features/decision rules.
• Machine Learning concentrates on learning
  discrimination rules.
• Key Issue do we have enough training data to
  learn?

19
Course Elements.

• Bayes Decision Theory as theoretical basis.
• Simple discriminative and generative methods.
• Machine Learning.
• Advanced Discriminative Methods.