Realtime Recognition of Orchestral Instruments - PowerPoint PPT Presentation

About This Presentation
Title:

Realtime Recognition of Orchestral Instruments

Description:

Realtime Recognition of Orchestral Instruments Ichiro Fujinaga McGill University – PowerPoint PPT presentation

Number of Views:54
Avg rating:3.0/5.0
Slides: 22
Provided by: Peabo3
Category:

less

Transcript and Presenter's Notes

Title: Realtime Recognition of Orchestral Instruments


1
Realtime Recognition of Orchestral Instruments
  • Ichiro Fujinaga
  • McGill University

2
Overview
  • Introduction
  • Lazy learning (exemplar-based learning)
  • k-NN classifier
  • Genetic algorithm
  • Features
  • Results
  • Conclusions

3
Introduction
  • Realtime recognition of isolated monophonic
    orchestral instruments
  • Spectrum analysis by Miller Puckettes fiddle
  • Adaptive system based on a exemplar-based
    classifier and a genetic algorithm

4
Overall Architecture
Live mic Input
Sound file Input
Data Acquisition Data Analysis (fiddle)
Recognition K-NN Classifier
Output Instrument Name
Knowledge Base Feature Vectors
Genetic Algorithm K-NN Classifier
Best Weight Vector
Off-line
5
Exemplar-based learning
  • The exemplar-based learning model is based on the
    idea that objects are categorized by their
    similarity to one or more stored examples
  • There is much evidence from psychological studies
    to support exemplar-based categorization by
    humans
  • This model differs both from rule-based or
    prototype-based (neural nets) models of concept
    formation in that it assumes no abstraction or
    generalizations of concepts
  • This model can be implemented using k-nearest
    neighbor classifier and is further enhanced by
    application of a genetic algorithm

6
Exemplar-based categorization
  • Objects are categorized by their similarity to
    one or more stored examples
  • No abstraction or generalizations, unlike
    rule-based or prototype-based models of concept
    formation
  • Can be implemented using k-nearest neighbor
    classifier
  • Slow and large storage requirements?

7
Exemplar-based learning
  • The exemplar-based learning model is based on the
    idea that objects are categorized by their
    similarity to one or more stored examples
  • There is much evidence from psychological studies
    to support exemplar-based categorization by
    humans
  • This model differs both from rule-based or
    prototype-based (neural nets) models of concept
    formation in that it assumes no abstraction or
    generalizations of concepts
  • This model can be implemented using k-nearest
    neighbor classifier and is further enhanced by
    application of a genetic algorithm

8
K-nearest-neighbor classifier
  • Determine the class of a given sample by its
    feature vector
  • Distances between feature vectors of an
    unclassified sample and previously classified
    samples are calculated
  • The class represented by the majority of
    k-nearest neighbors is then assigned to the
    unclassified sample

9
Example of k-NN classifier
10
Example of k-NN classifier
11
Example of k-NN classifier
12
Example of k-NN classifier
13
Distance measures
  • The distance in a N-dimensional feature space
    between two vectors X and Y can be defined as
  • A weighted distance can be defined as

14
Genetic algorithms
  • Optimization based on biological evolution
  • Maintenance of population using selection,
    crossover, and mutation
  • Chromosomes weight vectors
  • Fitness function recognition rate
  • Leave-one-out cross validation

15
Features
  • Static features (per window)
  • pitch
  • mass or the integral of the curve (zeroth-order
    moment)
  • centroid (first-order moment)
  • variance (second-order central moment)
  • skewness (third-order central moment)
  • amplitudes of the harmonic partials
  • number of strong harmonic partials
  • spectral irregularity
  • tristimulus
  • Dynamic features
  • means and velocities of static features over time

16
Data
  • Original source McGill Master Samples
  • Over 1300 notes from 39 different timbres (23
    orchestral instruments)
  • Spectrum analysis by fiddle (2048 points)
  • First 46232ms of attack (19 windows)
  • Each analysis window (46 ms) consists of a list
    of amplitudes and frequencies of the peaks in the
    spectra

17
Results
  • Experiment I
  • SHARC data
  • static features
  • Experiment II
  • fiddle
  • dynamic features
  • Experiment III
  • more features
  • redefinition of attack point

18
Conclusions
  • Realtime timbre recognition system
  • Analysis by Puckettes fiddle
  • Recognition using dynamic features
  • Adaptive recognizer by k-NN classifier enhanced
    with genetic algorithm
  • A successful implementation of exemplar-based
    classifier in a time-critical environment

19
Future research
  • Performer identification
  • Speaker identification
  • Tone-quality analysis
  • Multi-instrument recognition
  • Expert recognition of timbre

20
Recognition rate for different lengths of
analysis window
21
Comparison with Human Performance
Write a Comment
User Comments (0)
About PowerShow.com