Polyphonic Transcription

1
Polyphonic Transcription

• Bruno Angeles
• McGill University - Schulich School of Music
• MUMT-621
• Fall 2009

2
Outline

• Polyphonic vs Monophonic
• The Human Method
• Issues
• Methods
• Evaluation of Transcription Methods

3
Polyphonic vs Monotonic Transcription

• Audio signal with musical content ?
• symbolic format
• Monophonic a single instrument playing
  individual notes
• Polyphonic transcription
• A single instrument playing several concurrent
  notes
• An instrument playing individual notes with other
  instruments
• An instrument playing several concurrent notes
  with other instruments
• Several instruments playing concurrent notes

4
The Human Method

• Initial sketch? Sections and key phrases
• Chord scheme or bass line
• Melody and countermelodies
• Instrument playback
• Musical knowledge
• Beat tracking
• Style detection
• Instrument identification
• Hainsworth and Macleod (2004)

5
Issues with Polyphonic Transcription

• Restrictionsinstrument, genre, maximum
  polyphony
• Transcriptionpitch timing attack release
• Overlapping harmonics ? difficult frequencybased
  analysis
• Solution better understanding and reproduction
  of the human method?

6
A Few Definitions

• Musical note a discrete note pitch with a
  specific onset and an offset time
• Melodies consecutive note sequences with
  organized and recognizable shape
• Chords combinations of simultaneously sounding
  notes
• Ryynänen and Klapuri (2005)

7
Methods

• Bottomup methodsno highlevel analysis
• Blackboard systemsdevelop hypotheses at various
  levels
• Modelbased algorithmshigh and lowlevel
  analysis parameter extraction
• Common method
• Preprocessing (e.g., lowpass filtering)
• Event extraction
• Classification Support Vector Machines, Hidden
  Markov Models, Neural Networks, Gaussian Mixture
  Models, etc.
• Postprocessing (e.g., remove outliers)
• Other methods probabilistic note-based, matrix
  factorization

8
Evaluation of Transcription Methods

• NC number of correct events detected
• ND total number of events detected
• N actual number of events
• Precision P NC / ND
• Recall R NC / N
• FMeasure F 2RP/(RP) 2 NC / (N ND)

9
Evaluation of Transcription Methods

• Gillet and Richard (2005)
• 2 drummers
• Different musicalsequences
• Celtic Groove 5/4Many ghost notes

Table 4 of Gillet and Richard (2005)
10
Evaluation of Transcription Methods

• Ryynänen and Klapuri (2005)R P 40Multiple
  instrumentsMultiple concurrent notes
• Hainsworth and Macleod (2005)P 78.7Bass
  guitarPolyphonic context
• Gillet and Richard (2005)R P 84Drums

Ryynänen and Klapuri (2005)
11
Conclusion

• Analogy with speech recognition
• The problem has not yet been solved
• Still easier for humans to do although complex
• Combine with source separation?

12
Thank you!
13
References

• Bruno, I., S. L. Monni, and P. Nesi. 2003.
  Automatic music transcription supporting
  different instruments. In Proceedings of the
  Third International Conference on Web Delivering
  of Music. Leeds, UK. 3744.
• Cemgil, A. T., B. Kappen, and D. Barber. 2003.
  Generative model based polyphonic music
  transcription. In Proceedings of the IEEE
  Workshop on Applications of Signal Processing to
  Audio and Acoustics. New Paltz, NY, USA. 1814.
• Gillet, O. K., and G. Richard. 2005. Drum Track
  Transcription of Polyphonic Music Using Noise
  Subspace Projection. In Proceedings of the
  International Conference on Music Information
  Retrieval. London, UK. 9299.
• Hainsworth, S. W. 2001. Analysis of musical audio
  for polyphonic transcription. In First Year Ph.D.
  Report.
• Hainsworth, S. W., and M. D. Macleod. 2001.
  Automatic bass line transcription from polyphonic
  music. In Proceedings of the International
  Computer Music Conference. Havana, Cuba.
• Hainsworth, S. W., and M. D. Macleod. 2004. The
  automated music transcription problem. In
  Cambridge University Engineering Department.
• Klapuri, A. 2004. Signal processing methods for
  the automatic transcription of music. In Ph.D.
  Dissertation.

14
References

• Lidy, T., A. Rauber, A. Pertusa, and J. M.
  Iñesta. 2007. Improving genre classification by
  combination of audio and symbolic descriptors
  using a transcription system. In Proceedings of
  the International Conference on Music Information
  Retrieval. Vienna, Austria. 616.
• Marolt, M. 2001. SONIC Transcription of
  polyphonic piano music with neural networks. In
  Proceedings of the Workshop on Current Research
  Directions in Computer Music. Barcelona, Spain.
  21724.
• Nichols, E., and C. Raphael. 2007. Automatic
  transcription of music audio through continuous
  parameter tracking. In Proceedings of the
  International Conference on Music Information
  Retrieval. Vienna, Austria. 38792.
• Niedermayer, B. 2008. Non-negative matrix
  division for the automatic transcription of
  polyphonic music. In Proceedings of the
  International Conference on Music Information
  Retrieval. Philadelphia, PA, USA. 5449.
• Paulus, J. 2006. Acoustic modelling of drum
  sounds with Hidden Markov Models for music
  transcription. In Proceedings of the IEEE
  International Conference on Acoustics, Speech and
  Signal Processing. Toulouse, France.
• Pertusa, A., and J. M. Iñesta. 2003. Polyphonic
  music transcription through dynamic networks and
  spectral pattern identification. In IAPR
  International Workshop on Artificial Neural
  Networks in Pattern Recognition. 1925.
• Ryynänen, M., and A. Klapuri. 2006. Transcription
  of the singing melody in polyphonic music. In
  Proceedings of the International Conference on
  Music Information Retrieval. Victoria, BC,
  Canada.