Pitch Detection for Midi Conversion and Melody Transcription

1
Pitch Detection for Midi Conversion and Melody
Transcription

• For the next millennium

A Farsheed Hamidi-Toosi and Jason Laska Project
2
Overview

• Motivation
• reasons for wanting this technology
• Theory of Pitch Detection
• harmonics of a musical note
• HPS Algorithm
• other Algorithms
• Implementation/Goals
• HPS Algorithm on the DSP
• duration estimation through the serial output
• conversion to midi file format
• monophonic melody detection
• polyphonic detection of 2 notes (intervals of at
  least 5ths)
• Experiments/Simulation
• Matlab analog frequency detector script
• Simulink simulation of frequency detection
• Matlab Mex-Function for generating midi files
  based on any given input
• Problems/Caveats
• low pitch fundamental is detected less than the
  1st octave harmonic.

3
Motivation for Project

• many musicians would like to jot down their
  ideas quickly
• ideal for musicians who would like to use their
  instrument of choice to play ideas, rather than
  play on a keyboard
• helpful for musicians who dont know how to
  notate music (and for non-traditional approaches
  to music composition)
• helpful for trained musicians who do not want to
  deal with time-consuming music notation software
  such as Finale
• output to midi allows for standardization and
  exportation to a large suite of available
  applications for further processing

4
Frequencies and Harmonics of a Musical Note

• a musical note is made up of a fundamental
  frequency, plus many other harmonics
• the fundamental frequency is what we hear as a
  note or a pitch
• different instruments have different timbres, or
  harmonic components

Diagrams Professor Fred Skiff Ahmed Diallo
5
Harmonic Product Spectrum Algorithm (HPS)
This algorithm provides a simple way for
extracting the fundamental frequency of a musical
note.
The Algorithm

• First convert the sampled signal into the
  frequency domain using the FFT. Large peaks will
  appear, corresponding to the harmonic components
  of the note.
• Downsample the original DFT at multiple rates (in
  parallel) multiples of the fundamental frequency
  line up.
• Multiply each downsampled version of the DFT
  together to enhance the value of the fundamental.
• After the multiplication, the remaining peak is
  the fundamental frequency. The original analog
  frequency of the short sample can be obtained and
  the note determined through a look-up table.

• The Algorithm (Math)
• Y(w) is the multiplication of downsampled
  versions of X(w)
• The fundamental is the maximum value of Y(w)

6
de la Cuadra, P., Master A., Sapp, C. 2001.
"Efficient Pitch Detection Techniques for
Interactive Music"
Pitch Detection Theory
7
Problems and Solutions (HPS)

• Problem Significant change between notes or
  noise on signal
• can cause detection of incorrect pitch (wrong
  fundamental frequency detected)
• can be solved by looking for long patterns of the
  frequency to detect outliers
• Problem Low pitches are detected with the
  frequency at the 1st octave harmonic.
• this happens because the spacing between notes at
  low frequencies is very narrow (a high resolution
  FFT is needed to truly compute the pitch which
  would require a lot of computational power per
  sampling set)
• one solution is to choose the lower octave when
  the peak before the chosen peak has amplitude of
  about half the chosen peaks amplitude
• another solution is to detect that a pitch might
  be high or low, and perform a high resolution FFT
  on just the low samples

Pitch Detection Theory
8
Other Algorithms

• Maximum Likelihood
• compares the frequency spectrum of the input
  samples to predetermined frequency spectra
• fast for small set of pitch choices
• works well with sounds of consistent timbre
• Zero Crossing
• time domain algorithm
• count the number of times samples cross a
  zero-level reference in a given block
• very inaccurate
• detects high vs. low pitch

Pitch Detection Theory
9
Using the DSP for Pitch Detection

• take in blocklen samples of sound
• process each block to find the pitch using HPS
  algorithm
• map the calculated analog frequency of this
  pitch to a predetermined pitch number for
  serial output on the DSP (a look-up table)

10
Using the Serial Port to approximate duration of
pitch

• each processed block will output its pitch (or
  no sound) at a constant rate on the serial port
• this constant rate is known and can be used to
  calculate the duration of a note based on how
  many times in a row its value was output
• a simple Matlab script or function can compute
  this based on the output vector and generate a
  midi file
• error checking (for incorrect pitches) can be
  applied here

Implementation
11
Converting to Musical Instrument Digital
Interface (MIDI)

• What is MIDI?
• a powerful communications protocol for digital
  music first introduced in 1983 that allows for
  storage of musical data (.mid file) or
  transmission of data from a keyboard or MIDI
  triggers
• MIDI contains information about tempo, time
  signature, key, notes, even the instrument
  patch (i.e. piano or bass guitar)
• any of these parameters in the midi file can be
  modified independently (i.e., can patch to
  choices of several general instruments as well as
  custom instruments)
• MIDI can also be used for music notation
  (Finale, Cakewalk)
• How commonly used is midi?
• Midi is the industry standard for digital music.
  Almost all professional (and high end consumer)
  digital keyboards support midi, as well as
  professional audio recording suites

12
Converting to Musical Instrument Digital
Interface continued

• How does the midi format work?
• information about the track is stored in a
  header
• information about the composition (key, tempo,
  etc.) can be stored in a track chunk that
  applies to the whole file
• information about notes can be stored into
  several track chunks. This can allow for
  multiple instruments to play at the same time.
• note information is as simple as note on and
  note off. The note on is given a specific
  time offset from the previous note, and the note
  off is given a time offset from its
  corresponding note on.
• there are many extra features that support
  specifics of different instruments such as soft
  pedal, decay of note, vibrato settings, etc
• How do midi synthetic sounds work?
• general MIDI sounds mimic real instruments by
  generating harmonic components associated with
  specific instrument timbres

13
findPitch.m

• findPitch.m
• This is a demonstration file which takes
• the name of a wave file and detects the
  fundamental
• frequency and returns this number in
• analog freq.
• function findPitch(filename)
• Get File
• note,samprate,bitdepth wavread(filename)
• Get Size of file
• notelen length(note)

• Take the fft
• fftNote (abs(fft(winNote))).2
• figure, plot(fftNote(1halflen))
• title('fft of the note')
• Go through the fft and downsample/multiply (3
  times)
• fftNote2 fftNote
• fftNote1 fftNote
• fftNote3 fftNote
• downsample(fftNote1,2)
• downsample(fftNote2,3)
• downsample(fftNote3,4)
• totalfft fftNote.fftNote1.fftNote2.fftNote3
• figure, plot(totalfft(1halflen))
• title('Fundamental Frequency Selected')
• Find the freq of max peak
• Y,I max(totalfft(1halflen))

14
Hanning Window
15
findPitch ( C4, 261.6255653006 Hz, piano)
Example for middleC.wav
findPitch middleC.wav TheNote 262.1984
16
findPitch ( A0, 27.500 Hz, piano)
Example for A0Piano.wav findPitch('A0Piano.wa
v') TheNote 55.4950
Note This corresponds to A1, not A0. The HPS
algorithm makes the error of choosing a frequency
one octave higher than the fundamental this is
because for low frequencies we see two large
spikes, with the higher frequency being greater
in amplitude. However, if the smaller, lower
frequency spike is greater than half of the
larger, higher frequency spike, the lower spike
is the true fundamental frequency.
17
MidiGen
This MIDI file was generated with a Matlab
function. You can see the portability of the
MIDI format as we exported the MIDI file to Lime
which transcribed the MIDI file to musical notes.
18
Timeline

• decide on DSP output format and achieve serial
  output ability (the week of 4/12/04)
• implement pitch detection and correction on the
  DSP (the week of 4/19/04)
• modify MidiGen for determined format (the week
  of 4/19/04)
• Extras
• experiment with polyphonic sounds (starting
  4/19/04)
• modify project for polyphonic detection if
  possible (starting 4/19/04)

Goals