Audio Information Retrieval

1
Audio Information Retrieval

• By Georgi Dzhambazov

2
Outline

• Physical basics of sound and digital audio
• Audio processing and feature extraction
• Music Information Retrieval

3
Physics of waves

• Sound is a mechanical wave
• created by vibrating objects
• propagated through a medium
• carries vibrating particles
• pressure causes waves
• Sound is a waveform
• Sinusoidal waveform

4
Anatomy of waves

• Crests and troughs
• Amplitude - displacement from rest
• Period and frequency
• Formula
• Angular frequency

5
Analogue Sound Recording

• Microphone
• consists of a vibrating membrane called diaphragm
• translates physical movements into electrical
  signals

6
Digitizing Signal

• Quantization as series of samples
• At discrete equal time intervals sampling rate
• e.g. CD quality 44100 Hz
• At finite size of sample bit depth
• e.g. CD quality 16 bits
• Bitrate number of bytes per second
• e.g. CD quality 2 channels x 44100 samples x 16
  bits 1378 Mb per second

7
Time domain

• Sound as a function of time
• Sound is recorded in Time domain
• Any waveform can be expressed as a sum of
• Pure sound waves
• At different frequency

8
Frequency Domain

• Sound as a function of frequency
• Plots the amplitude of each frequency present in
  the signal
• Power spectrum the combination of all
  frequencies constituent waves
• Calculated
• for a short interval
• called window

9
Transformations between the domains

• Fourier Transform - mathematical transform for
  analogue signal
• Calculates the spectrum for a given window of
  time domain signal
• Usually applied to a window of 1024 samples
• Inverse Fourier Transform
• Given a power spectrum it restores the original
  signal in time domain
• NB
• Discrete Fourier transform(DFT) discrete
  equivalent of FT for digital signal
• Fast Fourier Transform (FFT) algorithm for fast
  computation of DFT

10
Showcase in Audacity I

• Sound waveform generation
• Power spectrum
• Used in MPEG 1 Layer 3 encoding

11
Outline

• Physical basics of sound and digital audio
• Audio processing and feature extraction
• Music Information Retrieval

12
Digital Signal Processing

• Analyzing and transforming digital audio
• DSP cycle
• Can be applied in time or frequency domain
• Effects
• Filtering
• Reverberation
• Special effects

13
Showcase in Audacity II

• Effects in time domain
• Creating karaoke
• Changing tempo
• Effects in Frequency domain
• Changing pitch

14
Feature extraction

• Retrieving useful features from a sound snippet
• Important Features
• Fundamental frequency - pitch
• Audio Power
• Chroma keys
• Onsets

15
Showcase in jaudio

• Fundamental frequency extraction

16
Outline

• Physical basics of sound and digital audio
• Audio processing and feature extraction
• Music Information Retrieval

17
Music Information Retieval (MIR)

• Information Retrieval
• Science of indexing and searching for content in
  data
• Includes classification, clustering, similarity
  measurement, pattern recognition, data mining
• MIR Information Retrieval from Music
• Based on metadata and extraction of low-level
  features
• Aggregating low level features
• Reveals high-level patterns
• Reduces dimensionality/complexity

18
Applications of MIR

• Beat detection
• Acoustic fingerprinting
• Query by singing
• Audio Semantic Webs
• Artist /Genre classification

19
MIR example I

• Liveplasma

20
MIR example II

• coMirva

21
MIR example III

• Last.fm

22
References

• Introduction to Digital Signals
• http//www.bores.com/courses/intro/
• http//www.musiq.com/recording/digaudio/index.html
  
• Ken Steigzlitz A DSP Primer
• Java sound library resources
• http//www.dickbaldwin.com/tocadv.htm
• http//www.jsresources.org/links.html
• Audacity
• http//www.edhsonline.org/other/audacity/
• http//audacityteam.org/manual/index.php?titleMai
  n_Page

23
Questions