A Musical Data Mining Primer

1
A Musical Data Mining Primer

• CS235 Spring 03
• Dan Berger
• dberger_at_cs.ucr.edu

2
Outline

• Motivation/Problem Overview
• Background
• Types of Music
• Digital Representations
• Psychoacoustics
• Query (Content vs. Meta-Data)
• Categorization Clustering
• Finding More
• Conclusion

3
Motivation

• More music is being stored digitally
• PressPlay offers 300,000 tracks for download
• As collections grow organizing and searching
  manually become hard
• How to find the right music in a sea of
  possibilities?
• How to find new artists given current
  preferences?
• How to find a song you heard on the radio?

4
Problem Overview

• Music is a highly dimension time series
• 5 minutes _at_ CD quality gt 13M samples!
• It seems logical to apply data mining and IR
  techniques to this form of information.
• Query, Clustering, Prediction, etc.
• Application isnt straightforward for reasons
  well discuss shortly.

5
Background Types of Music

• Monophonic one note sounds at a time.
• Homophonic multiple note sound all starting
  (and ending) at the same instant.
• Polyphonic no constraints on concurrency. Most
  general and difficult to handle.

6
Background Digital Representations

• Structured (Symbolic)
• MIDI stores note duration intensity,
  instructions for a synthesizer
• Unstructured (Sampled)
• PCM stores quantized periodic samples
• Leverages Nyquist/Shannons sampling thm. to
  faithfully capture the signal.
• MP3/Vorbis/AAC discards useless information
  reduces storage and fidelity
• Use psychoacoustics
• Some work at rediscovering musical structure.

7
Background Psychoacoustics

• Two main relevant results
• Limited, freq. dependant resolution
• Auditory masking
• We hear different frequencies differently
• sound spectrum broken into critical bands
• We miss signals due to spectral /or temporal
  collision.
• Loud sounds mask softer ones,
• Two sounds of similar frequency get blended

8
Query Content is King

• Current systems use textual meta-data to
  facilitate query
• Song/Album Title, Artist, Genre
• The goal is to query by the musical content
• Similarity
• find songs like the current one
• find songs with this musical phrase

9
Result Query By Humming

• A handful of research systems have been built
  that locate songs in a collection based on the
  user humming or singing a melodic portion of the
  song.
• Typically search over a collection of monophonic
  MIDI files.

10
Content Based Query

• Recall music is a time series with high
  dimensionality.
• Need robust dimensionality reduction.
• Not all parts of music are equally important.
• Feature extraction remember the important
  features.
• Which features are important?

11
Similarity/Feature Extraction

• The current hard problem there are ad-hoc
  solutions, but little supporting theory.
• Tempo (bpm), volume, spectral qualities,
  transitions, etc.
• Sound source is it a piano? a trumpet?
• Singer recognition whos the vocalist?
• Collectively Machine Listening
• These are hard problems with some positive
  results.

12
Compression Complexity

• Different compression schemes (MP3/Vorbis/AAC)
  use psychoacoustics differently.
• Different implementations of a scheme may also!
• Feature extraction needs to be robust to these
  variations.
• Seems to be an open problem.

13
Categorization/Clustering

• Genre (rock/rB/pop/jazz/blues/etc.) is manually
  assigned and subjective.
• Work is being done on automatic classification
  and clustering.
• Relies on (and sometimes reinvents) the
  similarity metric work described previously.

14
Browsing Visualization

• LOUD
• physical
• exploration
• Islands of Music uses self organizing maps to
  visualize clusters of similar songs.

15
Current Efforts

• Amazon/iTunes/etc. use collaborative filtering.
• If the population is myopic and predictable, it
  works well, otherwise not.
• Hit Song Science clusters a provided set of
  songs against a database of top 30 hits to
  predict success.
• Claims to have predicted the success of Nora
  Jones.
• Relatable musical fingerprint technology
  involved with Napster 2

16
Finding More

• Conferences
• Int. Symposium on Music IR (ISMIR)
• Int. Conference on Music and AI (ICMAI)
• Joint Conference on Digital Libraries
• Journals
• ACM/IEEE Multimedia
• Groups
• MIT Media Lab Machine Listening Group

17
Conclusion

• Slow steady progress is being made.
• Music Appreciation is fuzzy
• we cant define it but we know it when we hear
  it.
• References, and more detail, are in my survey
  paper, available shortly on the web.
• http//www.cs.ucr.edu/dberger

18
Fini

• Questions?