Gamera%20Optical%20Music%20Recognition%20in%20a%20New%20Shell

1
GameraOptical Music Recognitionin a New Shell

• Michael Droettboom, Karl MacMillan Sheridan
  Libraries
• Johns Hopkins University
• Ichiro Fujinaga
• McGill University

2
Content

• Levy Project
• Levy Sheet Music Collection
• Digital Workflow Management
• Optical Music Recognition
• Gamera
• Guido / NoteAbility

3
Lester S. Levy Collection
4
Lester S. Levy Collection
5
Lester S. Levy Collection

• North American sheet music (17801960)
• Digitized 29,000 pieces
• including The Star-Spangle Banner and Yankee
  Doodle
• Database of
• text index records
• images of music (8bit gray)
• lyrics (first lines of verse and chorus)
• color images of cover sheets (32bit)http//levysh
  eetmusic.mse.jhu.edu

6
Digital Workflow Management

• Reduce the manual intervention for large-scale
  digitization projects
• Creation of data repository (text, image, sound)
• Optical Music Recognition (OMR)
• Gamera
• XML-based metadata
• composer, lyricist, arranger, performer, artist,
  engraver, lithographer, dedicatee, and publisher
• cross-references for various forms of names,
  pseudonyms
• authoritative versions of names and subject terms
• Music and lyric search engines
• Analysis toolkit

7
Optical Music Recognition (OMR)

• Trainable open-source OMR system in development
  since 1984
• Staff recognition and removal
• Lyric removal
• Stems and notehead removal
• Music symbol classifier
• Score reconstruction
• Lyric classifier?

8
The problem

• Suitable OCR for lyrics not found
• Commercial OCR systems are often inadequate for
  non-standard documents
• The market for specialized recognition of
  historical documents is very small
• Researchers performing document recognition often
  re-invent the basic image processing wheel

9
The solution

• Provide easy to use tools to allow domain experts
  (people with specialized knowledge of a
  collection) to create custom recognition
  applications
• Generalize OMR for structured documents

10
Introducing Gamera

• Framework for creation of structured document
  recognition system
• Designed for domain experts
• Image processing tools (filters, binarizations,
  etc.)
• Document segmentation and analysis
• Symbol segmentation and classification
• Feature extraction and selection
• Classifier selection and combiners
• Syntactical and semantic analysis
• Generalized Algorithms and Methods for
  Enhancement and Restoration of Archives

11
Features of Gamera

• Portability (Unix, Windows, Mac)
• Extensibility (Python and C plugins)
• Easy-to-use (experts and programmers)
• Open source
• Graphic User Interface
• Interactive / Batchable (scripts)

12
Architecture of Gamera
Graphic User Interface (wxWindows)
Plugins (Python)
Plugins (C)
GAMERA Core (C)
13
Example of C Plugin

• // Number of pixels in matrix
• include gamera.hh
• ifdef __area_wrap__
• define NARGS 1
• define ARG1_ONEBIT
• endif
• using namespace Gamera
• template ltclass Tgt
• feature_t area(T m)
• return feature_t(m.nrows() m.ncols())

14
Example of Python Plugin

• // This filters a list of CC objects
• import gamera
• def filter_wide(ccs, max_width)
• tmp
• for x in ccs
• if x.ncols() gt max_width
• x.fill_matrix(0)
• else
• tmp.append(x)
• return tmp

15
Gamera Interface(screenshot in Linux)
16
Gamera Interface(screenshot in Linux)
17
Histogram(screenshot in Linux)
18
Thresholding(screenshot in Linux)
19
Thresholding(screenshot in Linux)
20
Staff removal Lute tablature
21
(No Transcript)
22
Classifier Lute(screenshot in Linux)
23
Staff removal Neums
24
Classifier Neums(screenshot in Linux)
25
Greek example
26
GUIDO Music Notation FormatH. Hoos, K. Renz, J.
Kilian

• A formal language for score-level
  representation
• Plain text readable, platform independent
• Extensible and flexible
• Adequate representation
• NoteServer Web/Windows
• GUIDO/XML
• NoteAbility (K. Hamel)

27
GUIDO An example
\beamsOff \cleflt"treble"gt \keylt"D"gt
f1/8. g1/16 a1/4. d21/8 d1/4. c1/8
e11/2 _1/4 f1/8. g1/16 c21/4. b11/8
a1/4. g1/8 e1/2 f1/4 f1/8. g1/16
a1/4. d21/8 d1/4. c1/8 e11/2 _1/4
f1/8 g c21/4. b11/8 a1/4. c1/8 ,
28
NoteAbility Demo
29
Conclusions

• Gamera allows rapid development of
  domain-specific document recognition applications
• Domain experts can customize and control all
  aspects of the recognition process
• Includes an easy-to-use interactive environment
  for experimentation
• Beta version available on Linux
• OS X version in preparation

30
Acknowledgements

• National Science Foundation
• Institute of Museum and Library Services
• The Levy Family
• levysheetmusic.mse.jhu.edu

31
(No Transcript)
32
Overall Architecture for OMR
Image File
Staff removal Segmentation
Recognition K-NN Classifier
Output Symbol Name
Optimization Genetic Algorithm K-nn Classifier
Knowledge Base Feature Vectors
Best Weight Vector
Off-line