Determining Scribble Intention in Handdrawn Musical Notes

1
Determining Scribble Intention in Hand-drawn
Musical Notes
Katie Dahmen, Doctoral Student in Computer
Science, Texas AM University Dr. Tracy Hammond,
Assistant Professor in Computer Science, Texas
AM University

• PROJECT GOALS
• Recognize hand drawn musical notes
• Distinguish between scribble gestures meant to
  color or fill in a shape and those meant to
  delete a shape

Recognizing Scribble Type
Recognizing Music Tab
Deleted notes together with filled-in notes.
Users will be able to color in or scribble
out (delete) a note. Color (drawn by scribbling)
is used to distinguish between half notes and
quarter notes (as shown to the left). A similar
scribble gesture specifies that a shape should be
deleted. We would like to be able to distinguish
between these two gestures, not only in this
domain, but provide a general method for
distinguish between the two that is applicable in
many domains.
We intend to create a program that allows the
user to sketch a few bars of sheet music, then
hear the music played back to match what is drawn.
BACKGROUND Sketch recognition is the area of
artificial intelligence dealing with recognizing
the intended meaning of input drawn with a
pen-based system, such as with a Tablet PC.
IMPLEMENTATION We will use LADDER (a language to
define how shapes are drawn, displayed, and
edited in a domain for use in sketch recognition)
to describe the shapes, and GUILD (a system that
automatically generates user interfaces from
LADDER descriptions) to recognize to recognize
basic primitive shapes, such as circles and
lines.
IMPLEMENTATION We will perform a user study to
determine how people sketch deletion scribbles
and coloring scribbles differently. We expect to
find that context plays an important part in
recognizing the users intention. The coloring
gesture may be bounded by a previously drawn
shape. Information such as the speed, curvature,
changing direction of the drawn stroke may help
indicate the scribble type.
Different styles of coloring-in and
scribbling-out.
Scribbling out a note v.s. scribbling out a whole
bar of music.