CALI: An Online Scribble Recognizer for Calligraphic Interfaces

1
CALI An Online Scribble Recognizer for
Calligraphic Interfaces

• By
• Manuel J. Fonseca
• Cesar Pimentel
• Joaqui, A. Jorge

11 November 2002
2
Introduction

• Alternative user interface based on scribble
  recognition
• Recognition of basic shapes (such as triangle,
  rectangles, circles) and basic commands (such as
  copy and delete)
• More robust than others, trainable, multi-stroke

Recognizable Shapes
3
Non-trainable Recognizer

• Global geometric properties
• Filters for identification of shapes and removal
  of unwanted shapes
• Fuzzy logic for uncertainty in sketches
• Strokes ? Scribbles ? Shapes
• Return a list of plausible shapes ordered by
  degree of certainty

4
Geometric features (non-train.)

• Compute the convex hull of shapes points
• Compute largest area triangle, quadrilateral
  inscribed in the convex hull and smallest area
  enclosing rectangle, area and perimeter
• Compute percentiles of distributions of feature
  values for each shape (class)

Polygons
Percentiles
5
Deriving Fuzzy Sets

• Each shape is defined by several fuzzy sets
• E.g. thinness for lines
• For each feature there is a trapezoidal fuzzy set
  with 4 values
• All design of fuzzy sets is done manually

Fuzzy Sets
6
Recognition Rules

• If Scribble is very thin, then shape is a line
• If A(lt)/A(lq) is like diamond and
• A(lq)/A(ch) is not like Ellipse and
• A(lq)/A(er) is not like Bold Line and
• A(lq)/A(er) is not like rectangle
• Then shape is a diamond
• Line style is identified after basic shapes

7
Re-segmentation

• Some shapes cannot be recognized based entirely
  on global geometric properties
• E.g. Arrows and Crosses
• If strokes gt 2 and
• Last stroke is like triangle or
• Last stroke is like move
• Then shape is an arrow
• If strokes 2 and
• First stroke is like line and
• Second stroke is like line
• And first stroke intersects second stroke
• Then shape is a cross

Arrows
8
Ambiguity

• Return a list of plausible candidates when in
  doubt, ordered by degree of membership

Ambiguity
Fuzzy Sets
9
Trainable Recognizer

• Need for new gesture (shape) specification
• Naïve Bays training
• KNN should be better after implementation of
  indexing structure for high-D data (in the works)

10
Results

• Non-train. about 50 ms on a Pentium II
• 95.8 recognition rate
• Train. 93.3

Confusion Matrix
11
Thank you for your attention

• Have a nice evening...

12
Shapes Identified by CALI
13
Polygons used to estimate features
14
Percentiles of feature values

• H(er)/W(er) - aspect ratio
• Bars are for 25-75
• Lines are for 10-90
• Line (small)

• Per(ch)2/A(ch) - thinness
• Bars are for 25-75
• Lines are for 10-90
• Circle (small but visible)
• Lines (so small thats invisible)

15
Fuzzy Sets

• b and c correspond to 10 and 90 percentiles
• a and b are for min and max after clearing out
  outliers

16
Different types of arrows
17
Ambiguity cases among shapes
18
Fuzzy sets for ambiguity
19
Confusion Matrix for non-train.