FeatureBased Sketch Recognition Using Geometric Features

1
Feature-Based Sketch Recognition Using Geometric
Features

• Pedro Davalos
• Brandon Paulson
• Pankaj Rajan

2
Outline

• Background
• Goals
• Approach
• Features
• Results/Observations
• Conclusions

3
What is sketch recognition?

• Recognizing hand-drawn sketches in real-time,
  typically using a Tablet PC, SmartBoard, or other
  pen-based input
• Motivation sketching is a more intuitive form of
  interaction when using traditional CAD tools

4
What is sketch recognition?

• Typical domains
• UML diagrams
• Mechanical engineering
• Sheet music
• Finite State machines
• Many more

5
What is sketch recognition?

• Typical domains
• UML diagrams
• Mechanical engineering
• Sheet music
• Finite State machines
• Many more

6
What is sketch recognition?

• Typical domains
• UML diagrams
• Mechanical engineering
• Sheet music
• Finite State machines
• Many more

7
What is sketch recognition?

• Typical domains
• UML diagrams
• Mechanical engineering
• Sheet music
• Finite State machines
• Many more

8
What is sketch recognition?

• Typical domains
• UML diagrams
• Mechanical engineering
• Sheet music
• Finite State machines
• Many more

9
Typical Approaches

• Gesture/Feature-based classification
• Earliest approach to sketch recognition
• 2-D gesture recognition
• Traditional features are user-dependent
  individual training required
• Focuses on HOW a sketched stroke was drawn rather
  than WHAT it actually looks like

?
10
Typical Approaches

• Geometric-based classification
• Most recent paradigm in sketch recognition
• Recognize low-level primitive shapes and
  construct higher level shape hierarchically
• High level shapes depend on accurate low-level
  interpretations

11
Problem Statement

• Classify a single-stroke (X,Y,T) into 9
  low-level, geometric primitives

Line
Ellipse
Curve
Circle
Polyline
Arc
Spiral
Helix
Complex
12
Goal
13
Goal

• Provide accurate classification of shapes into
  the 9 geometric primitives
• Determine which geometric and gestural features
  are most significant (remove redundant features)
• Show that our approach is user independent
  (robust to various user drawing styles)

14
Data

• 1800 single stroke sketch samples
• 20 different users (90 samples from each user
  10 of each primitive shape)
• Each sketch sample contains a series points
  (x,y,t) that define the sketch.
• x and y are the screen coordinates
• t is the timestamp
• Each sketch sample is labeled with its
  corresponding class (based on what the user was
  asked to draw)

15
Feature Set

• 44 total features (31 geometric, 13 gestural)
• all features can be computed in real-time
• Example geometric features
• feature area (of line, ellipse, arc, etc.)
• least squares error (of each primitive)
• normalized distance between direction extremes
• direction change ratio
• Example gestural features
• stroke length
• sin/cos of the starting/ending angles
• total/absolute rotation
• bounding box length

16
Data Analysis

• Classification Data 82 Train, 18 Test
• Quadratic Classifier Regularization
• ?reg ?orig ? I

96.6 correct 94.6 correct
77 correct
17
Quadratic Classifier

• 100 experiments with random data selection 82
  train, 18 test
• With all 44 features

Avg 96.3 Avg 96.7
Avg 79
18
Feature Subset Selection

• Sequential forward selection with quadratic
  wrapper

Best Rate 98.33 (31 features) Best
Rate 99.33 (20 features) Best Rate
96.67 (10 features)
19
Feature Subset Selection

• Gestural features removed (7/13)
• Stroke length
• Cosine of starting angle
• Length of bounding boxs diagonal
• Cosine of ending angle
• Sin of ending angle
• Absolute rotation
• Rotation squared

20
Feature Subset Selection

• Geometric features removed (6/31)
• Average curvature
• Line fit least squares error
• Line fit feature area error
• Arc fit radius length
• Spiral fit radius length
• Ellipse fit feature area error

21
Cross Validation

• Classification using selected features from each
  data space
• 82 train, 18 test

Avg 96.2
Avg 95.7 Avg 94.2
Orig 96.3
Orig 96.7
Orig 79 Features 31/44
Features 20/44
Features 10/44
22
Ensemble Learning

• Parallel ensemble with voting

23
Ensemble Learning Results
24
User/Style Robustness Analysis

• How well does a user perform as a single trainer
  of the classifier?
• How well does a user perform as a single tester
  of the classifier?
• How many users/styles do we need to give good
  recognition results?
• What are some relationships we can derive between
  different users and their styles?

25
One user to train, others to test

• How well does a user perform as a single trainer
  of the classifier?

26
One user to test, others to train

• How well does a user perform as a single tester
  of the classifier?

27
User Robustness

• How many users/styles do we need to give good
  recognition results?

28
Relationships between users

• What are some relationships we can derive between
  different users and their styles?
• One user to train, one user to test

Similar
Train user
Different
Test user
29
Conclusion

• Geometric features can be used in feature-based
  classifiers to give accurate results
• Certain geometric and gestural features are
  redundant and less significant
• Our approach is user-independent and robust to
  various user styles

30
Future Work

• Collect and test more complex shapes
• Can we also use these features to classify users
  rather than shape class?

31
Questions?