Resolving Ambiguities to Create a Natural Sketching Environment

1
Resolving Ambiguities to Create a Natural
Sketching Environment

• Christine Alvarado and Randall Davis
• MIT AI Laboratory

2
Our Model

• The Designer Sketches with Pen and Paper
• The Observer Interprets the Sketch
• The Observer and Designer Interact

3
Sketch Interpretation
4
Accuracy vs. Freedom
Free Sketch
ASSIST
Single Stroke Recognition
Recognition Difficulty
Put That There
Menu
Drawing Freedom
5
Accuracy and Freedom

• Smarter interpretation increases accuracy

• Better interaction design increases perceived
  freedom

6
Resolving Ambiguities

• Levels of Interpretation
• Fluid Interpretation
• Commitment to an Interpretation

7
3 Stages of Interpretation

• Recognition
• Reasoning
• Resolution

8
Recognition

• Generate All Possible Interpretations
• Circle
• Circular Body
• Pin Joint

9
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

10
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

11
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

1 arrow or 3 rods?
12
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

13
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

14
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

15
Reasoning Heuristics

• Temporal Evidence
• Simpler Is Better
• Context
• Domain Knowledge
• User Feedback

Total Score
16
Resolution
17
Resolution
0
6
3
18
Resolution
0
6
3
19
Resolution
0
0
10
10
5
5
20
Resolution
0
0
10
10
5
5
21
Resolution
0
0
0
0
6
6
6
6
3
3
3
3
10
22
Resolution
0
0
0
0
6
6
6
6
3
3
3
3
10
23
Resolution
0
0
0
0
0
6
6
6
6
3
6
3
3
3
3
8
10
24
Resolution
0
0
0
0
0
6
6
6
6
3
6
3
3
3
3
8
10
25
Resolution
0
0
0
0
0
6
6
6
6
3
6
3
3
3
3
8
10
26
Limitations

• Relies heavily on bottom-up recognition

Line Line Line ??? ? ???
27
Limitations

• Relies heavily on bottom-up recognition
• Heuristics all weighted equally

H1
Prefer interpretations resulting in fewer objects
H2
Prefer objects drawn with contiguous strokes
28
Limitations

• Relies heavily on bottom-up recognition
• Heuristics all weighted equally

H1
Prefer interpretations resulting in fewer objects
H2
Prefer objects drawn with contiguous strokes
29
Limitations

• Relies heavily on bottom-up recognition
• Heuristics all weighted equally

H1
Prefer interpretations resulting in fewer objects
H2
Prefer objects drawn with contiguous strokes
30
Structured Application of Context

• Blackboard recognition architecture
• Heuristics applied probabilistically

31
Recognition Blackboard
Blackboard
Forces push bodies
Force(f1)
Sketch
Arrow(a1)
Connects(l1, l2)
Connects(l4, l5)
Connects(l1, l2)
Connects(l7, l4)
Connects(l1, l2)
Line(l1)
Line(l2)
Line(l3)
Line(l5)
Line(l4)
Line(l7)
Stroke(s2)
Stroke(s1)
Stroke(s3)
Stroke(s5)
Stroke(s7)
Stroke(s6)
Stroke(s4)
32
Bayesian Network Structure
Heuristics influence prior

Property1
Property2
Line1
Line2
Line3

Low-level information influences recognition
33
Related Work

• Gross and Do (1996)
• Landay and Meyers (2001)
• Stahovich (1998)
• Matsakis (1999)