Automated Assessment of Sketched Diagrams

1
Automated Assessment of Sketched Diagrams
Presentation by
Tracy A. HammondBrandon C. PaulsonJoshua B.
Johnston Joshua M. Peschel
Department of Computer ScienceTexas AM
University College Station
2
BACKGROUND
An example of Sketch Recognition
http//srl.csdl.tamu.edu/videos/mecheng.avi
3
PRESENTATION OUTLINE

• Today we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

4
PRESENTATION OUTLINE

• First we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

5
BACKGROUND
Graphics are pervasive throughout education,
business, engineeringalmost everywhere!
6
BACKGROUND
Diagrams are a natural way to describe a variety
of information. So what is this?

• Body Polygon, center (2.79, -3.98), points
  (1,-2.4) ? (4.89,-4.96) ? (4.71, -5.37) ? (.8,
  -5.56) ? (0.69, -2.66)
• Body Polygon, center (2.29, -1.91), points (2,0,
  -1.1) ? (2.89, -1.72) ? (2.47, -2.04) ? (3.21,
  -2.18) ? (2.77, -2.72) ? (1.37, -1.9)
• Body Circle, center (2.49, -2.43), radius .35
• Body Circle, center (1.78, -1.99), radius .3
• Pinjoint center (1.83, -1.95)
• Pinjoint center (2.54, -2.39)
• Anchor center (1.6,-4.36)
• Gravity on

7
BACKGROUND
Diagrams are a natural way to describe a variety
of information. Which is more intuitive?
8
BACKGROUND

• Computer-Aided Drawing (CAD) systems provide a
  lot of power for creating diagrams

? Wide variety of information? Understood
by computers? Can be animated? Quick
feedback
But
9
DISCUSSION PREMISE

• There is something special about a blank sheet of
  paper

? Natural expression? Drawing
freedom? Sketcher is creator (i.e. active
learning)? Visualization
Sketch Recognition allows us to capture all of
these things
10
BACKGROUND

• Computer-Aided Drawing (CAD) systems provide a
  lot of powerbut

? Not freeform drawing? Requires
training? Time better spent learning
content? Restrict creativity
11
BACKGROUND
A simple equation Sketch Recognition
Creativity Function where
Creativity Freeform drawings Function CAD
system understanding
12
BACKGROUND
Question What is Sketch Recognition?
Answer Identification of hand-drawn shapes.
STAR
13
PRESENTATION OUTLINE

• Sketch Recognition allows us to
• ? Capture what is drawn by a user? Interpret
  what was drawn? Assess the meaning of the
  drawing

14
BACKGROUND
In the context of real-time educational
assessment Assessment is time-consuming. Solutio
ns ? Scantrons? Online multiple choice
questions?Sketched diagrams are often omitted
from real-time assessment processes
15
BACKGROUND
Sketched diagrams are often omitted from
real-time assessment processes But Testing and
feedback are important ? Testing combined with
study is more effective than studying alone
Roediger? Real-time feedback is crucial to the
learning process? CAD systems provide
real-time feedback and thus provide
functional visualization
16
BACKGROUND
The benefits of Sketch Recognition ? More
natural design that encourages
creativity? Enhance student learning through
animated application and real-time
feedback? Reduce teacher time by automating
answer corrections and question
generation? Enhance teacher feedback by
real-time collation of grades showing
mistake themes
17
BACKGROUND
A simple equation Sketch Recognition
Creativity Function where
Creativity Freeform drawings Function CAD
system understanding
18
BACKGROUND
Is Sketch Recognition hard to implement? ? Requir
es programming expertise? Often a lengthy build
time involved? Definitely impractical for
instructor But, does it have to be this
way? No! We have made significant progress to
overcome these barriersnow for some fun.
19
PRESENTATION OUTLINE

• Now we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

20
SHAPE LEARNING APPLICATION
? Children learn through sketching? Address
children through kinesthetic and
tactile learning? Recognition with
PaleoSketch
21
SHAPE LEARNING APPLICATION
http//srl.csdl.tamu.edu/userstudy/childre
n.shtml
22
GEOGRAPHY APPLICATION
? Real-time assessment of geographic
skills? Allows corrections? Can be easily
expanded
23
GEOGRAPHY APPLICATION
http//srl.csdl.tamu.edu/geography.shtml
24
PRESENTATION OUTLINE

• Next we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

25
SENTENCE DIAGRAMMING
And now another demonstration
http//srl.csdl.tamu.edu/videos/sentenceDiagrammin
g.wmv
26
PRESENTATION OUTLINE

• Now we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

27
ENGINEERING APPLCIATION
Readiness Assessment Tests (RATs)? Formative
assessment tools (incremental)? Usually to test
one or two specific concepts? First introduced
by Michaelson1 for business? Introduced into
engineering in early 1990s? Basically a pop quiz
with a two-way purpose 1Michaelson, L.K., W.E.
Watson, J.P. Cragin and L.D. Fink. 1982. Team
Learning A Potential Solution to the Problems of
Large Classes. Exchange Journal of
Organizational Behavior, 713-22.
28
ENGINEERING APPLCIATION
Readiness Assessment Tests (RATs)? Standard
medium is paper-based? Feedback delays caused by
need for grading? Classroom content therefore
hard to adjust ? Students may miss critical
knowledge points
29
ENGINEERING APPLCIATION
Readiness Assessment Tests (RATs)? Some near
real-time RATs developed2? Scantrons used and
graded in class? Course management systems
available? BlackBoard and WebCT could be
used? Features and cost may not be
practical 2Yost, S., D.J. Lane and G. Blanford.
2005. 2006-2306 RATS Students Working in Teams,
Do They Really Benefit? In Proceedings of the
2005 ASEE Annual Conference, Portland, Oregon.
30
ENGINEERING APPLCIATION
Our goal for this system is to enhance learner
understanding within a large, two-semester
freshman engineering course at Texas AM
University.
31
ENGINEERING APPLCIATION
The flow of information can be described as
32
ENGINEERING APPLCIATION
The quiz interface for the learner
33
ENGINEERING APPLCIATION
The graphic results output displayed to all
34
ENGINEERING APPLCIATION
But its not quite real-time, is it?
Sketch Recognition
35
ENGINEERING APPLCIATION
Sketch Recognition can be used for creating quiz
content in real-time
http//srl.csdl.tamu.edu/videos/STRAT_Final.mov
36
PRESENTATION OUTLINE

• Finally we will be discussing
• Background Information
• Geography Education Application
• Sentence Diagramming Application
• Engineering Education Application
• Summary and Ongoing Work

37
SUMMARY AND ONGOING WORK
Over 30 applications built so far across a myriad
of domains ? Mechanical engineering,
electrical circuits, military course of
action diagrams, finite state machines, music
notation, math notation, device creation,
flow charts, games, chemistry diagrams,
Japanese kanji, many more
38
SUMMARY AND ONGOING WORK
Over 20 applications have been built by students
in HCI / sketch recognition courses as a
prototyping tool
http//srl.csdl.tamu.edu/videos/kanji.avi
39
SUMMARY AND ONGOING WORK
In summary ? Looking for collaborators
interested in implementation!? Please visit
our website http//srl.csdl.tamu.edu
40
SUMMARY AND ONGOING WORK
SRL Members ? Dr. Tracy Hammond, Brandon
Paulson, Josh Johnston, Katie Dahmen, Brian
Eoff, Pankaj Rajan and Aaron Wolin
Funding ? NSF Developing Perception-Based
Geometric Primitive-Shape Recognizers to
Empower Instructors to Build Sketch Systems
in the Classroom
41
Automated Assessment of Sketched Diagrams
Questions or Comments? For more information
contactDr. Tracy Hammond hammond_at_cs.tamu.edu
Department of Computer ScienceTexas AM
University College Station