A

1
A Tutoring System for Novices LearningObject-Orie
nted Design

• Glenn D. Blank
• Computer Science Engineering Department
• Lehigh University, Bethlehem, PA, USA

With support from the National Science
Foundation (Grants No. EIA-0087977 and 0624553)
and the Pennsylvania Infrastructure Technology
Alliance
2
Outline

• A design-first approach to the first CS course
• Elaboration of objects-first approach
• Multimedia support
• Eclipse and LehighUML plug-in support
• Intelligent Tutoring System (ITS) support
• Expert evaluator analyzes each step of UML design
• Student Model evaluates understanding of concepts
• Pedagogical advisor provides feedback
• Evaluation results
• Widening the pipeline of women and minorities
  from middle school to college

3
Whos on first?From syntax to objects to design

• Ongoing debate about how to introduce
  object-oriented concepts to novices
• Syntax-driven approach (traditional)
• Typically starts with Hello, World then
  arithmetic
• Objects-first approach
• Starts by manipulating classes, objects and
  methods
• But objects-first textbooks still tend to
  emphasize coding and syntax rather than object
  think
• Design-first approach
• First learn how to solve a problem
• Novices need to see the big picture
• Software development is more than just coding
• Its understanding a customers problem and
  designing a solution

4
So, what is design-first?

• Students learn object-oriented concepts early
  and apply them from the start

• Given a problem description, students learn how
  to design a UML class diagram
• within first two weeks of a CS1 course
• Using modern software development tools
• Eclipse with DrJava LehighUML

LehighUML plug-in
Eclipse IDE
DrJava plugin
5
Four easy lessons

• Big picture of software development
• Compare building a house to constructing software
• Discussion and small group work with handout (an
  hour)
• Objects and classes
• Multimedia lesson (preferably in lab, 90 minutes)
• From problem description to class diagram
• Step by step procedure based on OO design
  principles
• Design a class, with ITS help
• Lab exercise, takes about 90 minutes

6
Objects and classes multimedia lesson

• Implemented in Flash with graphics, animation
  sound
• Core concepts of classes, methods and attributes
• Eclipse IDE set up for novices
• LehighUML and DrJava
• Manipulateconstructorsand methodsfor shapes
• Create amethod thatdraws ascene

7
From problem description to class diagram (in
eight steps)

• 1. Read the problem description at least twice.
• Ask questions about anything you dont understand
  or needs more explanation.
• (The students receive a problem description and
  ask questions.)
• 2. Put an S next to each subject, a V next to
  each verb and an O next to each object.
• The ATM verifies whether the customer's card
  number and his her PIN (a number) are correct.
• S V
  O
  Oher PIN (a number) are correct. If it is,
  then the customer can check the account balance,
• O
  S
  V O
• deposit cash, and withdraw cash. Checking the
  balance simply displays the account balance.
• V O V O
  S O
  V O
• Depositing asks the customer to enter the
  amount, then updates the account balance.
• S V O
  O V
  O Withdraw cash asks the
  customer for the amount to withdraw if the
  account has enough cash,
• S O V O
  O V
  S V O
• the account balance is updated. The ATM prints
  the customers account balance on a receipt.
• O V
  S V
  O O

8
Analyze subjects and objects

• The ATM verifies whether the customer's card
  number and his her PIN (a number) are correct.
• S V
  O
  Oher PIN (a number) are correct. If it is,
  then the customer can check the account balance,
• O
  S
  V O
• deposit cash, and withdraw cash. Checking the
  balance simply displays the account balance.
• V O V O
  S O V
  O
• Depositing asks the customer to enter the
  amount, then updates the account balance.
• S V O
  O V
  O Withdraw cash asks the
  customer for the amount to withdraw if the
  account has enough cash,
• S O V O
  O V
  S V O
• the account balance is updated. The ATM prints
  the customers account balance on a receipt.
• O V
  S V
  O O
• 3. Analyze each subject and object as follows
• Does it represent a person performing an action?
  Then is an actor, R.
• Is it also a verb (such as deposit)? Then it
  may be a method, M.
• Does a subject take a simple value, such as
  color (string) or money (number)? Then it is
  probably an attribute, A.
• Which remaining unmarked noun occurs most
  frequently? Make it C for class.
• In Eclipse, draw a class for your first class.

A
C
A
R
A
A
A
A
A
A
M
A
A
M
R
A
A
A
M
A
A
C
9
(No Transcript)
10
Analyze methods and attributes

• 4. If a verb has a non-actor object its a
  method, M
• 5. Distinguish attributes, A, and parameters, P
• If it doesnt distinguish instances of a class or
  store information about objects, its probably a
  parameter, not an attribute
• Add attributes to a class in the UML class
  diagram
• Add methods in the class diagram
• While adding methods, include the parameters
• See the return type of your methods
• get methods take the type of the attribute
  returned
• set methods take void return type
• compute methods take the type of the computed
  value
• Methods that answer true/false questions take
  boolean type

11
Any more classes?

• 6. If any remaining classes has no attributes,
  its probably an attribute of another class
• 7. Otherwise, add another class to the diagram
  and go back to step 4 (analyze its methods and
  attributes)
• 8. Draw association links between the classes
• Check your design to make sure it makes sense!

12
Eclipse for novices

• Eclipse
• Open source, so its free!
• Widely used in academia and industry
• Students learn one IDE learn features as needed
• Good error messages and syntax support
• But its complicated for novices to set up and
  learn!
• Multimedia explains how to set it up and get
  started using it
• DrJava plug-in (www.drjava.org)
• Interprets Java expressions from compiled code
• Supports test-driven programming
• LehighUML plug-in
• Students enter class diagram
• Generates code stubs for class automatically
• Instead of syntax, students focus on design

13
Lab exercise Design a movie ticket vending
machine
The movie ticket machine displays the movie
title, displays the show time (e.g. 2006-11-16
700pm), and displays the price of a ticket
(e.g. 8.50). A customer enters money (e.g. 20.50)
into the machine. The machine displays the
customers balance (e.g. 20.50). The customer
enters the number of tickets (e.g. 1) into the
machine. The machine prints the tickets, and
returns the customers balance (e.g. 12.00).
Print tickets also tracks the number of available
seats in the theater so it can tell the customer
when the available seats are sold out.
14
DesignFirst-ITS architecture
15
Expert Evaluator Sally Moritz

• Interprets each step of a students solution
• Matches student step with pieces of expert
  solutions
• Applies concepts in Curriculum Information
  Network
• Produces info packets for rest of ITS
• Allows variations from expert solutions
• Matches synonyms and abbreviations
• An attribute in expert solution can also be
  defined as a class (e.g. class Time with
  TicketMachines attribute showTime)
• Unknown elements (cant be matched to anything
  in problem description or solution) are the most
  problematic
• Pedagogical advisor encourages student to use
  terms related to the problem description

16
Feedback when student enters an unknown class
17
Expert Solution Generator

• Teachers problem description

Suggested solution
18
Expert Solution Generator

• Generates multiple possible expert solutions
• Analyzes natural language problem description
• Algorithm similar to what we now teach novices
• Uses MontyLingua NL parser to tag parts of speech

19
Student Model Fang Wei

• Challenges
• OO design requires understanding of concepts
  (not just procedural rules)
• E.g., class, method, attribute, data type, etc.
• Concepts depend on prerequisites
• E.g., attribute vs. parameter ? attribute,
  parameter
• Student works in an open-ended environment
• LehighUML plug-in provides some scaffolding
• Students learn how to apply concepts in design
• Limited time for updating the Student Model
• EESMPA must respond in real student time
• Bayesian networks can take exponential time

20
Student Model in Wei Blank 2006
(ITS2006) compared with other Bayesian Network
Student Models
Authors System Context Diagnose Concept Pre- requisites Real Time
Murray (1998) Desktop Associate skills v
VanLehn et al.(2001) Solve physics problems rules, not concepts v
Millan et al.(2002) CAT for math v
Butz et al. (2004) C programming v No evaluation
Millan et al.(2005) CAT for math v Post-process
WeiBlank (2006) OO Design (UML) v v v
21
Curriculum Information Network of concepts
int
double
int_string
actor_method
string
double_string
datatype_returntype
object_method
datatype_variable
class
class_constructor
attribute
method
returntype
parameter
method_returntype
A
B
A is prerequisite of B
attribute_parameter
22
Two kinds of concepts (KU)

• Unique concept, such as attribute or parameter
• Relationship concept, such as attribute_parameter
• Relationships emerge because of student
  confusions between concepts
• E.g., student defines movieTitle as a parameter
  when he has already defined movieTitle as an
  attribute

23
Two kinds of concepts (KU)

• Unique concept, such as attribute or parameter
• Relationship concept, such as attribute_parameter
• Relationships emerge because of student
  confusions between concepts
• E.g., student defines movieTitle as a parameter
  when he has already defined movieTitle as an
  attribute

24
Prerequisite relationships

• Prerequisite is relationship between concepts or
  knowledge units
• The concepts a learner needs to understand before
  understanding a concept
• E.g., one needs to under int and double in order
  to understand numeric_datatype
• Relationship concepts are prerequisites of unique
  concepts and vice versa
• E.g., class_constructor -gt constructor
• Understanding constructor doesnt imply
  understanding of class, just how to define a
  constructor for a class

25

Atomic Bayesian Network (ABN)


d-prereq(ku)N

d-prereq(ku)1
d-prereq(ku)2


multiple edges (direct-prerequisite(ku),ku)
Noisy-and generalizes logical-and
Students must understand all direct
prerequisites of the concept ku in order to
understand ku
ku

composed of one edge(ku,au) (concept or knowledge
unit implies action unit or step in student
solution)
A directed graph
au

Since unique concepts (attribute, parameter) and
relationship concepts(how to distinguish between
attribute and parameter) are not transitive, ABN
only needs to consider direct-prerequisites
26
Three-level architecture

• CM recognizes a students cognitive strategies
  (e.g., hacking vs. analogy)
• HM tracks history of students conceptual
  knowledge
• PDM simulates students conceptual knowledge

• Evaluation using 240 simulated students
• Problem Domain Layer
• is 94 accurate
• where accuracy means that given probabilities
  of a simulated
• students understanding of
• relevant concepts, SM correctly predicts
  students solution step
• Adding Historical Model increases accuracy to
  98.5

27
Pedagogical Advisor with Learning StylesShahida
Parvez

• What is an individual learning style?
• The way a learner perceives and interacts with an
  environment.
• Some people like information in the form of
  pictures / images, while others like it in
  written form.
• Some like to work alone while others like to work
  in groups.
• Some prefer concept and theory while others like
  facts
• and concrete experience.
• Visual Learner
• Auditory
• Active Learner Learner

28
Pedagogical Advisor architecture
29
Sample Pedagogical Feedback for Visual Learner
30
(No Transcript)
31
Evaluation

• 49 CS1 students completed four lesson
  introduction to object-oriented design
• After multimedia, students took a multiple choice
  test on object-oriented concepts
• Out of 31 questions, mean of 5.5 errors
• Confirms previous pre-test/post-test results
  demonstrating effectiveness of multimedia (Blank
  et all 2003, 2005)
• All students completed TicketMachine design
• Average graded score of 96.9/100 (1 point per
  error)
• Out of 31 questions, mean of 2.3 errors
• Using paired-samples t test, the T value is
  10.2111, plt.001
• Lab exercise with ITS help had a significant
  effect
• Future studies will do a finer grain analysis
  with control group

32
Evaluation Expert Evaluator

• Overall accuracy 75
• Most errors are false negatives
• Student solution step marked as error, but
  should have been accepted as correct
• Even in such a small problem, students come up
  with reasonable variations
• Example Method seatsAvailable returns true if
  there are enough seats to fulfill customers
  request, false if not
• Possible solution Add feedback loop to Expert
  Evaluator to learn from experience of novel
  correct solutions

33
Evaluation Pedagogical Advisor

• Questionnaire asked students about feedback
• Of 32 returned, 23 received visual feedback
  (textimages), the rest received just text
• based on ILS questionnaire
• 27 paid attention to advice in window
• 23 said they understood the advice
• 20 said it helped them figure out their errors
• 16 of 23 liked images in visual feedback
• 17 of 23 understood information in images
• 9 of 32 found advice annoying
• Observations from student comments
• Students want advice on how to fix the problem
  (beyond hints)
• They want to know correctness of their
  solution(we plan an evaluation button)
• They want occasional positive feedback
• Some suggested that PA be made optional

34
Widening the pipeline

• The pipeline for women and minorities entering
  computer science (CS) and information technology
  (IT) is shrinking, at a time when the projected
  demand for IT professionals is growing
• Camp, 1997, 1999, Cohoon, 2002
• We must take direct action to attract and retain
  more women to computing at all points in the
  pipeline (i.e., K-12, undergraduate, graduate,
  faculty and industry).
• Camp 1999

35
Lehigh Valley STEM

• An NSF-sponsored GK-12 project
• Faculty and Graduate Fellows lead outreach teams
• Work with K-12 teachers to develop new curricula
• Martian landscape with robot rovers created in
  the basement of an Allentown middle school, now a
  NASA Explorer school
• Flash introduced in a Bethlehem middle school
• Design-first curriculum in an Allentown high
  school
• NSF program manager has indicated likely
  recommendation for a continuing GK-12 project
• New project will also feature Outreach Graduate
  supplements for RAs to implement curricula
  developed by Fellow teams

36
START-IT

• Build on success of LV STEM and S.T.A.R.
  Academies
• Promote academic achievement for at-risk middle
  high school students in the Lehigh Valley
• 98 of S.T.A.R. students go to college (now in
  18th year)
• Saturdays (once a month) and summer (three weeks)
• Corporate partners, regional colleges schools,
  and parents
• Focus on Information Technology (IT)
• Connect widen pipeline laid that STEM laid in
  schools
• Self-directed multimedia, intelligent tutoring,
  and one-on-one mentoring by college undergraduate
  and graduate students
• Three teams led by IT faculty and Graduate
  Fellows
• Rising 7th and 8th graders science, math and IT
  missions using remotely controlled robots in a
  simulated Martian landscape
• Rising 9th and 10th graders create web-based
  juke box or iPod as an interactive, animated
  web site using Flash
• Rising 11th and 12th graders learn Java design
  first and get AP Java college credit

37
Thank you!
glennblank_at_gmail.com www.cse.lehigh.edu/glennb ww
w.cse.lehigh.edu/designfirst

• Questions?