Introductory Computer Programming, Problem Solving and Computer Assisted Assessment

1
Introductory Computer Programming, Problem
Solving andComputer Assisted Assessment
Charlie Daly, DCU John Waldron, TCD
2
Preview A Problem
A Programmable Robot
3
(No Transcript)
4
A Problem

• Solving a maze.
• Program a Robot to Solve a maze
• Analyse a program that is supposed to solve a
  maze

5
But ...

• Anybody can check if the program works.
• Create a maze and check if the robot can solve
  it.
• And provide feedback if it doesnt work
• Show what the robot did.

6
So ...

• Certain problems are very challenging ...
• ... but they can be checked very simply
• ... easy to provide useful feedback to faulty
  programs
• End of Preview

Mathematical proofs similar, easy to check,
difficult to conceive
7
Programming is Problem Solving

• "programming is a complex intellectual activity
  based on deep logical principles" (Felleisen
  2002)
• Polya's "How to Solve it" "sheds more light on
  the activity of software design than any
  software design book I've seen" (McConnell 2002)

8
Learning Programming

• Learner programmers must master
• programing language syntax
• problem solving techniques.
• Computer feedback is useful for both
• Compiler catches syntax errors
• Testing catches logic errors

9
Talk Outline

• Programming Courses are not working.
• Programming ability is difficult to assess
• The Solution Proper Assessment
• Implementation issues (software and peopleware)
• Results
• Conclusions

10
Programming Courses are not working

• Students in Introductory Programming Courses do
  not learn to program!

An international multi-institutional study of
introductory programming courses ITiCSE 2001
Do students in introductory computing courses
know how to program at the expected skill level?
11
No!
12
What is Wrong?
Assessment
"The spirit and style of student assessment
defines the de facto curriculum" Assessing
Students, Derek Rowntree '77
13
What is Wrong?

• It is difficult to assess programming ability in
  a traditional written exam.
• Programming exercises are subject to plagiarism
  a serious problem in introductory programming
  courses.
• If you do not assess something, the students will
  not learn it.

14
What's wrong with Exams

• Two sides of programming
• language syntax (easy to examine)
• problem solving (hard to examine)
• Unoriginal (repeat) questions
• Marks for attempting a question
• Assuming insight where none exists
• Objectivity

One exam
Lecturer doesn't want to fail whole class.
15
What's wrong with Assignments

• Most students don't see a problem with using
  somebody else's code ("if I understand it it's
  OK")
• Plagiarism is a huge problem.
• Lecturers frequently know it's happening and do
  nothing.

don't understand there's a difference between
writing and understanding
16
The Bright Shining Lie

• Lecturers think the students know how to program
• they passed the exams
• Students think they know how to program
• they passed the exams
• Students have an excuse for students education
  is about passing exams

17
Talk Outline

• Programming Courses are not working.
• Programming ability is difficult to assess
• The Solution Proper Assessment
• Implementation issues (software and peopleware)
• Results
• Conclusions

18
The Solution Proper assessment

• Come up with original challenging problems.
• Mark properly only give marks if the solution is
  completely correct.
• Allow the student to get computer feedback.
• (compiler errors, testing) programming is a
  process
• Make students aware of the assessment!

19
Context the course

• Introductory Programming Course
• one semester 12 weeks of lectures
• Students
• 400 (300 pure Computing)
• no prior programming experience
• education passing exams
• Exams
• programming exams week 6 and 12 (Weighting 30)
• written exam week 16 (Weighting 40)

20
RoboProf

• Automated Program marker
• WWW interface
• Runs a student's program using different input
  and check that the program produces the correct
  output.
• Student is shown the result and (if not correct)
  may modify and resubmit their program.

more
21
RoboProf
The student writes a program and submits it
Server
An applet compiles and runs the program locally
22
The Programming Exam

• 3 Questions (in increasing difficulty)
• 2 hours (first half hour without computer)
• Not open book, but may use a 'cheatsheet'
• During the exam the students
• Write a program
• Submit it to RoboProf
• View feedback
• May resubmit without penalty

23
The Programming Exam

• Students know their result at the end of the
  exam.
• Fewer complaints
• Greater insight into their ability
• More likely to listen when subsequently shown a
  correct solution
• General effect formative assessment works.

24
RoboProf Vs Manual Exam

• Standard CAA advantages
• Objective
• Fast Feedback
• Avoids the problem of the manual marker
  interpreting the student solution
• Models standard program development process
  (computer feedback)
• Huge resource requirements (400 PCs)

25
(No Transcript)
26
(No Transcript)
27
RoboProf Vs Manual Exam

• Results Not much difference.
• Markers only awarded marks for programs that
  definitely looked like they might work.
• Students knew that the marking system was similar
  to the programming exams.

28
Student Impressions
29
Student Impressions
"I liked the web-based test"
30
Student Impressions
"The marking system was fair"
31
Comparison with previous years

• There were too many simultaneous changes to the
  course to draw firm conclusions (programming
  language, number of students, lecturers,
  facilities)
• Results of last three years
• 1999 30
• 2000 20
• 2001 ?

32
Conclusions

• CAA can evaluate problem solving ability
• Using approriate problems
• Programming problems have added advantages
• Models the program development process.
• Can show errors without showing the solution.
• Avoids the inherent problems that humans have
  assessing programming problems

33
Future Work

• Improve Feedback
• Make it adaptable (can be used by other
  institutions)

34
Question 1 Write a program that asks for a price
and makes an assessment if the price is more
than 100.00, then print "too high", if it is
less than 50.00 print "that's cheap" otherwise
print "OK" A sample run might be What is the
price? 120.99 too high or What is the
price? 51.50 OK
35
Question 3 (Haggling) A street seller needs a
program to help him haggle. He only sells one
item, worth 100, as soon as he's offered this
(or more) he accepts the offer. If he's offered
less, he demands a price such that the average of
the offer and the demanded price is 100. He
keeps this up until a deal is done. A sample run
might be What is your offer? 400 It's a
deal! or What is your offer? 50 I'll
sell it for 150. What is your offer? 90
I'll sell it for 110. What is your offer? 100
It's a deal! You can assume that a deal is
always done. Note that the first demanded price
was 150. This was in response to the offered
price of 50 (the average of 150 and 50 is 100).
36
Question 1 Write a program that reads in a
sequence of integers and calculates the average,
then prints out how many elements are greater
than the average. A sample run might look
like How many numbers 7 Enter 7 numbers -6 8
-17 67 10 -7 0 3 numbers are greater than the
average.   Note that the average is about 7.9,
the elements 8, 67 and 10 are greater than that.
37
Question 2 Write a program that finds the tags in
a String of characters. Print each tag on a
separate line. Tags are contained in braces ''
and ''. You may assume that the String is well
formed, i.e. all braces match.   A sample run
might look like Enter a string Here is a tagged
String one and another. The tags
are One another   Hint You may find the
following String methods useful.