Assignment Marking via Online Self Assess

1
Assignment Marking via Online Self Assess

• Margot Schuhmacher,
• Lecturer Higher Education Development Unit,
  Centre for Learning and Teaching Support, Monash
  University
• Robert Redpath,
• Lecturer School of Computer Science and Software
  Engineering, Faculty of Information Technology,
  Monash University

2
Overview

• The Role of Assessment
• The Assessment Process
• Assessment Criteria as a Quiz
• Evaluation of the New Process by Students and
  Tutors
• Conclusions

3
Introduction

• Assessment for formal accreditation and student
  learning
• Student as primary stakeholder in the process
• Assignment submission via a learning management
  system

4
The Assessment Process

• The LMS replaces a traditional paper based
  process
• This provides benefits
• Flexible student submission
• Persistent record
• BUT the student view of the marking process is
  still as a black box

5
Black Box View
6
The Tutor and an LMS

• Assignment is online
• Marking guide is separate
• The process is tedious
• Mark entry is separate step

7
Marking Guides

• Represented in many forms
• What if it is presented as a simple rubric?

8
Rubric Score for Essay Structure
Amateur Critical Essential Desirable Extension
Descriptor Essay seems to have coherent structure Essay structure has an introduction body and conclusion All critical elements and a discussion All desirable elements and extended with analysis
Points 1 2 3 4
Select X
9
Desired Improvements

• Can the technology incorporate marking criteria
  into the assignment?

10
Assignment tool in Vista

• Undergraduate database unit
• Students needed to implement a database
• Assessed on logical and physical design
• Use of constraints
• Valid syntax
• Results of queries

11
Assessments Tool in Vista

• Quizzes question types include
• Multiple choice (single and multiple selection)
• True False
• Short Answer
• Open-ended (paragraph)
• Feedback and results can be returned to students

12
Combine two tools?
13
Marking Criteria Examples

• The constraints were
• 1. Not explained
• 2. Had a poor explanation
• 3. Had a good explanation
• 4. Had a very good explanation
• Did you use sensible sample data that allowed the
  queries to produce output?
• 1. Yes
• 2. No

14
Further Examples

• Which of the following SQL Features did you use
  correctly and successfully in one or more of your
  queries? Please select all features used.
• Compound/complex search conditions
• LIKE search criteria
• Date function (such as TO_CHAR, TO_DATE)
• Join 2 tables
• Join 3 or more tables
• ORDER BY clause
• GROUP BY with aggregate function
• Sub-queries

15
A Demonstration

• Using WebCT Vista (MUSO)

16
A White Box View

• The student sees the evaluation of their
  assignment specifically related to the marking
  criteria

17
The Evaluation Process

• Trialled in undergraduate database unit
• Anonymous surveys of students and tutors
• 76 assignments
• 46 self assessments
• 12 student evaluations
• 3 (out of 4) tutor evaluations

18
Student Issues

• Preference for the new system?
• Ease of the process?
• Value of the process?
• Indication of performance from the quiz?
• Variation in grade expectation when result known?

19
Greater Student Satisfaction

• Early and strong indications of assessment
• Improved student feedback at both times of self
  assessment and assessment confirmation
• A greater understanding of the criteria applied
• Improved learning

20
Tutor Issues

• Ease of marking in comparison to previous
  approaches?
• Ease of management?
• Ease of providing feedback?
• Clarity of marking guide in comparison to other
  units?
• Preference for the process?
• Need to override the student score?
• Direction of adjustments

21
Tutors Prefer the Approach

• Ensures a clear assessment criteria
• Moderation carried out by confirming or
  overriding student self assessment
• An imposed structure exists for persistent
  records easing further moderation

22
Findings

• The criteria must be described clearly
• Improves consistency
• The student receives immediate feedback
• Self assessment is moderated by the tutor
• Persistent and central record
• Ease of moderation

23
Conclusions

• An improved assessment process
• Clearer criteria
• More immediate feedback
• Improved learning experience
• A persistent record of the entire process
• Greater efficiencies in the process

24
Future Work

• The process trialled again in 2005
• Process improvements
• Process training/education required
• Investigate other forms of assessment criteria