Title: Learning From Assessment: Evaluating the Benefits of DALI (Diagnostic Assessment Learning Interface)
1Learning From Assessment Evaluating the Benefits
of DALI (Diagnostic Assessment Learning Interface)
- Hershbinder Mann Guinevere Glasfurd-Brown,
- University of Essex
- 2006
2Background and Context
- The DALI Project is a Teaching and Learning
Innovation Fund (TALIF) pilot study at the
University of Essex. - Development of the student portal and online
formative assessment resources as part of the
SPRInTA Project. - Online formative testing in three large
Departments, primarily to address specific
objectives - large class teaching poor performance, high
drop-out rate, lack of engagement with case study
materials - threshold testing
- improving feedback (and opportunities for
feedback) to students (Nat Student Survey SSS)
3Issues that emerged from past practice
- Student experience
- Aims of the learner? (motivation hoe might we
improve engagement?) - Is there a gap between what students expect from
online formative assessment and what they
get/experience? - How can we encourage a dialogue between staff and
students on assessment? - How might this inform the guidance we provide to
staff?
4Background and Context
Learners have different priorities, preferences
and approaches to learning, and different
requirements for support. The learning
environment needs to reflect these differences.
Understanding how different learners experience
the tasks, resources and services offered to them
is an important precursor to developing
effectively personalised systems. -
Understanding My Learning, JISC
5DALI Project
- DALI tool (issues) enables students to revisit
and query assessment data and identify areas of
weakness (related to specific learning outcomes
at topic level) - Survey of students
- Assessment Think Tank student focus group
- iLAB session which brought staff and students
together to discuss assessment.
6DALI
- Diagnostic Assessment Learning Interface
- DALI is an add-on to Perception. It focuses on
the provision of assessment feedback and provides
students with information aimed at giving them
greater control over their own learning. - The interface
- allows a student to select the score level they
wish to achieve - highlights the last topic-based feedback
provided to the learner - provides statistics associated with any selected
topic and - displays topic descriptions as learning
objectives.
7Using DALI
8Aims
- To identify the factors that affect the uptake
of computer-aided assessment among students - Strengths and weaknesses from a student
perspective - The importance of feedback
- Good / bad feedback
- What should feedback do?
- How can this be achieved?
- The role of DALI
- Guidelines for users
9Data Collection
- Participation from three University Departments
- Accountancy and Financial Management, Electronic
Systems Engineering, and Biological Sciences. - Online student survey
- Student focus group
- Staff and student iLab meeting
10DALI Project Survey - Results
- Identifying my strengths and weaknesses in
assessment is important to me.
- It is important for me to be able to track my own
progress.
- The online practice tests are a useful learning
tool.
11DALI Project Survey - Results
- The tests help me learn more effectively.
- I would take more tests if available.
- I pay little attention to answering the questions
correctly.
12DALI Project Survey - Results
- I use the tests mainly for the feedback.
- The feedback is what I expected.
- The feedback on the questions could be improved.
13DALI Project Survey - Results
- The survey also invited respondents to add
additional comments relating to feedback and to
online assessment more generally. - These comments were categorised and placed in one
of the following groups - Feedback, Knowledge/Learning, Revision/Planning,
Accessibility, Technology, Convenience, Question
Format, Self-Test, Time, Motivation, Memory,
Relevance, Other.
14DALI Project Survey - Results
15DALI Project Survey Results
16Think Tank
- As a supplement to the survey, a focus group was
assembled. Participants were asked about their
experiences of online assessment. - Questions concentrated on likes/dislikes, reasons
for doing the tests, and views on the feedback
received.
17Think Tank Results
18Focus Group Findings
- The focus group corroborated the survey findings.
- Positives
- Students perceive the benefits of using online
assessment in relation to learning,
comprehension, and revision, and were pleased to
get instant feedback. - Negatives
- Feedback is vague, access is difficult, questions
are not engaging enough, and the questions are
not relevant.
19iLab Session Key Results
- Student comments
- Revision
- Personal strengths and weaknesses
- QMP is not used to its full potential
- Feedback could be enhanced to make it more
relevant to the individual - Staff comments
- Time, workload, and support
- Developing assessments and question banks is
time-consuming as is writing effective feedback - The ability to extend and update tests is very
useful
20Conclusions
- Project findings have been used to inform a
good practice staff handbook on online
formative assessment. - The project encountered difficulties with the
DALI interface and this is under investigation. - The project was instructive in highlighting the
importance students place upon effective feedback
and the potential to personalise their learning. - The University is developing an intelligent
assessment environment which has the ability to
generate personalised learning profiles.