The Constructive Comments, Assisted Assessment, Faster Feedback and Enhanced Experiences CAFEx2 proj

1
The Constructive Comments, Assisted Assessment,
Faster Feedback and Enhanced Experiences (CAFEx2)
project
2
Acknowledgements

• HEA-ICS
• Ge Qin
• Diane Bush
• Neil Newbold
• Surrey e-Learning team / CEAD

3
Motivation

• National Student Survey (NSS) provides one
  motive
• Some measure of how well HE institutions satisfy
  the expectations of their student cohort.
• Some may use it as a proxy for value-for-money.
• Largely intended for consideration by final year
  students who, it is hoped, can comment
  objectively and accurately on a wide range of
  experiences throughout their years at the
  University.
• Prospective students use in decision-making
  indirectly in league tables.
• In 2008, 4 key areas for improvement, 3 of which
  were in Assessment and Feedback.
• Potential efficiency gains time an increasingly
  finite resource
• THES noted cryptic feedback..
• Individual final year projects marked across the
  Department

4
Talk Outline

• About Feedback
• Related Research
• Building feedback software
• Demonstration
• Conclusions and Future work

5
About Feedback

• NSS and Literature
• Feedback on my work has been prompt.
• I have received detailed comments on my work.
• Feedback on my work has helped me clarify things
  I did not understand.

6
About Feedback

• Surveys
• In two surveys of students (2008, 2009), key
  findings were largely in keeping with the key
  points from the literature
• knowing when feedback should be returned, and
  when it actually is, is generally important
• students prefer form-based written feedback on
  work, and are increasingly demanding greater
  detail
• students are increasingly comparing marks with
  peers, and standardization issues may exist in
  variations in comments for same marks, and vice
  versa
• a majority would prefer standard initial comments
  with detailed individual feedback.
• Additionally, these students prefer written
  comments.

7
About Feedback

• Related Software
• (Presentation about, and demonstration of) Aston
  e-CAF
• http//ecaf.aston.ac.uk/
• Software not available?
• Electronic Feedback 9
• http//www.ljmu.ac.uk/cis/software/feedback.asp
• Excel
• Turnitin GradeMark
• http//turnitin.com/static/grademark.html
• Support for comment reuse?
• HEA-ICS comment bank
• Easily searched?
• Constructive?

8
Building Feedback Software

• Develop a software demonstrator for speeding the
  production good quality feedback.
• To improve speed of feedback
• To ensure a minimal feedback standard
• To ensure positively formulated feedback
• To enable production of standardized feedback
• To enable flexibility in the marking scheme

9
Relevant Work

• Can students read feedback?

10
Relevant Work

• Can students read the feedback?

11
Relevant Work
12
Relevant Work

• Is feedback constructive?
• Sentiment analysis has been variously used for
  determining strengths of opinions on a variety of
  subjects, including those of customers towards
  companies and for reviews of products

13
Relevant Work

• Can we search quickly - reusability?
• Visual Search to refine for access to commonly
  used elements
• Single-selection issue to overcome

14
Building Feedback Software

• Software features
• Server-based
• Readability Analysis
• Grammar Checking
• Sentiment Analysis
• Report Generation
• Visual Search - but updating of comment bank and
  multiple-selection efforts under review

15
Demonstration
16
Demonstration
17
Demonstration
18
Demonstration
19
Demonstration
20
Demonstration
21
Demonstration
22
Conclusions and Future Work

• Ambitious..
• Some way yet from being fully configurable or
  supported tool further development needed,
  e.g. for effective visual search integration, for
  supporting multiple units of assessment, variant
  forms of marking criteria, etc.
• A good demonstration of how to account for
  certain considerations of feedback in an
  integrated software application.
• Well received to date.
• Project web presence at http//www.computing.surr
  ey.ac.uk/personal/st/L.Gillam/Feedback/
• Further publication in preparation.

23
Questions?Comments? Criticisms?(Remember
be constructive..!)

• http//www.computing.surrey.ac.uk/personal/st/L.Gi
  llam/Feedback/