Fomative Feedback on OBTL Courses

1
Fomative Feedback on OBTL Courses May
2007 John Biggs Catherine Tang


2
Intended Outcomes of the Workshop 1. Identif
y areas in your courses on which you would like
to have feedback. 2. Identify the evidence
required. 3. Design strategies for obtaining
evidence. 4. Develop action plan for formative
feedback and evaluation.

3
After implementing OBTL, we need to
know 1. How effectively is OBTL being
implemented? 2. Are there any problems? If so
how can they be fixed? 3. How can we
improve future OBTL courses?

4
Share in your group 1. Any problems that you
have encountered in implementing OBTL
courses. 2. Areas in your courses on which you
would like to have feedback.
5
Implementing OBTL through Reflective Practice as
Action Research 1. Obtain feedback on and
evidence of progress. 2. Reflect on what seems
to be working and what seems not to be
working. 3. Plan what to do to resolve the
problem(s). 4. Implement solution/changes. 5.
Obtain feedback/evidence on effect of
changes. 6. Reflect on the feedback/evidence
and repeat step 3 as appropriate.
6
Formative Feedback and Evaluation at Course
Level Sources of feedback/evidence . Hard
evidence . Students . Teachers
themselves . Peers Formative Feedback and
Evaluation at Departmental Level
7
Hard Evidence . Grade distributions Quantitati
ve distribution nature of the grades
changed as compared with previous years and
pre- OBTL and post-OBTL. . Library of
student performances on assessment tasks
for different grades (worst D, best A and some
Cs)
8
Evidence from Students' Perspective From
appropriate questionnaires and focus group
interviews. . Students' perceptions of
teaching/learning conditions . Students'
approaches to learning . Student reflections
9
Students' Perceptions of Teaching/Learning
Conditions . Student Experience Questionnaire
(First year) . Student Exit Questionnaire
(Final year) . TFQ evaluates the teachER, but
we are here interested in evaluating
teachING. . LASSI
10
Repeated measures design Entry scores Components
11
Repeated measures design Final scores Components
12
Students' Perceptions of Teaching/Learning
Conditions In OBTL, we would particularly
like to have feedback on 1. Were the ILOs
clear? 2. Did the students feel that the TLAs
helped them achieve the ILOs? Which did
not? 3. Did they think that the ATs addressed
the ILOs? 4. Did the students understand the
grading criteria? Standard Student Feedback
Questionnaires are unlikely to be useful in this
respect. Specifically designed questionnaires
may be required.
13
Students' Approaches to Learning Are the
Roberts becoming more like Susan after the
implementation of OBTL? The two-factor
version of the Student Process Questionnaire
(SPQ) (Biggs, Kember and Leung 2001) provides
information on students' reactions to teaching
in terms of their approaches to learning. The
class means are the issue, not individual
scores.

14
Student Reflections Focus group interviews
provide in-depth qualitative evidence
on . Does stating the ILOs help students plan
their learning? . Do the students go about
their learning differently in
OBTL compared to non-OBTL courese? . Insights
into the way the TLAs helped students achieve
the ILOs. . Did the students think the ATs
were fair (were they aligned to the
ILOs)? . Any elaboration that you would like to
have on the questionnaire. Selected students
can be invited to keep a learning diary.

15
Discuss in your group 1. What issues you
want to include in a specially designed
questionnaires for students? Design 5 sample
questions. 2. What issues you want students to
address in a focus group interview? Design
3 example questions. Write your questions on
the transparencies to be shared with the whole
group.

16
Evidence from The Teacher Keep a Teaching
Portfolio . Your understanding of what you are
doing your theory of teaching, and how does
that relate to OBTL. . Difficulties you have
had in implementing OBTL, with respect to ILOs,
TLAs, ATs, grading or any other relevant
issues. . Insights into teaching and learning
you have gained. . Evidence of successful
teaching incidents with OBTL. . Comparisons
with the old way. . Suggestion for improving
OBTL in your teaching area. . Feedback and
evidence from students can be included in the
portfolio.
17
Spend a few minutes to think what you would
include in your teaching portfolio in relation
to implementing OBTL. Share your ideas with the
group.
18
Evidence from Peers The Role of 'Critical
Friend' Invite your colleague to review and
provide formative feedback on your teaching,
assessment and/or teaching materials for
profressional development. Stages of peer
review 1. Pre-review meeting 2. Review of
real-time teaching session or teaching
materials 3. Post-review meeting 4. Post-re
view reflection Keep review report in your
teaching portfolio for reference.
19
Identify some areas of your teaching,
assessment or grading that you would like your
colleague to provide you with feedback on with
respect to implementing OBTL.
20
Having obtained the feedback and evidence from
your students, yourself and your peers, you
need to 1. reflect and evaluate this
information in light of OBTL 2. identify
what seems to be working and what seems not to
be working. 3. develop an action plan to
resolve the problem. 4. implement
solution/changes. 5. obtain feedback/evidence
on effect of changes.
21
Refer to the problems/areas in your courses that
you would like to have feedback on (identified
at the beginning of the workshop). 1. Identify
the feedback and evidence required to address
these problems/areas. 2. Develop an action plan
to obtain and evalaute the feedback and
evidence. Give examples of three actions that
you will take. Share your ideas with your
group.
22
Formative Feedback and Evaluation at
Departmental Level Each department should set
up processes to obtain and evaluate formative
feedback on implementation of OBTL. 1. Impact
on teaching. 2. Impact on student
learning. 3. What operational structures has the
department with respect to implementing and
monitoring OBTL? 4. Concerns regarding
continuing implementation of OBTL. 5. An
action plan for future improvement of OBTL.
23
Intended Outcomes of the Workshop 1. Identif
y areas in your courses on which you would like
to have feedback. 2. Identify the evidence
required. 3. Design strategies for obtaining
evidence. 4. Develop action plan for formative
feedback and evaluation. How well have you
achieved these outcomes?

24
Biggs, J. Tang, C. (2007) (3rd edition).
Teaching for quality learning at university.
Maindenhead McGraw Hill/Open University
Press. Chapter 12 has further details on
formative feedback and evaluation on OBTL
courses.