Using diagnostic assessment to enhance teaching and learning

1
Using diagnostic assessment to enhance teaching
and learning

• A study of the impact of research-informed
  teaching materials on science teachers practices

Robin Millar Vicky Hames
Paper presented at ESERA Conference,
Noordwijkerhout, Netherlands, 19-23 August 2003,
as part of the symposium Improving Science
Teaching Through Research
2
Research Network Towards Evidence-based Practice
in Science Education Professor Robin Millar
(York) (Co-ordinator) Professor John Leach
(Leeds) Professor Jonathan Osborne (Kings
College London) Dr Mary Ratcliffe (Southampton)
3
Background

• Much research on learners ideas about the
  natural world, in many countries.
• Many pupils have ideas which differ from the
  accepted ones.
• These are resistant to change.
• This research has had little systematic impact on
  science teaching.

4
Increasing the influence of research on practice
Often suggested

• better communication of research findings to
  teachers (and other users)

Instead

• provide teachers with research tools, so they can
  collect better data on their own practice

5
What we did

• worked with a group of teachers
• developed banks of diagnostic questions for three
  science topics, for pupils aged 9-16
• used questions developed by researchers as a
  starting point where possible
• wrote new ones where necessary
• tested and improved questions, and reworked them
  into forms that could be more readily used in
  teaching.

6
Diagnostic questions

• a few examples .

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
(No Transcript)
11
Outline of the study

• gave each teacher a bank of diagnostic questions
  on one science topic,
• to use as they wished in their teaching
• monitored how they used them, and the effects of
  this.

12
Sample and data

• 10 schools (8 secondary, 2 primary)
• 23 teachers
• 16 interviewed in depth (some twice)
• Questionnaires completed by all
• Samples of pupils work.

13

• Ways of using
• Teachers views of the questions
• Impact on teaching and learning

14
First impressions
T1 when I first saw these questions I thought,
oh good, questions. Because were so short of
good questions. We needed a new test so I
looked in the pack and just pulled a selection
out.
15
Second thoughts
T8 I flicked through them and thought, oh yeah,
they look like tests, you know, tons of it. Then
when you look at it, you think, oh, no, no. And
they were so interesting to use. The use for me
is opening up the discussion, thinking about how
theyre actually perceiving things, that was the
interesting bit.
16
Ways of using summary

• end-of-topic tests
• revision for tests and examinations
• pre-tests (at start of topic)
• for starting and ending lessons
• whole-class discussion
• small-group discussion
• staff development

17
Starting and ending lessons
T4 For starts and ends, thats been my principal
use. at the start of a lesson, basically
testing the ideas taught the previous lesson. .
And sometimes at the end of a lesson. ------------
------------- T6 . I imagined them on OHP for
open class discussion. Come the end, in the
last 10 minutes of the lesson, weve done this,
that and the other, lets just check and get
people to discuss.
18
Small-group discussion
T3 I photocopied the questions in little
booklets and got them to discuss in pairs and
then in a four .. ------------------------- T1
I put them in groups of two I gave them only
two questions at a time and maybe initially
about five minutes on each. And then I asked
around the room for their answers . and when
wed been round the room discussing these, .then
they turned on to the next page.
19

• Teachers views of the questions

20
Detecting misconceptions
T4 When I first saw these, I thought, theyre
absolutely superb for attacking the
misconceptions and misunderstandings, and very
quickly finding out what theyre really
thinking. ------------------------- T14
without the questions, I might never have been
aware of how widespread particular misconceptions
were.
21
Discriminating
T1 Seeing these, immediately I thought, well
those are . unambiguous, which is important.
when youre trying to work out what kids can and
cant do, I think. ------------------------- T
1 these are very sensitive, they discriminate
really well between kids who really do grasp and
dont grasp these ideas.
22
Into the open
T3 However we teach a topic, pupils always have
misconceptions. But if theyre more open about
them and we can talk about them, then youre
likely to challenge the misconceptions and move
them forward. So, I think thats what it did,
brought them out into the open a bit more.
23
Stimulating discussion
T4 so much of what is generated from this is
discussion with the pupils, which is what these
have prompted a great deal, which wouldnt have
been there without them. ... Its prompted more
discussions than I would normally have had
which is good.
24
Stimulating discussion (2)
T10 thats a very useful feature of them, the
fact that they give alternatives, so the kids
arent thinking in a vacuum. They have a
starting point. ----------------- T13 It made a
lot more openings for discussion The children
had lots of ideas in front of them And then
they can bring in their own.
25
Discussion - about science ideas
T10 question 5, about the motion of a
football, proved to be a real problem for them.
the question as to whether or not there was a
forward force provoked a heated debate. What I
got - from one EPSE question - was an entire
unplanned lesson with pupils fully engaged and
making real progress with their thinking.
26
Across the ability range
T10 With the upper sets I expected them to want
to talk about these things .. But the bottom,
that Set 5, were talking about it just as well
listening to each other, the actually talked
about things, I thought, very well.
27

• Impact on teaching and learning

28
Fitting in with your own teaching
T3 It fitted in with my own teaching. Thats
why I chose those questions, because I thought I
can use those in a similar way to that I would
teach anyway. ------------------------- T6
you have things that fit with you, dont you, as
a person. There were some things I looked at and
thought, yes, I do that already. And some things
I looked at and I thought, oh yes, I would like
to do that now. Its still me, but its
something I havent done before. Ill have a
go at that.
29
Changing teaching
T9 Oh it has influenced my teaching, without
question, in a beneficial way. I mean
if I was the sort of teacher that was always
prompting discussion then it probably wouldnt
have been a necessity, I wouldnt have needed
that. But I did need that and its helped,
without question its helped. Im having more
discussions in class than previously, which is a
good thing.
30
Teaching across the sciences
T15 I've taken an approach with this that has
been much more the approach that I would take
with chemistry much more open, you know, rather
than me just giving information and working
through things, a much more sort of interactive,
discursive approach, which is a style of teaching
I prefer. I think it's a better way of going
about things, but perhaps I haven't been as
confident in physics before to risk it.
31
.continued
I Is it a matter of confidence, or having
materials that suggest a way you could do
it? T15 I think thats right. I come up with
ideas myself for biology and chemistry, but I
find it much more difficult to come up with them
for the physics area.
32
Teaching style
T13 Im not very experienced and, especially
with forces and things, when I bring it together
at the end or when Im trying to bring it in, the
fact when you have the force opposing motion,
what would happen to the speed and so on - I tend
to find that I more or less give it to them in a
way, because trying to get them to discuss it, to
bring it in, I find it quite hard. Whereas it was
done for us, for them to discuss. And I was able
to do the diagrams on the board and they came up
and did the forces and arrows and things. We came
to it together, which is a lot better. I thought
like Id actually done proper teaching then.
33
Pupil learning
I Do you think their answers were better than
you would have expected in the past? T5 Much
better. the marks were very good. Apart from
an odd one or two everybody was able to
understand much, much easier, and was able to
answer those questions.
34
Assessment
I You didnt use any of the questions for an
end of topic test, summatively? PT2 No, I
didnt do that because, in a sense, by using them
way I did, Id found out what I needed to know.
35
Impact on practice

• clearer focus on important ideas
• better data on prevalence of misconceptions,
  informing actions
• more discussion and interaction
• pupils engaged with the key ideas
• beginning to integrate assessment with teaching.

36
Research into practice

• So did it work?
• What do we learn about the interface between
  research and practice?

37
Practice became more research evidence-informed
in three ways

• lessons influenced by previous research findings
  (stronger focus on key ideas and known learning
  challenges)
• choices based on better evidence of learning
  (collected in the course of teaching)
• assessment better integrated with teaching, with
  feedback to pupils on understanding of key ideas.

38
Enabling research to influence practice

• Research findings have to be translated, or
  even reworked, if they are to have an impact on
  practice
• New materials and approaches are judged first on
  their practicality and fit to the context of
  application they must solve a problem or
  satisfy a need
• Working with teachers, and building in
  flexibility of use, helps uptake..

39

• For more information about the EPSE Research
  Network, including
• - copies of published articles and conference
  papers
• sample teaching materials produced by the EPSE
  projects
• visit the EPSE web site at
• www.york.ac.uk/depts/educ/projs/EPSE