Title: The Continual Assessment of Confidence or Knowledge with Hidden MCQ? Short Paper W.I.P.
1The Continual Assessment of Confidence or
Knowledge with Hidden MCQ?Short Paper W.I.P.
- Phil Davies
- School of Computing
- University of Glamorgan
2Conference 1999Learning Through Assessment
- Phil Davies
- School of Computing
- University of Glamorgan
3Implementation of OLAL
- Tests Weeks 3,6,8,12 (60 Subject Mark)
- Percentage Marks 5,15,15,25
- Beep Test
- 2 Correct -1 Incorrect(Really -3)
4(No Transcript)
5Student Concentration
- Guessing
- Why should the student care if incorrect?
- Allow TWO PASSES of the test
- By concentrating on first pass
- will get reward on second pass
- formative
6Results of The Tests
Improvement of Test Results Pass One to Two
7Who benefited from the test?
8Student Feedback
- Excellent way of gaining knowledge
- 2nd Test Allowed us to learn
- Good assistant to learning
- Encourages Revision
- Very good, promotes learning of a subject early
on - Better than sex!!!!
9Theres no confidence in multiple-choice testing
- CAA Conference 2002
- Phil Davies
10What decides the QUALITY of a question?
- In the 2001 World cup qualifier between England
and Greece, who scored the all important
equalizing goal? - David Beckham
- Victoria Beckham
- Brooklyn Beckham
- Rebecca Loos
11Rewards?
- Know it
- On the tip of my tongue
- If I see the answers Ill know it
- Deduction
- Guess
- Know it oh no Ive got it wrong ..
misinformation
12(No Transcript)
13Proportion of Questions Correct
- High Confidence gt 84 correct
- Fairly Confident gt 58 correct
- Not confident gt 47 correct
- 72 of all questions answered with high confidence
14Student thoughts on Confidence
- Eliminates guesswork
- I was waiting for a leggy blond to bring on my
prize at the end - separated the lucky students from me
- at least I didnt lose marks for selecting no
confidence - are you testing my confidence or knowledge
15Quality of a multiple choice question?
- In the 2001 World cup qualifier between England
and Greece, who scored the all important
equalizing goal? - Teddy Sheringham
- Michael Owen
- Robbie Fowler
- QUALITY OF QUESTION NOT DISTRACTERS
16Bite The Bullet
- Use Confidence Testing for all four tests (10,
15, 20 25) - Weighting of Passes (50/50, 60/40, 70/30 80/20)
- How to judge if the students had really
learned? - Identify improvement gt
- Select five questions that were answered
poorest in test one pass through to test two - Select five questions that were answered
poorest in test two pass through to test
three ( original five) . Etc.
17How to identify Poorest
- Using confidence, degree of correctness of
answering - Using confidence scores to provide a degree of
poorness - i.e. 4, 2, 1, -6, -5, -4
- High degree of poorness could just be a BAD
question - Swing between 1st and 2nd pass of degree of
correctness should be Reasonable
18(No Transcript)
19(No Transcript)
20Identifying Poor Questions Test One
21Identifying the Poorest Questions Test two to
three
22Identifying Poorest Questions from Three to Four
23Passed through to Test Four
- 13 of the poorest question
- 6 new questions based upon additional work
between tests 3-4 - 21 questions (cross section of questions and
topics from previous tests)
24Improvement of Results Through Tests
25Improvement Pass One of Questions
26Improvement for Question 5
- Stronger Students improved quickly through the
tests - Became more confident in answering the questions
- Weaker Students improved less quickly than
stronger through the test progress
27Is it just Confidence or Knowledge i.e. questions
right or wrong?
28Some Concerns
29Overall Comments
- Work in Progress
- Results look very positive
- Correctness Factor is dependant upon weighting of
scores - Guessing not so much of a problem
- Changed students mode of revising / study
- Fine granularity in the QUALITY OF A QUESTION
- Possibility of Computerized Adaptive Testing
- However note the quality of the question change
throughout the course of the cumulative tests - Does gender come into it??
30Contact Information
- Email pdavies_at_glam.ac.uk
- Phone 01443 482247
- School of Computing
- University of Glamorgan
- Pontypridd
- Mid Glamorgan
- South Wales