Item Analysis: Difficulty

1
Item AnalysisDifficulty Discrimination

• Mark J. Kittleson, PhD, FAAHBSouthern Illinois
  University
• kittle_at_siu.edu

2
Introduction

• This tutorial will provide you a basis to analyze
  items in your assessment tool. You will
  determine item difficulty (how difficult were
  each item) and item discrimination (how the items
  were scored by high and low groups).

3
Assumptions

• Completed the Excel Data Statistics Tutorial
• Familiar with Excel

4
Here is a list of scores from 25 students.This
knowledge test had 20possible points.
5
You can sort the data by highlighting thenumbers
you want sorted, then pulling down the Data
menu and click sort
6
Youll get this menuit will askyou whether you
want to have thedata ascending or
descending.You can do a secondary sort if you
hadother data (i.e., age)
7
For the sake of this example, wellhighlight the
top 9 scores and the bottom 9 scores in yellow.
8
Now what Ive done is moved thescores to column
F and IYou can do that by highlightthe info you
want movedclicking copy, then paste where
youwant it to move to.
9
For this example, we are only looking atitems
1,2,4,5,10,12,13,18 of the test. Normally one
would look at all items
10
What we have here are the number ofpeople in
the high group that had thecorrect answer
(Column g) the next column shows the number
ofthe low group who scored correctly (column h)
11
To determine level of difficulty of theitem, add
the total number ofpeople in the upper group
(u)and the number of people inthe lower group
(l)
12
To determine index of discrimination of theitem,
subtract the number ofpeople in the lower group
(l) with correctscores from the number of people
inthe higher group (u)
13
Note that the item discriminationand item
difficulty have been movedto different
columnsfrom the previous example
14
Ok, lets have you do this yourself. Click here
to downloadactual excel sheet.
15
First, convert all of the letters to either 1s
or 0s1s indicate a correct answer 0s
indicate an incorrect answer
16
First, figure out the total score for each
person. You can do this byhighlighting the cell
you want the score (i.e., L4) Click the sum
symbol anddrag the entire line for James (b4
through k4). Hit the enter button and youll
get the total score for James. (Note that you
can learn how to do this via the Excel tutorials
on my web page)
17
You can do that for each person, or you can do a
shortcut. Highlight L4 that shows James score.
Right click the mouse and you will see amenu
like that above. Drag down to copy and hit the
mouse button.
18
Now, highlight the remaining column that you want
individual scores to be posted. Right click the
mouse and youll see the above menu.
Nowhighlight paste and click. The scores of
each person will then bepasted into their
respective cells.
19
Here are the scores for each person.
20
OK, now lets look at each item individuallyand
see how difficult they were. First, highlightthe
cell B23. Click the sum buttonthen replace
sum with average.Drag the number of cells
you wantto be included (in this case, B4 to B22.
Then hit the enter button
21
What this tells you is that70 (0.7) of the
people gotQuestion 1 correct. Now, you wantto
do this for all 10 questions.First, highlight
B23, rightclick, highlight copy and then click.
22
Like before, highlight all remaining cells you
wantthe item difficulties to be assessed. Right
click, highlight pastethen click. The scores
will appear automatically.
23
You can now see the item difficulty of this test.
Everybody got question 10correct. That
probably should be eliminated. Question 2 should
alsobe eliminated, since 90 got that question
correct. These are non-discriminatingquestions
and could be replaced with questions that could
help you betterdistinguish smart and dumb people
24
To sort, highlight what you want to sortpull
down data highlight sort and click
25
So here we have the scores among the ten
studentssorted in an ascending manner. For this
example, we are only going to look at the lowest
3 and the highest 3 scores to conduct an item
discrimination analysis.
26
So here are the scores of those in the high
scores and low scores.Low scores are the salmon
color, whereas the high scoresare gray. Our
first task is to determine the number ofcorrect
responses for the low and high groups for each
item.
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)