Developing a Valid Test Item Bank

1
Developing a Valid Test Item Bank

• by
• Dr. Bob N. Cage
• College of Education and Human Development

2
THINK TEST ITEMS,NOT TESTS
3
USE OF ITEM ANALYSIS
4

• Item Analysis
• Process of collecting, summarizing, and using
  information from students responses to make
  decisions about each item.

5
COMPUTING ITEM DIFFICULTY INDEX
6

• Formula
• number of students choosing the
  correct answer
• total number of students taking the
  test
• number of students choosing the correct
  answer
• for the upper the middle the lower
  groups
• total number of students taking the test

p

7
Item Data Summary
n34 students in class

• Upper Group Lower Group Middle Group
• 0 1 0
• b. 10 7 12
• c. 0 1 0
• d. 0 1 0
• b is the correct answer
• two students omitted this item

8

• 10712 29
• 34 34
• Item Difficulty Index .85
• 0.00 lt p lt 1.00 (range of p)

p


.85
9
Groups

• When n34 students
• Upper 10 students
• Middle 14 students
• Lower 10 students
• Use approximately 1/3 of total number of
  students in class (when nlt 40) to determine size
  of upper and lower groups when nlt 20, there is no
  middle group.

10
Interpretation of Item Difficulty Index

• When p is near zero, item is too difficult when
  p is near one, item is too easy.
• Want p to be between .40 and .60 for best results
  when test represents several different abilities.
  

11
Computing Item Discrimination Index

• Formula
• fraction of the fraction of the
• upper group __ lower group
• answering the answering the
• item correctly item correctly

D
12
Using Previous Item Data Summary

• 10 _ 7 3
• 10 10 10
• Range ( -1.00 lt D lt 1.00)
• You want items that have an Item Discrimination
• Index of .50 or above when test represents
• several different abilities.

D

.3
13
DEVELOPING TEST ITEMS
14
Lets Take a Test

• 1. The Augustine National Party has
• its headquarters in
• a. Camden, New Jersey.
• b. St. Augustine, Florida.
• c. Palo Alto, California.
• d. Dallas, Texas.
• 2. Hermann Klavermann is best known
• for
• a. developing all musical scales used in the
  western world.
• b. composing every sonata during the Romantic
  Era.
• c. translating all Russian classics into
  English.
• d. inventing the safety pin.
• 3. The Davis Act of the 20th century
• a. provided more money for schools
• b. struck down an earlier law.
• c. prohibited the manufacture, sale,
  transportation, or use of several specific drugs
  that were being used for illegal purposes.
• d. gave a raise to government employees.

• 4. Harold Stones book The Last Friendship, is an
  
• example of an
• a. political satire.
• b. autobiography.
• c. science fiction.
• d. biography.
• 5. The population of Franktown is more than
• a. 50 thousand.
• b. 60 thousand.
• c. 70 thousand.
• d. 80 thousand.

15
DISTRACTORS
16

• The purpose of distractors or foils in a
  multiple-choice item is to appear plausible to
  those students lacking sufficient knowledge to
  choose the correct answer.

17

• Every distractor should have at least one lower
  group student choosing it, and more lower group
  students than upper group students should choose
  it.

18
Item Data Summary (B)

• Item Upper Group Lower Group
• A 10 7
• B 0 0
• C 0 2
• D 0 1
• Distractor B has no responses in lower group.
• Needs to be revised or deleted.

19

• Summary of Advantages and Disadvantages to
• Using Multiple-Choice Items, pp 173-174
• Nitko, A.J. Educational Assessment of Students,
• 4th ed., Pearson, Upper Saddle River, NJ

20
Nursing Multiple-Choice Items

• Which of the following statements are accurate
  concerning REM sleep patterns?
• a. The first sleep cycle contains long REM
  periods and short periods of deep sleep.
• b. REM sleep periods increase in length while
  deep sleep decreases, as the sleep cycle
  progresses.
• c. Brain activity during REM sleep is not
  similar to that recorded during wakefulness.
• d. REM sleep occupies 10-20 of total adult
  sleep.
• 18.52 got item correct. a. 3 b. 10 c. 6
  d. 35
• 2. The patient in 432 has been NPO for 8 hours
  and received 4 units regular insulin for a blood
  glucose of 250. He calls you to the room. He is
  sweating and shaking. What is your first
  intervention
• a. notify the physician immediately
• b. ask the patient what happened prior to his
  signs/symptoms
• c. obtain a blood glucose
• d. administer 6 oz of juice

21

• 3. In caring for a child preoperatively with
  Wilms tumor, the nurse has little time to
  prepare the child and family for all lab and
  operative procedures. There are several
  preoperative concerns, the most important of
  which is
• a. explaining the major benefits and side
  effects of radiotherapy and chemotherapy since
  they are begun immediately after surgery.
• b. encouraging the parents to discuss their
  feelings concerning the shock of the diagnosis.
• c. reinforcing caution that the tumor is not
  palpated by posting a sign the bed that reads,
  Do Not Palpate Abdomen.
• e. teaching the parents how to care for a child
  with one kidney.
• 4. A nurse recognizes that a breast with peau
  dorange is generally associated with which of
  the following
• a. malignancy or mammary ducts
• b. blocked mammillary glands
• c. benign breast disease
• d. breast cancer

22

• 5. A client delivers an 8 pound 7 ounce baby girl
  this morning at 600 AM. Which of the following
  assessments has the highest priority in the
  immediate postpartal period?
• a. Bonding with the baby
• b. Rest for the mother
• c. Assessment of the fundus
• d. Monitoring intake and output

23
Item Summary Data

• Upper Group Lower Group Middle Group
• 0 2 0
• B. 11 3 17
• C. 1 7 4
• D. __0_ 0 0__
• 12 12 21
• n45
• 31 11 3 8
• 35 12 12 12

_
p

.69
D

.67

24
Item Summary Data

• Upper Group Lower Group Middle Group
• 6 4 7
• B. 0 1 1
• C. 6 6 12
• D. __0_ 1 1__
• 12 12 21
• n45
• 24 6 6
• 45 12 12

_
p
D
25
Summary

• Arrange test papers from highest number correct
  to least number correct.
• Choose top 30 of papers as high group and lowest
  30 of papers as low group.
• Choose a test item of concern and record in the
  Data Summary Table the number of papers in the
  high group and in the low group that have the
  item correct.
• Choose each distractor for this item and record
  in the Data Summary Table how many in the top
  group and how many in the low group chose this
  particular distractor.

26
Summary

• Apply the Item Difficulty Index (p) formula
  apply the Item Discrimination Index (D) formula.
• Determine whether test item meets criteria for
  being a valid item.
• If not, rewrite or replace distractors or
  rewrite stem or rewrite correct response.
• Use revised item in next examination.