Test Construction

1
Test Construction

• Schroeder
• PSY/SPED 572

2
Stages of test development

• Conceptualization
• Construction
• Tryout
• Item analysis
• Revision

3
Conceptualization

• An idea

4
Conceptualization

• What will it measure?
• What is the objective?
• Is there a need?
• Who will use it?
• Etc

5
Conceptualization

• Pilot work
• How best to measure construct
• Literature reviews
• Experimentation
• Item creation

6
Construction

• Scaling
• Age-based
• Grade-based
• Stanine-based
• Dimensions
• Comparative v. categorical
• Writing items item pool/bank
• Range of items
• Item formats
• No. of items
• Computer adaptive testing (GRE example)
• Scoring
• Cumulative
• Class or category
• Ipsative

7
Tryout

• Tried on similar population to that of interest
• Standardized conditions
• 5-10 people for each item on the test, but the
  more the better
• Good items
• Determined by item analysis

8
Item Analysis

• Process of determining which items are good
• Tools in item analysis
• Item difficulty index
• Item reliability index
• Item validity index
• Item discrimination index

9
Item Difficulty Index

• Underlying assumption Every item should be
  failed or passed based on the testtakers level of
  knowledge about the material
• Proportion of the total number of testtakers who
  got the item correct
• Pn
• Can calculate the average item difficulty on a
  test
• Optimal average item difficulty is the midpoint
  between 1.00 and chance success
• For true/false .50 1.00 1.5/2 .75
• For four option multiple choice???

10
Item Reliability Index

• Internal consistency of a test
• Higher this index greater internal consistency
• Use factor analysis
• Want to maximize internal consistency so choose
  those items

11
Item Validity Index

• Indication of the degree to which a test measures
  what it is supposed to measure
• Higher item validity index higher
  criterion-related validity
• Want to maximize criterion-related validity so
  choose those items

12
Item Discrimination Index

• Indicates how well an item discriminates between
  high scores and low scorers
• Want high scorers to answer correctly and low
  scorers to answer incorrectly otherwise throw
  out item
• d
• Higher value of d, the great number of high
  scorers answering correctly
• Negative d, low scorers more likely than high to
  answer correctly
• p. 216, Table 7-2

13
Item characteristic curves

• Graph of item difficulty and discrimination
• p. 218, Figure 7-6
• Item A good or bad item? Why?
• Item B good or bad item? Why?
• Item C good or bad item? Why?
• Item D good or bad item? Why?

14
Test Revision

• When to revise?
• p. 227
• Cautions when comparing two testsnew and revised