Measuring Reading

1
Measuring Reading

• Julie Alonzo Luke Duesbery
• University of Oregon

2
Construct of Reading
National Reading Panel (2000). Teaching children
to read An evidence-based assessment of the
scientific research literature on reading and its
implications for reading instruction
3
Implementation

• Basic background of assessment
• Large K-12 school district in the Northwest
• 3 upper level curriculum based measures (CBMs) of
  reading
• IRT implications
• more information from scores
• guide instrument improvement

4
The Model
e
xx
xx
xx
xx
xx
5
Model
e
ORF
STATE TEST
VOCAB
COMP
6
ORF

• ORF measures fluency in reading - student asked
  to read aloud for one minute
• ORF passages were selected from fiction and
  non-fiction texts. Different (comparable)
  passages chosen for each grade.
• Each passage is about 250 words long, typed,
  double-spaced, on one side of a single sheet of
  paper.

7
Vocabulary

• measures word knowledge
• a word followed by four possible answer choices
  (one near, one far, one in between)
• correct answer is the 2nd most common definition
  of the word
• 25 items

8
Comprehension

• Students read a passage and answer comprehension
  questions
• 4 answer choices per item.
• Questions vary in terms of cognitive demand
  literal, inferential, evaluative skills
• Passages might be fiction or nonfiction
• 15 multiple choice items.

9
Classical Analysis
10
Classical Analysis
11
Classical Analysis
12
Data Analysis Method

• Scan bubble sheets, data into Excel
• Write SPSS script file to convert from Excel to
  SPPS and flat text file (recode variables if
  necessary)
• Create Bilog or Winsteps control file to analyze
  flat data file
• Move data back into SPSS file (?)
• Use Modfit to generate fit statistics
• Use trait values in regression

13
Step 1

• Scan bubble sheets, data into Excel

14
Step 2

• Write SPSS script file to convert from Excel to
  SPPS and flat text file (recode variables if
  necessary)

15
Step 3

• Create Bilog or Winsteps control file to analyze
  flat data file
• This generates the trait scores we want
• (phase1, phase2, phase3)

16
Step 4

• Move data back into SPSS file (?)

17
Step 5

• Use Modfit to generate fit statistics

18
Results (Form A preIRT)
e
.24
.60
.43
.69
.18
19
Results (Form B preIRT)
e
.32
.40
.20
.36
.36