Boletim da Escola

1
School-level Correlates of Achievement Linking
NAEP, State Assessments, and SASS NAEP State
Analysis Project
Sami Kitmitto
CCSSO National Conference on Large-Scale
Assessment June 2006
2
Overview of the Study

• Create a valuable data set for policy analysis by
  adding achievement scores to a comprehensive
  school survey
• School and Staffing Survey (SASS)
• Extensive information from a national survey of
  schools, but no achievement scores
• National Assessment of Educational Progress
  (NAEP)
• Nationally representative scores comparable
  between states
• State Assessment Database (NLSLSASD)
• Collection of all available school-level state
  assessment data
• Scores comparable within states

3
Research Questions

• What are the important school characteristics
  that correlate with achievement?
• Do the results of Don McLaughlin and Gili Drori
  (2000) compare to the results from a larger and
  more recent set of data?
• 2000 SASS vs. 1994 SASS
• 36-38 states vs. 20 states

4
Data Assembly NAEP Data

• NAEP 1998, 2000 and 2002
• Used 2000 Math Grades 4 8 and 1998 2002
  Reading scores for Grades 4 8
• Used full population estimates
• Mean and standard deviation at the school level
• Mean and standard deviation at the state level
• Replicate weights used

5
Data Assembly NLSLSASD 2000 Data

• NLSLSASD 2000
• Selected two scores for each grade/subject
• Grade 4 Math, Grade 4 Reading
• Grade 8 Math, Grade 8 Reading
• Remove between state variation
• Create standard score within each state

6
Data Assembly NAEP and NLSLSASD School-Level

• NAEP and NLSLSASD Correlation
• Using only schools in both NAEP and NLSLSASD
• Calculated correlation between NAEP and NLSLSASD
  scores at the state level for matched schools

7
Data Assembly NAEP State-Level and NLSLSASD

• Used NAEP to introduce between state differences
  and variation to standardized scores
• Rescaled to mean of 50 and standard deviation of
  10

8
Data Preparation Step 2 SASS 2000

• School Level Information
• From school, principal, teacher and district
  surveys
• Social Background
• Organizational Characteristics
• School Behavioral Climate
• Teacher Characteristics

9
Data Set Used for Analysis

• Analysis Sample
• Dropped schools with less than 50 students
• Did not include schools that were combinations of
  elementary, middles and or high schools
• Missing values list-wise deletion of
  observations
• Teacher Qualifications Dropped
• Teacher sample is not random or representative at
  the school level
• High percent of variation was within schools not
  between schools
• Results indicated that these measures were mostly
  noise

10
Data Numbers
Number of Schools With Two Valid
Scores Number of Schools in Analysis Sample
11
Analysis Methodology

• Structural Equation Modeling
• Similar to multiple regression analysis
• Allows for multiple measures of concepts
• Models measurement error
• Observed variables Measures
• Conceptual factors Latent Variables

12
Model
Path Model Relating Latent Variables
13
Model
Measurement Model
14
Replication Results
Fit Statistics
15
Replication Results (cont)
Estimated Coefficients for Achievement Equation
16
Interpretation of Coefficients

• Latent variables are scaled to one of their
  measures
• Class Size is scaled to student/teacher ratio
• Coefficients are standardized
• A one standard deviation increase in Class Size
  is correlated with a -.23 standard deviation
  difference in math achievement in elementary
  schools
• Standard deviation of student/teacher ratio in
  the sample is 4 students/teacher
• Mean is 15.5 students/teacher

17
Literature on Class Size
Reported Estimated Effects of Student/Teacher
Ratio and Class Size
18
Avenues for Future Research

• Add principal responses to school climate
  questions
• Add additional controls urbanicity, IEP,
  magnet school indicator
• Principal Leadership
• Resources
• Per pupil expenditures (district level)
• Number of computers
• Parent Involvement
• Teacher and principal reports of parent
  involvement being a problem
• School programs to involve parents

19
Conclusions

• Linking NAEP, NLSLSASD and SASS provides a
  powerful national sample of schools matched to
  achievement scores
• SASS provide multiple measures of key conceptual
  factors
• SEM provides a methodology to take advantage of
  the depth of SASS information
• Class size found to be correlated with
  achievement
• In middle schools, more important for reading
  than math
• Results on achievement are similar to McLaughlin
  and Drori 2000 with improved fit