Curriculum Renewal: Fidelity of Implementation

1
Curriculum Renewal Fidelity of Implementation

• WERA/OSPI State Assessment ConferenceSeattle
  Airport Hilton December 4, 2008
• Peter Hendrickson, Ph.D.
• Everett Public Schools

2
Program Evaluation

• Fidelity of Implementation
• Program Model
• Theory of Action or Logic Model
• To what extent, consistent?
• Program Impacts
• Knowledge
• Skills
• Dispositions

3
Theory of Action

• What are you doing?
• Who are you doing it with?
• What resources do you have?
• What effects do you expect?
• (Green, 2008)

4
Logic Model fragment
5
FOI Critical Components

• Structural/ procedural
• Structural/ educative
• Instructional/ pedagogical
• Student engagement

• What to do
• Need to know
• Instructional strategies
• Expectations for students
• (Century et.al., 2007)

6
Degree implemented

• On Model, Off Model
• Various Models, e.g.
• Read 180 90 minute
• Local 55 minute HS
• Local 45 minute MS
• Local 45 minute, plus 30 SSR

7
Direct Observation--Tools

• Observation protocol
• Train observers
• Timely feedback
• Additional information
• Use the data

8
FOI Framework, p.1
9
FOI Framework, p. 2
10
Zoomerang survey

• Objectives
• Design
• Format, pilot
• Administer, remind
• Organize data
• Analyze
• Report

11
Focus groups

• Representative
• Possible time
• Field test
• Conduct
• Transcribe, validate
• Content analysis

12
Interviews

• Go to their turf
• Probe for key elements
• Keep short
• Listen for unexpected
• Dont argue
• Act on findings

13
Software monitoring

• Who enrolled
• Time on software
• Books read
• Vocabulary growth
• Comprehension growth
• Coasting?

14
Student Information System

• Present
• Tardy
• Gender
• Ethnicity
• Prior learning
• Special programs

15
References

• Bamberger, M., J. Rugh, L. Mabry. (2006). Real
  World Evaluation. Thousand Oaks, CA Sage
  Publications. Three veteran program evaluators
  reveal tricks of the trade with limited
  resources, time or commitment.
• Borman, G., Slavin, R.E., Cheung, A.,
  Chamberlain, A., Madden, N.A., Chambers, B.
  (2007). Final reading outcomes of the national
  randomized field trial of Success for All.
  American Educational Research Journal, 44 (3),
  701-731. When implementing Success for All,
  considerable emphasis is placed on fidelity.
  External evaluators, extensive training for
  building principals, building-wide commitment and
  exhaustive instructional rubrics set the program
  apart from most other literacy models.
• CEMSE. (2007). Fidelity of Implementation
  Questionnaires for Mathematics Curriculum in
  Grades 1-5. , Chicago, IL University of
  Chicago.
• CEMSE. (2008). Critical Components Definitions
  and Explanations. Retrieved November 21, 2008
  from the Worldwide Web at http//cemse.uchicago.ed
  u/files/CCDefinitions summary 2008_11_110.pdf
  This fully fleshed math and science FOI checklist
  provides a useful base document for creating
  local checklists.

16
References

• Century, J., Freeman, C., Rudnick, M., Leslie,
  D. (2007). A conceptual framework for fidelity of
  implementation of instructional materials. Paper
  presented at the Annual Meeting of the American
  Educational Research Association, New York, NY.
  March 28th 2008.
• Fullan, M. (2001). Leading in a Culture of
  Change. San Francisco, CA Jossey-Bass.
• Hendrickson, P. (2008). Unpublished principal
  interview protocols for several program
  evaluations. Everett, WA Everett Public
  Schools.
• --. (2006). Read 180 Placement, Assessment and
  Reporting Guide. NYC, NY Scholastic. The PARG is
  a companion tool to the Research Protocol and
  Tools (2007).
• Kerman, Sam. (1979). Teacher Expectations and
  Student Achievement. Phi Delta Kappan, v.60,
  n.10, p. 716-18, June 1979.
• National Sciences Resource Center. (1977).
  Science for All Children. Washington, D.C.
  National Academy Press.
• Zoomerang is available free via www.zoomerang.com
  on the Worldwide Web to implement surveys for
  limited use. A full license is less than 500
  per year. Survey Monkey works, too.

17
Contact Information

• Peter Hendrickson, Ph.D.
• Curriculum Specialist
• Assessment, Research, Program Evaluation
• Everett Public Schools
• Tel [phone]
• E phendrickson_at_everettsd.org