Using Data to Improve Instruction: Building on Models that Work

1
Using Data to Improve InstructionBuilding on
Models that Work
Education Leaders Conference

• Council of Chief State School Officers
• St. Louis
• September 12-14, 2007

2
Tools for Aligning Instruction, Standards,
Assessments
Surveys of Enacted Curriculum

• Orientation for Educators

3
Conference Objectives

• Present discuss Strategies/designs/tools for
  using data to improve instruction that have been
  applied with schools
• Analyze findings/evidence about effectiveness of
  strategies/designs/tools, and discuss potential
  as model for others
• Provide leadership training on how to implement
  and use data-driven approaches for improving
  instruction

4
Panel of State Leaders Policy to Practice
Question

• How can States effectively integrate Education
  Improvement Initiatives with Use of Data?
• PANEL
• Gene Wilhoit, CCSSO Executive Director,
  (Facilitator)
• Gerald Zahorchak, Pennsylvania Secretary of
  Education
• Susan Castillo, Oregon Superintendent of
  Education
• Todd Flaherty, Rhode Island Department of
  Education
• Sally Wherry, Minnesota Department of Education

5
New members Session topics

• What is SEC? Why? How? -- RB
• What Collaborative does Tools --RB
• How to use SEC data charts Carolyn
  Karatzas
• How to plan an SEC project Diana Nunnaley
• SEC online survey system/alignment John
  Smithson

6
(No Transcript)
7
New Member Orientation

• What? are the Survey of Enacted Curriculum
  tools?
• How? are data collected, analyzed, reported?
• Why? are SEC data useful to educators, leaders,
  researchers?
• Now what?how do we implement locally and lead
  the use of SEC in schools?

8
STATE of

• MissourA
• MissourEE

9

Surveys of Enacted Curriculum

• SEC State Collaborative
• Members Planning Meeting
• St. Louis
• September 11, 2007

10
Rationale
Assessment
Standards
Curriculum
11
Applications

• Alignment analysis --instruction, standards,
  assessments
• Instructional improvement in schools
• Needs assessment/ Evaluation
• Indicators monitoring change over time

12
Multi-State Collaborative

• Build Knowledge of leaders
• Share Strategies
• Produce joint products, tools

13
SEC Collaborative Members 2007-08

• Delaware
• Idaho
• Illinois
• Indiana
• Iowa
• Kansas
• Maine
• Michigan

• Mississippi
• Montana
• Ohio
• Oklahoma
• Oregon
• Wisconsin
• Vermont
• Minnesota

14
Social Studies SEC Development 2005-2007

• Arizona
• Delaware
• Idaho
• Maine
• Ohio
• Oregon
• Vermont
• Wisconsin
• Minnesota
• Nevada

U of Wis-WCER NCSS Civics
Education National Geographic GENIP NCEE CCSSO

15
SEC Collaborating Organizations

• Council of Chief State School Officers
  www.SECsurvey.org
• Wisconsin Center for Education Research
  www.SEConline.org
• Learning Point Associates/NCREL
  www.SECsupport.org
• TERC Regional Alliance DEC Project
  www.ra.terc.edu/DEC

16
SEC Collaborative offers

• Core Components of SEC tools/services
• Surveys with teachers report instruction
• Alignment analysis content code standards and
  assessments
• How to use Data knowledge, skills
• 4) Leader development 2-3 meetings/yr
• In-state workshops a) orientation
• b) use of data

17
SEC Collaborative

• States, Districts
• CCSSO SEC Collaborative Management
• Rolf Blank, Carlise Smith
• WCER Surveys, Reporting, Alignment analysis
• John Smithson, Alissa Minor, Eric
  Osthoff
• Training Technical Assistance
• Carolyn Karatzas, Consultant, CCSSO
• Diana Nunnaley, TERC
• Gary Money, Maren Harris, Learning Point
  Associates
• Lani Seikaly, Hillstreet Main
• Jennifer Unger, The Groupworks
• Michael Erhlinghaus, Measured
  Progress

18
SEC by the Numbers 06-07

• 10,393 SEC Surveys 06-07
• 4674 Math
• 1993 Science
• 3646 ELAR
• 80 Soc Stud
• 131 Standards/Assess. Content analysis
• 1000 est. schools--Data use

19
SEC by the Numbers (2)

• By Comparison
• 1996 207 Science Teachers (5
  states)
• 1999 626 M/S Teachers (11 states)
• 2005 5,414 E, M, S teachers
• 2006 10,200 E, M, S teachers
• 2007 10,400 E, M, S, SSt (27 states)

20
SEC Collaborative

• How SEC Collaborative works for members
• Training and Assistance to project leaders
• Project Planning w/ budget meet local objectives
  
• Each state/district makes decisions on services
  project plan
• Multi-state benefits of collaborative
• e.g., learning from experience, sharing ideas,
  training

21
Accomplishments06-07 SEC Collaborative

• In-state Leader workshops, Project Planning
• PPTs for Orientation, Survey Admin, Data use
• Leader Development Standards document
• Website revisions, improvements (SECsurvey.org
  SEConline.org)
• E.g., State Projects, All Alignment results,
  Report generator

22
More Accomplishments SEC Collaborative

• Using Data to Improve Instruction Boulder, CO
  conference 200 leaders, 32 sessions (30 states)
  
• Social Studies Survey development
• 10 states, 5 prof. orgs.,
• NAEP Alignment analysis 2007 Math items
• Research/evaluation
• MSP study of professional development multi-site
  study
• Ohio case study of district/school
  implementation of SEC
• Follow-up data tool to track school level data
• Validity studylongitudinal data for teachers
  and students
• Ohio reading study
• John demonstrated SEC comparisons in Feb 07

23
Goals SEC 06-07

• State/local projectsFrom pilots to broad
  application, local leadership training
• Integrate SEC tools/data with Improvement
  Initiatives, especially data-driven
• Research/Evaluation --Broader use of Surveys,
  Alignment
• Assist States in planning with Service delivery
  agencies SEC team with CACs Labs
• Social Studies SEC complete implement

24
Key Question -- SEC ToolsResearch into Practice

• How can Educators obtain reliable, valid data to
  determine Alignment of instruction with required
  standards and assessments?

25
Survey Sections

• Instructional Readiness
• Teacher Opinions
• Professional Development
• Types, Frequency
• Content , Active,
• Collegial, Coherence
• Instructional Content
• Topic x Cog. Demand

• School Class Description
• Instructional Activities
• General
• Problem Solving Activities
• Pairs Small Group Work
• Use of Hands-on Materials
• Use of Calculators/Computers other Ed. Tech.
• Assessment Use
• Instructional Influences

26
Steps in SEC Development

• RD studies curriculum-- Early 90s
• ModelsTIMSS, NAEP, Analyze assessment
• OTL interest of States, Porter/ Smithson
  research,
• CCSSO Science Assessment project
• State collaborative Develop surveys-98-01
  (NSF)
• 11-State field study, Reports, Alignment, USI
• Use of data experiment DEC (NSF/ROLE01-04)
• Evaluation MSP RETA PD Study (02 05, NSF)
• English/ Language Arts survey (03 04)
• Current projects 15 States, 4 Districts, 5 MSPs

27
Questions Addressed by SEC Data

• How can in-depth data on content of instruction
  be collected and reportednot topic checklists,
  to analyze teaching content in relation to
  standards, assessments, achievement?
• How can methods of teaching practices be compared
  across classrooms, schools, districts, and
  states?
• How can enacted curriculum data be reported in a
  manner to encourage use by teachers to improve
  instruction?

28
MSP PD Study Sites

• Brockport/Rochester SUNY
• Cleveland Municipal SD
• Corpus Christi AIMS - TX AM
• El Paso UTEP
• State MSPs ID, IL, OR, OK, ME, MT
• NSF MSPs Boston
• NJ MSP -- 12 districts
• NC MSP -- 17 districts
• SW PA MSP
• SCALE Los Angeles

29
Key Education Questions (contd)

• How can we measure the effects of standards-based
  initiatives on instructional practices and
  curriculum in classrooms?
• How can we analyze effectiveness of professional
  development on changes in teachers instructional
  practices? (i.e. determine the quality of
  professional development)

30
What if

• for Intro of SEC to Educators
• You could use data on instructional quality and
  content to guide professional development?
• You could have consistency across grade levels?
• You could know how well aligned your state
  standards were to the state assessment?
• You could compare how you teach content compared
  to how others across the nation teach?
• You could use anonymous teacher data to start a
  powerful school discussion about what the teacher
  needs are?

31
(No Transcript)
32
Content Matrix
33
Science content matrix
34
Content Maps
35
Surveys of Enacted Curriculum
The intended curriculum State content
standardsWhat students should learn
The assessed curriculum State (and other)
assessmentstested learning
The learned curriculum Student outcomes based
on school learning
The enacted curriculum What teachers teach
36
The Enacted Curriculum
37
(No Transcript)
38
(No Transcript)
39
SEC Websites

• CCSSO, SEC Collaborative
• www.SECsurvey.org
• WCER, SEC Online Survey and Reports
• www.SEConline.org