Using Data from Standardsbased Science Assessment to Inform Instruction and Improve Student Achievem

1
Using Data from Standards-based Science
Assessment to Inform Instruction and Improve
Student Achievement Council of Chief State
School Officers July 30 - August 2, 2006 Boulder,
CO Kathy Comfort, PI/PD PASS at WestEd Deborah
Tucker, Science Assessment Consultant Partnership
for the Assessment of Standards-based Science
WestEd
2

• The agenda in a nutshell,
• Magnetic Quotes
• Brief overview of PASS
• Focus on an NSF ROLE study where we investigated
  relationships among assessment, instruction, and
  student achievement
• Provide an opportunity to try out a hands-on
  performance task used in the study and discuss
  the implications of results from this task for
  instruction
• Share our model for treatment teacher training
• Share the results of the study

3

• Research Question
• Does increased teacher understanding and use of
  data, coupled with targeted instructional
  interventions, contribute to improve student
  learning and achievement in science?
• Methods

Data Sources - Teacher Pre- and Post- Surveys
- Student Pre- and Post- Tests (PASS - NSF
supported) Teachers and students in both groups
were surveyed and pre-tested prior to treatment
teacher training.
4

• PASS Pre- Post- Science Assessments
• EMC (Enhanced Multiple Choice) Questions - 1
  form
• OMC (Ordered Multiple Choice) Questions - 1 form
• Constructed Response Investigation (CRI) - 1
  form
• Performance Task
• Hands-on Task (HT) with directions for organizing
  data
• Experimental Design Problem (EDP) no directions
  for organizing data

5

• Hands-on Performance Task
• Rainy Morning - Grade 8/9
• Task is secure everything will be collected.
• Work with a partner(s) to complete the task.
• OK to write in test booklet.
• Think about and discuss what kids need to know,
  do and understand in order to successfully
  complete the task.

6
Debrief Knowledge, Abilities, Skills, Standards
Assessed in the Task
7
Analysis of Student Work

• Provides a context for discussion of standards,
  curriculum, instruction, improved practice and
  improved student learning and achievement.
• A good starting point for thinking about the
  development of classroom assessments.

8

• Analysis of Student Work from Rainy Morning
• Overview of Student Work
• Review rubric for match to task.
• Read through 4 papers.
• Discuss your findings with your partner(s)
• How did the students do?
• Can you see any trends?

9
Performance Task Template

• Guide for development of performance tasks
• Comparability year-to-year
• Bridge large scale and classroom assessments and
  instruction

10
PASS Performance Task Template

• Begin with the National Standards
• Scenario Used to describe the problem and engage
  students in the task.
• 3. Problem Statement Provides students with
  a (concept based) testable question based to
  investigate.

Part I Part II Part III
Part IV Predictions and Set up and Conduct
Data Organization Use
of Evidence Explanations the
Investigation Analyze Apply
Limitations
11
Frequency Distribution Data

• Used in conjunction with student work to
• - analyze how well students are achieving
  scientific concepts and abilities to do inquiry.
• Findings can be used to
• create targeted instructional interventions
• develop similar classroom assessments
• duplicate this process to analyze findings from
  classroom assessments

12
(No Transcript)
13
(No Transcript)
14

• The Classroom Where the rubber meets the road or
  the intersection of assessment, curriculum and
  instruction
• Based on what you saw in student work and in the
  data
• What are the implications for instruction?
• What could you do to help students do better?
• How could you integrate PASS results with your
  instruction?
• How could you use PASS results to support
  standards-based curricula?

15

• Model for Treatment Teacher Training
• Collaborated with WestEds K-12 Alliance on
  design, development, and implementation
• Standardized training in 5 regions of Central and
  Southern California
• Treatment teachers implemented targeted
  interventions brought student work to training
  sessions maintained journals and participated
  in on-site discussions with fellow participants

16
Treatment Model for Teacher Training
17

• Targeted Intervention Strategy
• Teacher Guided Practice teachers model their own
  thinking constantly checking for student
  understanding and guiding collaborative student
  work.
• Targeted content organize data on x and y axes.
• Present small pieces of material at a time
• Guide student practice by modeling the procedure
  and discussing each step in the procedure out
  loud.
• I am going to graph data for independent
  (manipulated) variable on the horizontal or x
  axis, and data for dependent (responding)
  variable on vertical or y axis. This way I can
  tell which variable has purposely been changed,
  and which responds as a result.
• After modeling the procedure, the teacher
  invites a student(s) to model for the class,
  verbalizing their thinking, as teacher coaches.
• Students work in small groups--practice and
  explain material to each other teacher observing
  and listening.

18
(No Transcript)
19

• Student Results
• Reported in effect sizes (ES).
• ES is the number of standard deviation units the
  mean in the treatment group classrooms differed
  from the mean in the control group classrooms.
• The mean for the EDP in grade 5 treatment
  classrooms was almost three-fourths of a standard
  deviation unit higher than in the control
  classrooms.
• All ES for grade 5 were statistically
  significantly (.66-.88).
• Similar trends were found at grade 8, but only
  the EDP had effect sizes that were statistically
  significant (.13-.60).

20
R-squares and Effect Sizes by Task Type,
Score Type, and Grade Level
21

• Results-
• Support the hypothesis that the treatment had a
  positive effect on student learning and
  achievement.
• Demonstrate that the PASS measures can produce
  reliable scores that are sensitive to
  instructional interventions.