Texas Science Initiative State Partnership

1
Texas Science Initiative State Partnership
Research Grant

• Timothy P. Scott, Ph.D.
• Carolyn Schroeder, Ph.D.
• Homer Tolson, Ph.D.

2
Texas Education Agency Project Staff

• Shirley Neeley, Ed.D., Commissioner of Education
• Robert Scott, Chief Deputy Commissioner
• Christi Martin, Senior Advisor
• Susan Barnes, Ph.D., Associate Commissioner
• Chris Castillo Comer, Director of Science
• Gina S. Day, Texas Science Initiative Manager
• Sharon Jackson, Ph.D., Deputy Associate
  Commissioner
• Irene Pickhardt, Assistant Director of Science
• George Rislov, Managing Director of Curriculum

3
Texas AM University Project Staff

• Timothy P. Scott, Ph.D., Project Director
• Homer Tolson, Ph.D., Senior Analyst
• Carolyn Schroeder,Ph.D., Lead Researcher
• Yi-Hsuan Lee, Ph.D., Analyst
• Tse-Yang Huang, Ph.D., Analyst
• Xue Hu, Research Assistant
• Adrienne Bentz, Assistant Director, CMSE

4
Advisory Board

• Carol L. Fletcher, Ph.D., Texas Regional
  Collaboratives, UT Austin
• Ginny Heilman, Region VI ESC
• Anna McClane, Region IV ESC
• Sandra S. West, Ph.D., Texas State University
• Jo Ann Wheeler, Region IV ESC

5
Review Team

• Katherine (Kit) Price Blount, Ph.D., Texas
  Collaborative for Excellence in Teacher
  Preparation
• Patricia Castellano, North East ISD, San Antonio
• Diane Jurica, George West ISD
• Judy Kelley, Texas Rural Systemic Initiative
  South Texas Rural Systemic Initiative
• Mayra Martinez, TAMU-Corpus Christi
• Sharon Kyles Ross, Dallas ISD
• Fernando Ruiz, TAMU-Corpus Christi

6
Science Initiative Partnership/Research Grant
Timeline

• June
• Physics Teaching Resource Agents (PTRA)
• Texas Science Education Leadership Association
  (TSELA)
• Texas Urban Science Council (TUSC)
• July
• Preliminary meta-analysis results report
• Texas Regional Collaboratives
• August
• Rubric discussion meeting
• Harris County Science Leadership meeting
• September
• Rubric field test
• October
• Report to Science Initiative Task Force and other
  stakeholders
• Report to TEA

• February
• Commissioner's Science Initiative Task Force
• Teacher Quality Grants Program - Developers
• March
• Teleconference with the US Department of
  Education, Dr. Grover J. Whitehurst
• April
• Science ESC Specialists Network (SESnet)
• Texas Science Careers Consortium
• National Assessment of Educational Progress
  (NAEP)
• May
• Site visit, The University of Texas at El Paso
  Collaborative
• TEA, Gates Foundation, Texas High School
  Project, TAMU, UT
• Advisory board meeting

7

• What teaching strategies have been shown to
  improve student achievement in science???

8
Science Initiative Research Grant

• Meta-Analysis

9
Project Methodology

• Acquisition of Studies
• Coding of Studies
• Intercoder Objectivity
• Criteria for Selection of Studies
• Computation of Effect Size
• Statistical Methods and Analyses

10
Table 1. Frequencies and Reasons for Non-Inclusion
A large number of studies not collected because
they obviously did not fit within time frame.
11
Meta-Analysis

• Statistical Validation

12
Table 2. Frequencies of Characteristics of
Included Studies
13
Table 2. Frequencies of Characteristics of
Included Studies
14
Table 4. Dependent Variable (Test Type)
15
Table 8. CMA Results for Total Data and Treatment
Categories
16
Figure 4. Graphic Representation of 95
Confidence Intervals for Treatment Categories and
Total Data
17
Figure 5. Mean Effect Sizes for Treatment
Categories and Total Data
C1Questioning C2Manipulation C3Enhanced
Material C4Testing C5Inquiry C6Enhanced
Context C7Instructional Technology C8Collaborati
ve Learning
18
Table 9. Failsafe N for Total Data and Treatment
Description Categories
19
Conclusions

• Meta-Analysis

20
What teaching strategies have been shown to
improve student achievement in science???

• All of the innovative strategies have a positive
  influence on student achievement.
• Innovative science instruction is a mixture of
  teaching strategies.
• Teaching strategies are tools, and the right tool
  must be selected for the job at hand.

21
Table 12. Ranking of Teaching Strategies
22
Most Powerful Enhanced Context Strategies

• Make learning relevant to students
• Use real-world examples and problems
• Problem based learning
• Case based learning
• Use technology to bring real world into classroom
• Take students out of classroom into real world
• Use multiple contexts to teach concept

23
Rubric for Product Evaluation

• Based on Meta-Analysis

24
Rubric Design Based on Meta-Analysis

• Effective instructional practices
• Enhanced context strategies
• Inquiry strategies
• Instructional technology strategies
• Collaborative learning strategies
• Manipulation strategies
• Questioning strategies
• Equity and practicality
• Equity
• Practicality

• Science content
• Accuracy and alignment
• Safety
• Organization and structure
• Format of materials
• Coherency
• Meaningful assessment
• Alignment
• Formative
• Summative
• Metacognitive

25
Rubric Development

• Draft created using criteria
• Sent to advisory board and stakeholders for
  comment
• Revision
• Discussion with science teachers/ supervisors
• Further revisions, clarifications, weighting of
  categories
• Field test
• Statistical validation (Interrater reliability
  .945 using Cronbachs alpha)

26
Weighting

• Weighting of categories to reflect meta-analysis
  results
• Automatic since 10 of 16 total categories are
  from the meta-analysis results
• Exception Value of Enhanced Context Strategies
  doubled since ES was so large
• Weighting of classifications to enhance
  discrimination between Recommended and Not
  Recommended
• Highly recommended 4 pts.
• Recommended 3 pts.
• Not recommended 0 pts.
• Doubled for Enhanced Context Strategies

27
Rubric for Product Analysis
28
Scoring Notes

• Science Content
• Accuracy and Alignment
• Score 0 if either accuracy or alignment are weak
• Difference between a 4 and 3 is the inclusion of
  vertical articulation
• Safety
• Score 0 if unsafe
• Difference between a 4 and 3 is the clarity and
  completeness of safety precautions
• Organization and Structure
• Format of Materials
• Little or no information for implementation
  versus adequate information for implementation
  versus comprehensive information for
  implementation
• Coherency
• Score 0 if activities are unconnected
• Score 3 if somewhat-connected activities build
  toward understanding
• Score 4 if well-connected activities build
  understanding

29
Score Sheet
30
Science Initiative Research

• Next steps

31
Continuing Components of Initiative

• Publish Meta-Analysis
• Expand Study
• Develop Effective Instructional Strategies
  Booklet
• Document State of PK12 Science in Texas
• Support Efforts of T-STEM Centers and Academies
• Support Efforts of Texas Regional Collaboratives
  for Excellence in Science Teaching

32
Contact Information

• Dr. Tim Scott 979-845-7362
  tim_at_science.tamu.edu
• Adrienne Bentz 979-458-8001
  abentz_at_science.tamu.edu

• Dr. Carolyn Schroeder
  979-458-4450 cschroeder_at_science.tamu.edu
• Dr. Homer Tolson 979-458-1120
  htolson_at_tamu.edu

Texas AM University Center for Mathematics and
Science Education 3257 TAMU College Station,
Texas 77843-3257 http//www.science.tamu.edu/cms
e/