20042005 Forest Heights Elementary YearRound School Evaluation

1
2004-2005 Forest Heights Elementary Year-Round
School Evaluation
Office of Research and Evaluation
2
History of Forest Heights Elementary School

• In school year 2001-2002, two Richland District
  One elementary schools, Crane Creek and Denny
  Terrace, were combined into a new school, Forest
  Heights Elementary
• Forest Heights has followed a year-round
  schedule since its inception

3
Year-Round Education

• Reorganizes the school year to provide more
  continuous learning by breaking up the long
  summer vacation into shorter, more frequent
  breaks throughout the year
• Does not eliminate summer vacation, but reduces
  and redistributes it as vacation or intersession
  time during the school year

4
Year-Round Education

• Students at year-round schools receive the same
  instruction as students on a traditional
  calendar
• The year-round calendar is organized into
  instructional periods and vacation weeks that
  are more evenly balanced across 12 months than
  the traditional school calendar
• The balanced calendar is designed to minimize
  learning loss that occurs during a typical
  three-month summer vacation

5
Forest Heights Intersession Periods

• Students with Academic Assistance Plans are
  required to attend intersessions to receive
  additional instruction
• Students without Academic Assistance Plans may
  optionally attend intersessions to participate
  in various enrichment activities

6
Evaluation Plan

• To use PACT data to help determine if there are
  any significant achievement differences between
  students attending year-round versus traditional
  calendar schools
• For each PACT subject
• Define an experiment group of Forest Heights
  students meeting certain criteria
• Define a control group consisting of similar
  students at other district schools with similar
  Spring 2004 PACT results
• Determine if there are any significant
  differences in Spring 2005 PACT results between
  the groups

7
Experiment Group Selection

• Forest Heights students with following criteria
• In 4th or 5th grade for 2004-2005
• Took on-grade level 2004 and 2005 PACT subject
  tests at Forest Heights
• Progressed to next grade from 2003-2004 to
  2004-2005 school years

8
Control Group Candidates

• Students from other Richland District One
  elementary schools with following criteria
• In 4th or 5th grade for 2004-2005
• Took on-grade level 2004 and 2005 PACT subject
  tests at same school
• Progressed to next grade from 2003-2004 to
  2004-2005 school years

9
Control Group Selection

• For each experiment group student and subject,
  the group of students was selected from the
  control group candidates with the following
  criteria
• Same race (black, non-black)
• Same lunch (subsidized, full-pay)
• Same grade level each year
• Same 2004 PACT performance level
• Same 2004 PACT EOC point weight

10
Additional Group Selection Criteria

• Experiment group students for whom no control
  group match was found were excluded from the
  study
• ELA 1 student
• Math 1 student
• Science 3 students
• Social Studies 3 students
• After the control group was selected,
  performances for all control matches for a
  particular experiment student were averaged to
  create a virtual control student

11
Experiment Group Percent of 135-Day Enrollment
12
ELA
13
ELA Student Descriptions
117 Experiment Group Students
14
ELA Control GroupSchool Representation
1485 Control Group Students
15
ELA Pre-Test Comparison of Mean PACT EOC Point
Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2004 PACT EOC point
  weights

As designed
16
ELA Pre-Test Comparison of Mean PACT EOC Point
Weights
Significant?
No
No
No
P-value
1.0000
1.0000
1.0000
17
ELA Post-Test Comparison of Mean PACT EOC Point
Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT EOC point
  weights

18
ELA Post-Test Comparison of Mean PACT EOC Point
Weights
Significant?
No
No
No
P-value
0.3859
0.5432
0.1320
19
ELA Comparison of Mean PACT Improvement

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT improvement

20
ELA Comparison of Mean PACT Improvement
Significant?
No
No
No
P-value
0.2572
0.4717
0.0258
21
ELA Results Summary

• Significance tests for 2005 PACT achievement
  between experiment and control groups are
  summarized below

22
Math
23
Math Student Descriptions
121 Experiment Group Students
24
Math Control GroupSchool Representation
1667 Control Group Students
25
Math Pre-Test Comparison of Mean PACT EOC Point
Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2004 PACT EOC point
  weights

As designed
26
Math Pre-Test Comparison of Mean PACT EOC Point
Weights
Significant?
No
No
No
P-value
1.0000
1.0000
1.0000
27
Math Post-Test Comparison of Mean PACT EOC Point
Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT EOC point
  weights

28
Math Post-Test Comparison of Mean PACT EOC Point
Weights
Significant?
No
No
No
P-value
0.9633
0.9213
0.9918
29
Math Comparison of Mean PACT Improvement

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT improvement

30
Math Comparison of Mean PACT Improvement
Significant?
No
No
No
P-value
0.9491
0.9195
0.9850
31
Math Results Summary

• Significance tests for 2005 PACT achievement
  between experiment and control groups are
  summarized below

32
Science
33
Science Student Descriptions
123 Experiment Group Students
34
Science Control GroupSchool Representation
1628 Control Group Students
35
Science Pre-Test Comparison of Mean PACT EOC
Point Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2004 PACT EOC point
  weights

As designed
36
Science Pre-Test Comparison of Mean PACT EOC
Point Weights
Significant?
No
No
No
P-value
1.0000
1.0000
1.0000
37
Science Post-Test Comparison of Mean PACT EOC
Point Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT EOC point
  weights

38
Science Post-Test Comparison of Mean PACT EOC
Point Weights
Significant?
No
No
No
P-value
0.7168
0.9208
0.5555
39
Science Comparison of Mean PACT Improvement

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT improvement

40
Science Comparison of Mean PACT Improvement
Significant?
No
No
No
P-value
0.6684
0.9151
0.4341
41
Science Results Summary

• Significance tests for 2005 PACT achievement
  between experiment and control groups are
  summarized below

42
Social Studies
43
Social Studies Student Descriptions
122 Experiment Group Students
44
Social Studies Control GroupSchool
Representation
1682 Control Group Students
45
Social Studies Pre-Test Comparison of Mean PACT
EOC Point Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2004 PACT EOC point
  weights

As designed
46
Social Studies Pre-Test Comparison of Mean PACT
EOC Point Weights
Significant?
No
No
No
P-value
1.0000
1.0000
1.0000
47
Social Studies Post-Test Comparison of Mean PACT
EOC Point Weights

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT EOC point
  weights

48
Social Studies Post-Test Comparison of Mean PACT
EOC Point Weights
Significant?
Yes
Yes
No
P-value
0.0038
0.0030
0.1757
49
Social Studies Comparison of Mean PACT
Improvement

• Used analysis of variance with ?0.01 to
  compare group means of 2005 PACT improvement

50
Social Studies Comparison of Mean PACT
Improvement
Significant?
No
Yes
Yes
P-value
0.0025
0.0046
0.1100
51
Social Studies Results Summary

• Significance tests for 2005 PACT achievement
  between experiment and control groups are
  summarized below

52
Summary and Conclusions
53
Evaluation Results Summary
54
PACT Results Summary

• Only difference in achievement between
  experiment and control groups when comparing
  mean EOC point weights and improvement occurred
  for all grades combined and fourth grade Social
  Studies where experiment performance was
  significantly lower than control
• The Year Round Education Evaluation was first
  examined in 2002-2003. No differences on
  student PACT achievement between traditional and
  year round calendar schools.

55
Possible Explanations and Confounding Factors

• The studies comparing the year round to the
  traditional schedule are problematic because
  they are inconclusive. For one thing, it is
  difficult to isolate the year round calendar
  as the reason for any positive or negative
  results. - Melisa Kelly,
  Secondary School Educators

56
Possible Explanations and Confounding Factors

• Achievement differences more likely to occur at
  beginning of traditional school year immediately
  following learning loss during the summer, but
  PACT administered at end of year
• Schools following traditional calendar had
  other options for remediation causing
  traditional calendar to be atypical
• K-5 Lottery funds used to implement After
  School Tutorial Program

57
Possible Explanations and Confounding Factors

• If learning can be lost during the summer, it
  is very likely a low level of learning on
  Blooms Taxonomy. These days teachers are
  encouraged to develop students higher level
  thinking rather than rote memory, the so-called
  summer learning loss may not be a prevalent
  phenomenon.

58
Possible Explanations and Confounding Factors

• PACT was designed to measure students higher
  level thinking skills, thus, the rote memory
  type of learning differences that was assumed
  to occur during summer between two calendars
  were not easy to be detected.

59
Possible Explanations and Confounding Factors

• Student attendance at Forest Heights has been
  lower than the district average except for
  2004- 2005 since school opened. Learning or
  academic improvement is likely to occur when
  students attend school regularly.

60
Possible Explanations and Confounding Factors

• Teacher attendance at Forest Heights has been
  lower than the district average since school
  opened. Since teacher is the most essential
  factor in contributing to students learning, it
  would be hard to result in high student
  achievement with low teacher attendance.

61
Possible Explanations and Confounding Factors

• The EOC Survey results showed decline of
  positive responses from teachers.

62
Possible Explanations and Confounding Factors

• The EOC Survey results showed that the percent
  of students positive responses were lower than
  the district before 2005.

63
Results of Studies on Year Round Education

• Utah State Office of Education, Statewide
• Evaluation of Year Round and Extended-Day
• Schools, Executive Summary. 1989
• "Thus we can say that with the implementation
  of year-round education and all the changes that
  may take place simultaneously, including
  increased teacher enthusiasm, more structured
  curriculum, and increased testing and tracking of
  students, academic achievement of students is not
  hurt and may possibly be benefited.
• Virginia State Department of Education.
  Instructional Time and Student Learning A Study
  of the School Calendar and Instructional Time,
  December 1992
• States that data from year round schools
  offering optional
• summer quarters do not indicate superior
  student
• performance by students attending year round
  schools.

64
Results of Studies on Year Round Education

• Leslie Six, A Review of Recent Studies Relating
  to the
• Achievement of Students Enrolled in
  Year-Round
• Education Programs, January 1993
• Six reports that ten of thirteen studies
  favor year-round
• education when comparing test results between
  year-
• round and traditional calendar sites.
  However, the report
• is fundamentally flawed by its dubious
  methods of
• reporting.
• W. L. Winters, A Review of Recent Studies
  Relating to the Achievement of Students Enrolled
  in Year-Round Education Programs, National
  Association for Year-Round Education Programs,
  November 1994
• Only four of the nineteen studies actually
  reported differences in achievement which were
  statistically significant, and some of these
  claims appear spurious.

65
Results of Studies on Year Round Education

• Charlie Naylor, BCTF Research and Technology
• Division, May 1995
• I would conclude that the case for improved
  educational achievement caused by the
  implementation of year- round calendars is not
  proven and should be treated with some
  skepticism.
• Jennifer Fager, Scheduling Alternatives Options
  for
• Student Success, 1997 The data that
  would reinforce claims of the effectiveness of
  these schedule changes is relatively inconclusive
  at this time

66
Results of Studies on Year Round Education

• Dena Dossett Marco Munoz, Year-Round Education
  in a
• Reform Environment The Impact on Student
• Achievement and Cost-Effectiveness Analysis,
  2000 Multivariate and univariate analysis
  revealed no
  significant differences between students
  attending year- round schools and those
  attending schools with traditional calendars in
  both reading and math achievement and average
  percent attendance. Cost-efficiency analysis
  revealed that year-round schools were more
  expensive and less cost-effective than the
  regular-calendar schools in both reading and
  math.

67
Results of Studies on Year Round Education

• Janet Ferguson, The Effect of Year-Round School
  on Student Achievement in Mathematics, 1999
• In comparing 44 fifth- and sixth graders in
  year-round school with 40 on a traditional
  calendar, the traditional group improved math
  scores significantly over the summer.
• Carolyn Kneese, The Impact of Year-Round
  Education on
• Student Learning A Study of Six Elementary
  Schools,
• 2000
• After 4 years, the year-round programs
  produced
• acceptable academic growth in students,
  compared to
• controls. Gains were higher for math than
  reading and
• slowed after several years.

68
Results of Studies on Year Round Education

• Bradley McMillen, A Statewide (NC) Evaluation
  of
• Academic Achievement in Year-Round Schools,
  2001
• Results indicated that achievement in
  year-round schools
• was no higher than in traditional calendar
  schools, and
• differential effects for certain student
  subgroups,
• although statistically significant in some
  cases, were not
• of practical significance.
• Phyllis Sanders, Parent, Teacher, and Student
• Satisfaction with Year-Round School
  Intersession, 2001 Results show that all those
  interviewed were pleased with
• the enrichment activities during
  intersession, even those
• participants with negative feelings toward
  certain aspects
• of intersession. Most of the participants
  interviewed
• expressed a desire to return to the
  traditional calendar.