How to Use Statewide Trends When Analyzing Your Data

1
How to Use Statewide Trends When Analyzing Your
Data

• North Central Association
• Spring Conference
• Hyatt Regency Hotel
• March 6, 2006

2
The Value of Statewide Trend Data

• Statewide data can be used as a benchmark for
  local district and school results.
• Comparing local data to statewide results can
  help identify whether local results are similar
  or unique to that of other schools and districts.
  
• If you find that your local results do not mirror
  statewide trends, then it becomes important to
  find out why and what might be causing the
  difference.

3
Remember!!

• Both positive and negative results should be
  analyzed when making decisions about school
  improvement.
• If you are showing positive trends in a certain
  subject or with a certain student group, find out
  why and apply those positive instructional,
  organizational or environmental successes to
  other applicable areas that are in need of
  improvement.

4
Possible Reasons for Unique Positive Results

• Strong Teacher
• Smart Class
• Strong Curriculum
• More Time Appropriated
• Strong Focused Professional Development
• Successful Instructional Program or Intervention
• Parent Support
• Administrative Leadership
• Focus on Data

5
Focus On Data

• All stakeholders, including administrators,
  teachers, parents, community members and students
  need to be aware of the data in order to make
  informed decisions.
• An individual piece of data has little meaning
  until it is compared to a larger benchmark.
• A school that has strong local assessments will
  rarely be surprised by statewide test results.

6
Percent Passing ISTEP Grades 3-10 2004-2005

• A One Year Overview of Percent Passing Results by
  Grade and Student Group

7
2005 ISTEP English/Language Arts by Gender
8
(No Transcript)
9
(No Transcript)
10
2005 ISTEP Mathematics by Gender
11
(No Transcript)
12
(No Transcript)
13
2005 ISTEP English/Language Arts by Poverty
14
(No Transcript)
15
(No Transcript)
16
2005 ISTEP Mathematics by Poverty
17
(No Transcript)
18
(No Transcript)
19
2005 ISTEP English/Language Arts by Special Ed
20
(No Transcript)
21
(No Transcript)
22
2005 ISTEP Mathematics by Special Ed
23
(No Transcript)
24
(No Transcript)
25
2005 ISTEP English/Language Arts by Ethnicity
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
2005 ISTEP Mathematics by Ethnicity
30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
Are You Showing the Same Trends?

• Analyze the gap between groups at the state level
  and compare it to your results locally.
• This comparison becomes much more valuable as we
  add multiple years of data.
• With multiple years of data, we can look at both
  grade level and cohort improvement over time.

34
Percent Passing ISTEP Over Time by Ethnicity

• A Look at the Indiana
• Percent Passing Rate
• between 1999-2005

35
Grade 3 English/Language Arts
36
Grade 3 Mathematics
37
Grade 6 English/Language Arts
38
Grade 6 Mathematics
39
Grade 8 English/Language Arts
40
Grade 8 Mathematics
41
Grade 10 English/Language Arts
42
Grade 10 Mathematics
43
Scale Score Trends

• Scale score comparisons are valuable because they
  show improvement from all students.
• Percent passing only displays improvement of the
  number of students scoring above the minimum cut
  score and does not show increases in achievement
  for all students.

44
ISTEP Scale Scores Grades 3, 6 8 2002-2005

• A Three Year Comparison of Scale Score Results by
  Grade and Student Group Using Pass and Pass as
  Benchmarks.

45
ISTEP English/Language Arts Scale Score
Comparison 2002-2005 - Grade 3
46
ISTEP English/Language Arts Scale Score
Comparison 2002-2005 - Grade 6
47
ISTEP English/Language Arts Scale Score
Comparison 2002-2005 - Grade 8
48
ISTEP Mathematics Scale Score Comparison
2002-2005 - Grade 3
49
ISTEP Mathematics Scale Score Comparison
2002-2005 - Grade 6
50
ISTEP Mathematics Scale Score Comparison
2002-2005 - Grade 8
51
Other Trend Data Opportunities

• Attendance by student group, month, day of the
  week
• Participation in school activities/events
• Teaching staff by age, experience, subject area
• Student demographics by ethnicity, poverty,
  disciplinary referrals, family structure, parent
  education, family income, GPA

52
Discussion of Indiana Accountability
53
P.L.221 Accountability

• State Board of Education places schools in
  categories of school improvement and performance
  based on
• Percentage of all students who pass English and
  math tests (averaged across subjects and grade
  levels).
• Improvement in passing percentage of non-mobile
  cohort group of students (enrolled for 70 of
  school year).

54
(No Transcript)
55
Key Terms

• Percentage of All Students
• Students were enrolled on last day of school
  year.
• Students have ISTEP results for following test.
• Who Pass English and Math Tests (averaged across
  subjects and grade levels)
• English tests passed plus math tests passed
• DIVIDED BY
• English tests taken plus math tests taken

56
Key Terms

• Improvement in passing percentage of non-mobile
  cohort group of students
• Students were enrolled for 126 days.
• Students have ISTEP results for test at the
  beginning of the school year.
• Students have ISTEP results for following test.

57
Key Terms

• Improvement in passing percentage of non-mobile
  cohort group of students
• Compute passing percentage for each year
  (averaged across subject and grade level).
• Determine improvement from one year to next.
• Determine average improvement for period
  (initially two years for elementary and middle
  schools and one year for high schools ultimately
  three years for all schools).
• Determine higher of average improvement for
  period or latest year to year improvement.

58
Improvement Fall 2003 to Fall 2004

• Grade 3
• Grade 6
• Grade 8

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9
• Grade 10

59
Improvement Fall 2003 to Fall 2004

• Grade 3
• Grade 6
• Grade 8

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9
• Grade 10

K-5 Elementary
6-8 Middle School
60
Improvement Fall 2004 to Fall 2005

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9
• Grade 10

61
Improvement Fall 2004 to Fall 2005

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9

• Grade 3
• Grade 4
• Grade 5
• Grade 6
• Grade 7
• Grade 8
• Grade 9
• Grade 10

K-5 Elementary
6-8 Middle School
High School
62
Example Category Placement

• Students in school have 62 passing percentage on
  2005 ISTEP test.
• Non-mobile students passing percentage increased
  by 2 from 2003 to 2004.
• Non-mobile students passing percentage increased
  by 3 from 2004-2005.
• Average improvement for two-year period is 2.5.
• Latest year to year improvement of 3 is higher
  than two-year average of 2.5.

63
(No Transcript)
64
NCLB Accountability

• States, school districts, and schools must
  demonstrate adequate yearly progress (AYP).
• All students are expected to be at the
  state-defined proficient level by 2013-2014
  (Fall 2014 ISTEP).
• States calculated starting point based on
  2001-2002 test data (Fall 2002 ISTEP).

65
NCLB Accountability

• Indianas AYP starting points were
• 58.8 passing in English.
• 57.1 passing in mathematics.
• AYP targets for 2005 test increased to
• English 65.7.
• Math 64.3.

66
Increasing Goals Under NCLB
67
Determining AYP

• All students and student groups (duplicated
  count) must meet annual AYP goal in English and
  math (calculated separately), including the
  following groups
• Customary racial/ethnic subgroups (White, Black
  not of Hispanic Origin, Hispanic, Asian, American
  Indian)
• Students with disabilities
• Limited English proficient students
• Economically disadvantaged students

68
Determining AYP

• Additional indicator is graduation rate for high
  schools and attendance rate for other schools,
  with initial goal of 95. Any improvement is
  sufficient.
• Must test 95 of all students and each group.

69
Determining AYP - Reliability

• Minimum Ns are used
• 10 for reporting
• 30 for accountability
• 40 for 95 participation requirement
• AYP determinations will be based on the higher of
  the most current performance or a three-year
  average.
• Only students enrolled for 162 days, Indianas
  definition of full academic year, are included
  in AYP determinations.

70
Determining AYP - Reliability

• If a student group does not meet the goal but the
  percentage of non-proficient students is
  reduced by 10 from the previous year and the
  group meets the goal on the other indicator, the
  group has made AYP. This is known as safe
  harbor.
• A test of statistical significance is applied to
  AYP decisions. A school is considered as not
  making AYP only if there is 99 confidence (75
  for safe harbor) that the school did not meet AYP
  requirements.

71
Determining AYP - Reliability

• Students with severe cognitive disabilities, up
  to 1 of all students tested, may be counted as
  proficient based on alternate standards.
• For 2004, if a school or school corporation
  failed to demonstrate AYP solely based on the
  students with disabilities group, the passing
  percentage of students with disabilities was
  adjusted upward by 14 percent. This was a
  temporary proxy based on the percentage of
  students who receive special education services.

72
Determining AYP - Reliability

• For accountability purposes, a limited English
  proficient student remains a member of the LEP
  student group until the student achieves a
  proficient score on the English proficiency test
  for two consecutive years.
• LEP students will be tested using alternate form
  of assessment during their first three years in
  the U.S.
• ISTAR Rubric aligned with standards.
• Results linked to state assessment score.

73
(No Transcript)
74
(No Transcript)
75
Contact Information

• Gary Wallyn
• Director of Accreditation, Assistance, and Awards
• Indiana Department of Education
• 317-232-9060
• gwallyn_at_doe.state.in.us