PVAAS: Essential Connections for Statewide Implementation

1
PVAAS Essential Connections for Statewide
Implementation!
Pennsylvania Value-Added Assessment System

• National Value-Added Conference
• Columbus, Ohio
• October 14-16, 2007
• Kristen Lewald
• PVAAS Statewide Project Director

2
Session Objectives

• Participants will
• Identify key reports from EVAAS and the
  application to strategic educational initiatives
• Identify 2 initiatives from their
  state/region/district that would be an
  opportunity to integrate value-added reporting

3
Session Plan

• History and Context for PVAAS
• Conceptual Connections
• PA Data Tool Connections
• Process Connections
• Strategic Planning
• School Improvement Planning Framework
• Grade Level Team Meetings
• Student Level Planning
• Programs/Initiative Connections
• Educational Assistance Program/Tutoring
• Response to Intervention
• Districts with Distinguished Educators
• Supplemental Education Services
• Your Questions

4
History of PVAAS

• Requested by Districts
• PA League of Urban Schools recommended to State
  Board of Ed
• EVAAS Methodology from SAS, Inc. PVAAS
• 2002-2005 Pilot
• 3 Phases
• 100 Districts, Including SD of Philadelphia and
  Pittsburgh
• 2006 Reporting to all 501 Districts
• Grades 4 6 only
• 2007 Reporting to all 501 Districts
• Grades 4-8

5
Leverage Points for SUPPORT

• Conceptual
• Processes
• Products
• Protocols
• People
• Professional Development

6
Implementation Support

• PVAAS Statewide Workgroup
• Lead by Gerald L. Zahorchak, D. Ed.
• Secretary of Education
• Commonwealth of Pennsylvania
• Over 50 Members
• Representation Examples
• Districts
• Statewide Associations
• Higher Ed
• Business/Industry
• Legislative Representation
• Support and Input
• Communications
• Professional Development

7
Essential Connections

• CONCEPTUAL CONNNECTIONS

8
The Power of Two!
Achievement gt AYP Did NOT Meet Growth Standard
Achievement gt AYP Met/Exceeded Growth Standard
Proficient
Achievement lt AYP Met/Exceeded Growth Standard
Achievement lt AYP Did NOT Meet Growth Standard
Gain
9
Regional Scatter Plots

• Provided to each IU region in PPT
• All schools in region
• Grades 4-8
• Reading and Mathematics
• IUs Sharing with Local Districts
• Some IUs making district/school scatter plots
• PVAAS site not public
• Using scatter plots as vehicle to encourage
  regional connections based on high growth

10
Example Regional Scatter Plot
11
Example Regional Scatter Plot Same Schools
12
Essential Connections

• PA DATA TOOL
• CONNECTIONS

13
PDE Provides The Data Tools
14
Putting it All Together
Is the percent of non-proficient acceptable?
Has our school met all of its AYP targets?
Were the final 4Sight Performance Levels
acceptable?
Have student gains exceeded the growth standard
consistently?
15
Two PVAAS Methodologies
Looking Forward/Planning PVAAS Projection
Reports For Individual Students and Cohorts of
Students
Looking Back/Evaluation Value-added Growth
Reports For Cohorts of Students
Today
16
Essential Connections

• STRATEGIC PLANNING

17
Strategic Planning in PA

• Guide work of districts in all student meeting
  Standards as defined in PA regulations
• Six Year Plan
• 501 Districts
• 3 Phases I, II, III
• Plan Developed with Broad Community Involvement
• New Web-Based Tool eStrat Plan
• PVAAS Projections!

18
Projection Summary Tool

• An EXCEL template to summarize the projections of
  a entire cohort of students
• Template allows you to specify the level of
  proficiency and the criteria for the likelihood
  ranges

Available on PDE website NOW!
19
8th to 11th Projection to Proficiency
20
PVAAS Projection Inquiry
Strategic Planning What percent of students are
likely to be proficient on a future PSSA
examination?
21
Incorporating PVAAS into the eStrategic Planning
Tool

• Goal of the Data Tab in the eSP tool is to
  produce a district profile, using existing data
  from PDE and data uploaded locally from
  additional sources.
• Using the Data Tab in eSP, districts can add
  PVAAS projection information to their gallery
  walk of data, which can be used to create
  reflections for the district profile.
• The resulting tables and pie charts of the
  Projection Summary tool can be copied and pasted
  into a Word, Excel, or PowerPoint file that can
  be uploaded into the Data Tab.

22
Uploading PVAAS File into Data Tab
23
Uploading PVAAS File into Data Tab
24
District Data Profile
Custom data file will now be available along
with the preloaded data. Click on the boxes to
select 1 to 4 data sets to review and create
reflections.
25
Reflections
At the end of the data, there is a section to
enter the reflection and to identify it as a
strength or challenge.
26
Essential Connections

• DATA INFORMED DECISION-MAKING

27
School Structures for Data-Informed Decision
Making
District-Level Support (Budgetary Support,
Professional Development, Resources and Time)
Student Learning Data
Demographic l Perceptual l Process Data
Annual Building-Wide Planning Process Focus All
Students Who School-Wide Team How PDE Getting
Results, Data Retreat, School/Continuous Planning
Process

• Building Level
• School Demographic Data
• PennData
• Discipline Data
• Attendance Data
• Mobility Rate Data
• Parent Surveys

• Building Level
• PSSA PVAAS
• Final 4Sight Benchmark Test
• Standardized Assessments
• District End-of-Year Tests
• EAP/Tutoring Assessments

• Grade/Course Level
• Initial PSSA/PVAAS/Final Tests
• Class/Subgroup Levels
• Cyclical
• 4Sight Benchmark Data Grade Level
• District Quarterly Assessments
• Common Classroom Data
• Classroom Summaries
• EAP/Tutoring Assessments

Periodic Grade-Level Planning Process Focus
Groups of Students Who Teacher Teams How
Regular 1-2 Hour meetings

• Grade/Course Level
• Class Demographic Data
• Class Engagement Data
• Satisfaction Data
• Attendance Data
• Walk Through Data

Student-Planning Process

• Classroom Level
• Initial PSSA/PVAAS/Final Tests
• Student-Level Achievement and Growth Data
• Cyclical
• 4Sight Benchmark Data Student Level
• Continuous
• Individual Classroom Assessments
• EAP/Tutoring Assessments
• Progress Monitoring

• Classroom Level
• Qualitative Data
• Student Historical Information
• Student Medical Information
• Student Learning Information

28
Essential Connections

• SCHOOL
• IMPROVEMENT PLANNING

29
School Improvement

• What if the great things we are doing or plan
  to doare not the great things we need?
• Colonel Lee DeRemer
• Army War College

30
School Improvement in PA

• School Improvement Process/Plan is called
  Getting Results
• Process aligns clear standards to curriculum,
  instruction, assessments, interventions, and
  resources. 
• Data informs school improvement planning
• PVAAS Reporting
• School reports
• Performance Diagnostic
• Performance Diagnostic Subgroups
• Projections to Proficiency

31
(No Transcript)
32
Figure 1 The Three Phases of a Continuous
Improvement Planning Framework
33
Worksheet 1 IDENTIFY STUDENT DATA FROM MULTIPLE
SOURCES


34
Worksheet 2-B ORGANIZE STANDARD STUDENT
DATASETS IN EASY-TO-UNDERSTAND FORMATS
35
(No Transcript)
36
(No Transcript)
37
Essential Connections

• GRADE LEVEL
• TEAM PLANNING

38
Grade Level Teaming in PA

• School Leadership
• Data and leadership are a reciprocal
  relationship.
• (Schmoker, 2006).
• The principal must serve as the instructional
  leader and lead the effort to use data-based
  decision making
• (Dufour, 2002)
• Teams of Teachers
• Share a common vision and set of goals
• Collect data on student performance
• Analyze these data
• Meet regularly to interpret these data for the
  purpose of informing instructional decisions
  focused on increasing student achievement

39
Putting it All Together
How did our students perform at the end of the
year in each reporting category?
Did the various performance groups of our
students all show positive gains?
40
Putting it All Together
Did the students in the selected subgroup achieve
acceptable gains at the end of the year?
Did the students in the selected subgroup achieve
acceptable gains? How does this pattern compare
to the pattern of all students in the grade?
41
Essential Connections

• STUDENT LEVEL
• PLANNING

42
Student Level Planning in PA

• Individual Students
• Groups of Students
• 0-40 likelihood of proficiency
• 40-70 likelihood of proficiency
• 70-100 likelihood of proficiency
• Subgroups of Students
• Ex. Special Education

43
PVAAS Projection Inquiry
Intervention Planning What percent of students
are likely to need extra support in preparation
for a future PSSA examination?
44

• PENNSYLVANIAS EDUCATIONAL ASSISTANCE PROGRAM
  (EAP)/TUTORING

45
EAP/Tutoring in PA

• 66 Million
• 175/501 School Districts
• Various Approved Assessments
• Evolution of Reporting
• Attendance
• Participation
• Subgroups
• Progress
• NOW Research Questions

46
EAP Research Questions

• Does tutoring measurably impact student
  achievement in the Commonwealth?
• Was the progress in tutored students measurably
  different than for non-tutored students for the
  tutoring-eligible group by districts?
• For the tutored students within each grade and
  subject is there variability in effectiveness
  among districts?

47
Example Research Findings Gr 4-8

• Does tutoring measurably impact student
  achievement in Reading in the Commonwealth?
• Eligible students who were tutored lt22 hours in
  Reading tended to score no better than eligible
  students who did not participate in tutoring.
• Eligible students who were tutored 22-44 hours in
  Reading tended to score 5.8 points better than
  eligible students who did not participate in
  tutoring
• Eligible students who were tutored 45 or greater
  hours in Reading tended to score 12.7 points
  better than eligible students who did not
  participate.
• For eligible students, attending tutoring for at
  least 22 hours was beneficial to student
  performance. The increase in student performance
  was larger for those tutored 45 hours than for
  those tutored 22-44 hours. Providing tutoring in
  Reading for less than 22 hours was not found to
  have an impact on student performance.

48
Tutoring District Report
The Tutoring District Report compares the
predicted performance of the districts tutored
students to each students observed performance.
The District Effect is the mean of the
differences of the observed score predicted
score.
Line of Inquiry How did the tutored students in
each grade (Reading or Math) perform relative to
their predicted performances? Did we at least
meet the predicted performance?
49
Tutoring Diagnostic Report
All tutored students in the testing pool are
placed into a Low group or a High group based
on their achievement performance in the state.
For each group, the mean of the observed
predicted is calculated and displayed in the
Diagnostic Report.
Line of Inquiry Is there a difference in the
performance of our high performing tutored
students from our lower performing tutored
students relative to their predicted performance
score? If so, what might be causing these
differences?
50
Tutoring District Report
This report details the predicted scaled score
and the observed scale score of every student in
the grade or in the high or low subgroup.
Line of Inquiry For students that met or
exceeded their predicted score, why might that
have occurred? What may have been the reason
that some students did not meet the predicted
score?
51
Essential Connections

• RESPONSE TO INTERVENTION

52
Response to Intervention in PA

• Response to Intervention (RtI) is a
  comprehensive, multi-tiered intervention strategy
  to enable early identification and intervention
  for students at risk of academic failure or
  behavioral difficulties.
• Unlike the discrepancy model used to identify
  students with learning disabilities, RtI allows
  educators to identify and address academic and
  behavioral difficulties prior to student failure.
  
• Student response to a series of increasingly
  intense interventions assists in preventing
  failure and guides eligibility decisions for
  special education services.

53
Response to Intervention in PA

• Core Characteristics of RtI
• All students receive high quality, scientifically
  research-based instruction in the general
  education core program.
• All staff (general, remedial, and special
  education) assume an active role in student
  assessment and in instruction in the core
  program.
• Student progress is continually monitored
  (universal screening, progress monitoring) to
  identify those learners who are not meeting
  benchmark or other standards.
• Students receive increasing intense levels of
  targeted scientifically, research-based
  interventions dependent upon student need.  
• Data drives instructional decisions and student
  movement through the tiers.  
• Student movement through the tiers is flexible
  and based on student response to targeted
  intervention.

54
RtI and PVAAS

• Student level projections
• of Likelihood of a student being basic,
  proficient and/or advanced on a future PSSA (
  state assessment)
• Use PVAAS Projection as indicator with RtI
  strategy
• Use in conjunction with other assessment data
• Baseline Benchmark assessment data

55
Essential Connections

• DISTRICTS WITH DISTINGUISHED EDUCATORS

56
Distinguished Educator Program

• Providing district supports and targeted
  intervention is a crucial component in the
  Pennsylvania Accountability System. 
• The Distinguished Educator (DE) initiative is one
  strategy to provide direct assistance.
• Distinguished Educators will work with struggling
  districts and schools as part of a team to build
  capacity and to provide assistance aimed at
  improving student achievement.  DE's can be
  current or retired administrators, teachers,
  specialists and consultants with a wide range of
  experience and expertise, and are selected
  following a multiple-step application process.
• Distinguished Educators serve as full-time
  members of a core team focused on instructional
  leadership and providing specific assistance
  based on targeted needs.
• Annual professional development on PVAAS to DEs.

57
Essential Connections

• SUPPLEMENTAL EDUCATION SERVICES (SES)

58
SES in PA New for SY07-08

• Supplemental educational services (SES) are
  defined as tutoring or other supplemental
  academic enrichment services in reading/language
  arts and mathematics that are provided beyond the
  normal school day and are of high quality,
  research based, and specifically designed to
  increase academic achievement of students in
  schools in need of improvement.
• A Title I school that is in its second year of
  school improvement a school that has not met
  adequate yearly progress (AYP) for three years
  or is in corrective action or identified for
  restructuring must offer SES to eligible
  students.
• Measure Progress of Students by Provider

59
Essnetial Connectins for PA

• Conceptual Connections
• PA Data Tool Connections
• Process Connections
• Strategic Planning
• School Improvement Planning Framework
• Grade Level Team Meetings
• Student Level Planning
• Programs/Initiative Connections
• Educational Assistance Program/Tutoring
• Response to Intervention
• Districts with Distinguished Educators
• Supplemental Education Services

60
Discussion and Sharing

• Identify 2 initiatives from your
  state/region/district that would be an
  opportunity to integrate value-added reporting

61
Questions/Follow-Up

• Please feel free to
• email us with any questions.
• pdepvaas_at_iu13.org