Yolanda Yugar

1
Planning for Evaluation of 21st CCLC Programs
Evaluation Handbook Resources

• Yolanda Yugar
• Leslie Kirby
• Allegheny Intermediate Unit 3
• Elluminate Moderator Susan D'Annunzio, PDE

2
Rules of the Elluminate Live! Highway

• Raise your hand to ask a question during
  interactive sessions
• Use Emoticons to provide feedback
• Use Direct Messaging to Communicate
• Click on the Microphone Button to speak (click
  again to release when finished speaking)
• Session status - leave temporarily, connection
  status
• Whiteboard - For presentation and participation

3
Contents

• Evaluation requirements for 2007-08 program year
• Content of the Grantee Profile APR (PPICS)
• Content of the PA 21st CCLC Grantee Report
• Evaluation Handbook evaluation resources

4
Evaluation Requirements 07-08

• Grantees with cohorts 2, 3, or 3A funds must
  complete the Grantee Profile, APR, PA Grantee
  Report
• Each cohort is reported separately
• For Grantee Profile APR, centers must be
  reported separately
• All reports are due October 31, 2008
• Reporting covers programming during summer 2007
  and school year 2007-2008

5
Contents of the Grantee Profile

• Center information
• Operations
• Student demographics

6
Contents of the APR

• Activities
• Report card grades (fall to spring) for students
  attending 30 days
• Teacher surveys for students attending 30 days

7
Changes to the APR for PPICS Reporting Pilot

• Individual student results reporting instead of
  grantee aggregation
• Downloadable spreadsheet
• Web-based data entry
• Each regularly attending student will receive a
  PPICS ID number
• AIU Evaluation tools and resources accommodate
  PPICS reporting details
• Learning Point Associates will be providing a
  webinar or workshop in late June to assist
  grantees in understanding the new reporting system

8
Contents of the PA Grantee Report

• Evaluation implementation
• Assessment results (i.e. 4Sight, DIBELS, GMADE,
  etc.)
• Report card grades (spring to spring)
• Parent, student, school administrator, partner,
  staff feedback
• School attendance
• School discipline

What grantees report in the Grantee Report should
be based on what they identified in their
applications.
9

• For both the APR and the PA Grantee Report,
  student results are reported only for those
  students attending 30 days (a.k.a. regular
  attendees). These results must be further broken
  down into attendance categories
• 30-59 days
• 60-89 days
• 90 days

10
Evaluation Handbook

• Tips for data collection management
• Directions for using evaluation resources
• Download the handbook and resources at
  www.aiu3.net/evaluations 21st CCLC page

11
(No Transcript)
12
Resources for Evaluation Planning

• Data safeguarding FERPA
• Data safeguarding plans
• Parent permission forms
• Program evaluation framework

13
Data Safeguarding Plans Parent Permission Forms

• Templates are available for download
• Prior to completing templates, grantees should
• Review FERPA legislation
• Review the directions for each template
• Identify a contact person who can respond to
  questions about data safeguarding
• To complete the template, enter grantee-specific
  information where the blue text appears
• It is not enough for grantees to complete the
  templates. Data safeguarding measures should be
  adopted!

14
Program Evaluation Framework

• Provides a format for grantees to organize their
  entire evaluation plan, including required
  performance measure/indicators and additional
  grantee interests
• BEFORE Using the Evaluation Framework
• Review your grant application/contract to
  identify the performance measures, performance
  indicators, and evaluation methods outlined
  there.
• What are the reporting requirements?
• What else do you want to know?
• Identify an evaluator (external or internal staff
  with specialized skills)
• Establish your evaluation team evaluator(s),
  program implementers, administrators

15
(No Transcript)
16
Resources for Data Collection, Management,
Reporting

• Student information tracking (demographics,
  needs, cohort assignment, program attendance,
  school attendance, discipline, report card
  grades)
• Activities tracking
• Web-based PPICS Teacher Survey
• PA Grantee Report
• PA 21st CCLC Survey Builder

17
Student Information Tracking

• Demographics
• Cohort assignment
• Student needs
• Program attendance
• School attendance/tardiness
• School discipline
• Report card grades

18
Student Information Tracking

• Highlights duplicated ID and last name entries
• Most data can be copied pasted into and among
  the spreadsheets
• Pay attention to column order
• Do not sort or filter until you are finished
  copying pasting
• Where drop-down cell options are provided, cells
  may not paste if the contents of the cell and the
  possible answer options do not match

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Activities Tracking

• Grantees report activities provided in PPICS,
  including
• Frequency
• Duration
• Content area(s)
• Target audience
• Activity code(s)
• Because of the great variety and differences
  among grantee activities, grantees using this
  tool will need to sort and count activities
  manually

27
(No Transcript)
28
Teacher Survey

• Grantees must survey a classroom teacher for each
  student attending 30 days. A reading or math
  teacher is recommended.
• Grantees have 3 options for fulfilling the
  Teacher Survey requirement of the APR
• Paper-based survey distributed to and collected
  from classroom teachers
• Distribute a web link to classroom teachers so
  they can complete an online version
• Combination staff collect paper-based surveys
  and enter them into the online system
• For grantees using the web-based survey, AIU will
  generate a report with a summary of the teacher
  survey results and the raw data.

29
PA 21st CCLC Survey Builder

• Provides grantees with useful questions for
  collecting feedback from stakeholder groups such
  as parents, school administrators, students,
  center staff, and partners
• Includes Likert scale and open-ended questions
• Computes and graphs forced-choice survey results

30
(No Transcript)
31
(No Transcript)
32
(No Transcript)
33
Contact

• Allegheny Intermediate Unit
• 475 East Waterfront Drive
• Homestead, PA 15120
• [phone]
• www.aiu3.net/evaluations