NCLB and Program Evaluation

1
NCLB and Program Evaluation

• The goal, worth, merit, and value of public
  schools shall be determined by student
  achievement and school performance measures.
• A series of performance targets that states,
  school districts, and schools must achieve each
  year to meet the proficiency requirements.
• To meet AYP, schools must be making adequate
  yearly progress towards the 2013/2014 NCLB goal.

2
NC AYP Proficiency Targets
Year Reading 3 - 8 Math 3 - 8 Reading/LA Grade 10 Math Grade 10
2005-2006 76.7 81.0 35.4 70.8
2006-2007 76.7 81.0 35.4 70.8
2007-2008 84.4 87.3 56.9 80.5
2008-2009 84.4 87.3 56.9 80.5
2009-2010 84.4 87.3 56.9 80.5
2010-2011 92.2 93.7 78.4 90.2
3
Target Groups and Subgroups

• AYP focuses on all students and sub-groups of
  students in schools, school districts, and
  states, with a goal of closing achievement gaps
  and increasing proficiency to 100 percent.
• Ten student groups defined in NC public schools.

4
Ten Student Subgroups in NC

• 1. School as a whole (all students)
• 2. American Indian
• 3. Asian
• 4. Black
• 5. Hispanic
• 6. Multi-racial
• 7. White
• 8. Economically Disadvantaged (FARMS)
• 9. Limited English Proficient (LEP)
• 10. Students with Disabilities (SWD)

5
Program Evaluation NCLB

• All public school program evaluation (until
  further notice) must include
• PROFICIENCY achievement
• 2) school safety and
• 3) teacher quality goals and objectives.
• Regardless of the type of program evaluation
  conducted, NCLB requires program evaluation to
  include these standards and expectations.

6
AYP and Proficiency

• The ultimate goal of NCLB is to bring all
  students to PROFICIENCY (Level III or Above) as
  defined by North Carolina, no later than 2013-14.
  
• For the purpose of school, district and state
  accountability, the interim benchmark for
  progressing toward the goal is Adequate Yearly
  Progress (AYP) in raising student achievement.

7
NC DPI Report Card Web Site

• http//www.ncpublicschools.org/
• Provides principals with the following
• Overall School Profile
• High Student Performance Data
• Safe and Orderly School Data
• Quality Teachers

8
NC Reports Cards

• School report cards provide information and data
  which principals can use to evaluate programs
  and school activities.
• Principals must be able to read, interpret
  understand, and communicate school report card
  data to their stakeholders.

9
NC Report Card School Profile

• School size compared to district state
• Average course size (number of students)
• Specialized enrollments (AP, IB, Career)

10
NC Report Card High Student Performance

• Overall student performance
• Demographic student performance
• Additional student performance data (SAT, AYP,
  graduation rates, attendance rates, promotion
  rates)

11
NC Report Card Overall School Performance
Ratings

• AYP Results
• Met AYP or Did Not Meet AYP
• Targets met versus targets missed

12
NC Report CardSafe, Orderly, Caring School
Data

• Acts of crime and violence
• Number of OSS and Expulsions
• Student Attendance Rates
• Access to Books Technology
• School Heath Information

13
NC Report CardSafe, Orderly, Caring School
Data

• Parental Involvement Questions
• How does the school involve parents in
  decision-making?
• Are school personnel readily accessible to
  parents?
• Does the school make every effort to communicate
  with parents?
• Is the community invited to be involved with the
  school?
• How well does parental involvement at their
  child's school reflect the diversity of the
  parents and guardians whose children attend that
  school?

14
NC Report Card High Quality Teachers

• Number of Classroom Teachers
• Fully Licensed Teachers
• Number of Classes Taught by "Highly Qualified"
  Teachers
• Number of Teachers with Advanced Degrees
• National Board Certified Teachers
• Years of Teaching Experience
• Teacher Turnover Rate
• Teacher Working Conditions

15
NC School Report Card Snapshot

• Quick and dirty summary of school data sets
• Overall School Profile
• High Student Performance Data
• Safe and Orderly School Data
• Quality Teachers

16
Needs Assessments

• You will be expected to become very familiar with
  the NC Report Card
• You will need to be able to locate, select,
  extract, organize, interpret, and present
  information from all the data sets found in the
  NC School Report Card.
• Your mid-term exam will ask you to develop a
  comprehensive needs assessment for your assigned
  school.

17
Mid-Term Needs Assessment

• Distributed on 3/01/06
• Due on 3/15/06
• Three Needs Assessment Oral Presentations
• 3/15/06
• Signup Sheet For Oral Presentations

18
Needs Assessment Template

• 3 Major Components
• Analysis of Student Achievement Results
• Review of Current Requirements Future
  Expectations
• SMART GOALS Improvement Plan

19
Analysis of Student Achievement Results

• You will present longitudinal data by test
  scores, grade, ethnicity, free and reduced lunch,
  disability.
• You will include information on student
  performance, safe and orderly schools, and
  quality teachers.
• You will also include comparisons to district and
  state proficiency scores and data.

20
Review of Current Requirements and Future
Expectations

• North Carolina NCLB AYP Mandates
• Stair step percentages
• Three-year percentage increases
• Determined by NC Targets
• Future Expectations 100 Prof/Adv by 2014

21
SMART GOALS Improvement Plan

• Specific, Measurable, Attainable,
  Results-oriented, Timebound percentages.
• Long-term goals (3 years)
• Short-term objectives (annual)

22

Six Required Items Within the Components

1. You must use the past three (3) years of
   statistics and data from your school.
2. You must analyze your schools information in
   terms of how it relates and compares with current
   school district results, current state results,
   and the target proficiency goals.
3. You must answer a list of specific questions
4. Your needs assessment must be clear, concise,
   grammatically correct.
5. All data, tables, and graphs must be presented in
   terms that anyone can read, assess, and
   comprehend.
6. Your SMART Goals must support a rational,
   reasonable, and attainable plan with specific
   objects based on your data analysis.

23
1. Past three (3) years of statistics and data
from your school.

• Present overall and disaggregated achievement
  results of students (e.g., grade level,
  ethnicity, lunch status, ESL, special education)
  in the following areas Language Arts and Math
  achievement percentages,
• LA/Math achievement percentages by subgroups,
• Graduation rate percentages (high schools),
• Attendance rate percentages (elementary and
  middle schools),
• Safe and Orderly School data
• Teacher Quality data

24
2. Data Comparisons

• You must analyze your schools information in
  terms of how it relates and compares with
• Current school district results,
• Current state results, and
• Target Proficiency Goals.

25
3. You must answer the following questions

• What are the problems?
• What are the trends relative to student
  performance?
• Who do these problems most affect?
• What are your SMART Goals?
• What are your SMART Objectives?

26
Required Items Within Components

1. You needs assessment must be clear, concise,
   grammatically correct
2. All data, tables, and graphs must be presented in
   terms that anyone can read, assess, and
   comprehend.
3. Your SMART Goals must support a rational,
   reasonable, and attainable plan with specific
   objects based on your data analysis.

27
Written Narratives

• You must provide a written narrative that
  explains and presents all relevant student
  achievement data analysis
• You must provide a written narrative that
  explains and presents your review of current
  requirements and future expectations of NC NCLB
  mandates for your school.
• You must provide a written narrative that
  explains and presents all SMART Goals and
  Objectives.

28
Needs Assessment Purpose
The purpose of needs assessment is to identify
the strengths and weaknesses of existing
programs, practices, procedures, and
activities. The identification of strengths
weaknesses should include student test scores,
student demographic data, discipline
attendance rates, teacher quality measurements,
and report card rankings.
29
WHY ARE WE DOING THIS?

• If your teachers cannot interpret understand
  test scores, then student achievement information
  will have little to no value for anything.
• Test interpretation skills can be taught in a
  forty-five minute interactive workshop.
• You will be able to present this workshop at the
  end of this course (Yes, ..

30
THIS MEANS YOU!!!!!!!!!!!!!!
31
Student Achievement Data

• NC ABCs tests give administrators, faculty, and
  students specific information on what skills
  content areas students have mastered
• The most effective strategy for principals and
  faculty members is to analyze test data by
  dividing into groups to identify curricula
  strengths and weaknesses

32
Needs Assessment Process Timeline

• After identifying strengths and weaknesses in the
  curriculum, the principal and the faculty members
  should develop an action plan with SMART Goals to
  target these areas.
• No more than three areas should be targeted
  during any one academic year.
• The SMART GOAL Action Plan should be implemented
  at the beginning of an academic year.

33
Needs Assessment and SMART GOALS

• The three areas targeted for improvement in
• the Needs Assessment should be written as
• SMART Goals
• Specific
• Measurable
• Attainable
• Realistic
• Timebound

34
SMART Goals SMART Objectives

• SMART Goals are typically long-term statements
  about moving students from one achievement level
  to the next.
• If, for example, 70 of your students scored
  Proficient or higher on Reading/Language
  Arts/English tests at the end of the 2005/2006
  school year, and the NC Target AYP Proficiency
  Standard for 2009 is 85, then your SMART Goal
  could read as follows

35
SMART Goal Example

• Specific Students in our school will
  demonstrate
• mastery of Reading/Language Arts/English
• Measurable - as evidenced by NC EOC state
  assessment tests
• Attainable - by increasing the percentage of
  students scoring Prof/Advanced
• Results-oriented 15
• Timeboundin the 2008/2009 school year.

36
SMART Objectives for SMART Goals

• SMART Objectives are typically annual or yearly
  student achievement statements that support the
  long-term SMART Goal.
• In the previous example, your school could write
  a SMART Objective for each year (2007, 2008,
  2009) that supported the overall long-term goal
  of increasing student proficiency percentages in
  Reading/Language Arts/English by 15 in the
  2008/2009 school year. The SMART Objectives
  could be

37
Annual SMART Objectives for Increasing
Reading/Language Art/English Test Scores

• Specific Students in our school will
  demonstrate
• mastery of Reading/Language Arts/English
• Measurable - as evidenced by NC EOC state
  assessment tests
• Attainable - by increasing the percentage of
  students scoring Prof/Advanced
• Results-oriented Timebound 5 in the 2006/2007
  school year, 5 in the 2007/2008 school year, and
  5 in the 2008/2009 school year.

38
SMART GOAL Program Evaluation Matrix
SPECIFIC Based on needs assessment Measurable Information needed to address the needs and/or problems you are trying to solve Attainable Data collection, how you are going to collect (and if you can collect) the information to assess your progress Realistic Performance indicators, what you need to be doing to accomplish your objectives and goals Timebound When you will begin data collection, complete data collection, and report your evaluation
Determines long-term goals and yearly objectives Can we measure what we are trying to evaluate Observations, document review, interviews, test data, and surveys Can we realistically achieve these goals and objectives? Must establish and stick a schedule

39
Survey Data

• Student Surveys
• Teacher Surveys
• Parent Surveys

40
Needs Assessments Survey Data

• Surveys should be used to measure stakeholders
  perception of how well the school is performing.
• Students, parents, teachers, staff, and members
  of the school community should be surveyed to
  measure their perceptions of the schools
  performance.
• Students should be given the opportunity to
  participate in the survey.
• Students often provide valuable perceptions about
  school climate and discipline practices.

41
ISS Assessment Survey

1. Please describe the purpose of your schools ISS
   Program.
2. Do you have written procedures that detail how
   students are referred to the program? If yes
   please explain.
3. Who has the responsibility for determining if
   in-school suspension is the appropriate course of
   action?
4. What are the expectations for students who are
   assigned to ISS?
5. How do students who are referred to ISS keep up
   with their class work?
6. Do ISS students receive direction and support on
   how to modify their negative behavior patterns?
   If yes, please describe these support services.
7. How are parents of ISS students involved in the
   program?
8. Please describe how ISS students academic
   progress and behaviors are monitored after
   returning to the regular classroom.
9. How do students with special needs who are
   referred to ISS continue to receive their FAPE
   services written in their IEPs?
10. Do you believe the ISS program in your school is
    effective? Please explain.

42
Arts Integration Training Workshop Survey
What were your expectations coming into this
workshop? Were your expectations met? In what
ways were they or were they not? On a scale of
1-5 (5 being most satisfied), how would you
rank this session? What did you learn or
experience that you think will be particularly
useful for you working in the classroom? What
did you take away from this workshop that you
would like to have more of?
43
Teacher Survey
SA A DK D SD

1. New teachers in my school receive a quality
   mentoring program.
2. Teachers at my school plan and work together in
   grade-level, vertical, and other team formats.
3. My schools goals are clearly communicated to
   me.
4. I enjoy working at this school.
5. I receive a lot of support from other faculty
   members in my school.
6. I have the resources, supplies and materials I
   need to do my job.
7. If I had the chance, I would leave education and
   find a job in another field.
8. My school is safe for students and teachers.
9. I know I can depend on help from other teachers
   in my school.
10. I receive high job satisfaction from teaching in
    my school.

44
Parent Satisfaction Survey (Sample Items)

• My childs school provides high quality teaching
  each day.
• My childs teacher cares about him/her.
• My childs school does a good job communicating
  with parents.
• The tutorial assistance program at my childs
  school is improving student achievement.
• My child feels safe at school each day.
• The administration at my childs school does a
  good job.
• The principal at my childs school is open and
  willing to talk to parents.

45
Survey Categories

• There are seven general categories that most
• schools should collect data on
• School Climate
• School Culture
• School Administration (Principal/A.P.)
• Discipline
• Parental/Community Involvement
• Professional Development
• Technology

46
Survey Instrument Process

• Survey items should be aligned to the categories
  this simplifies the process of identifying
  patterns.
• Survey items must be simply written and easily
  understood.
• The reader should not have to apply problem
  solving skills to complete the survey.
• The survey should not contain more than 20 items.
• It should not take the reader more than fifteen
  minutes to complete.

47
Archival Data In Needs Assessments

• Previous school report cards issued to schools by
  state departments of education, school
  improvement plans, suspension and drop out rates
  of students
• The schools mission statements, school
  newsletters, school yearbooks, and any other
  materials that would provide any insight as to
  what events/persons direct the school to be as it
  is today.
• The more information collected, the more
  information that will be learned about the
  schools strengths, weaknesses, and improvement
  needs.

48
Demographic Data in Needs Assessments

• Demographic data includes student population,
  number of students who qualify for free or
  reduced meals, racial composition of students,
  mobility of students, education and occupation of
  students parents and guardians.
• The purpose of demographic data is to form a
  concept of the students home environment.

49
Demographic Data in Needs Assessments

• Educators should not have lower expectations for
  students from lower socioeconomic status
  environments.
• However, they may have to provide learning
  experiences that these students have not had due
  to lack of resources.
• For example, grants may need to be written or
  resources allocated so that students will have
  trade books in their homes during the summer.

50
Needs Assessment Process

• It is necessary to include all stakeholders in
  the needs assessment process.
• It is only good managerial practice to create a
  core committee to initialize, collect, manage,
  sort, and publish data.
• The core committee or steering committee should
  be composed of responsible individuals who will
  be key players in the needs assessment process.

51
Principals Role In Needs Assessment

• As the school leader, you must have a strong
  knowledge base and clearly understand the NC
  Accountability Model for Public Schools.
• Your stakeholders will expect you to be
  comfortable discussing Report Card Results, NCLB
  Mandates, AYP Benchmarks, and School Ratings.

52
Needs Assessment Summary

• A needs assessment is a comprehensive evaluation
  plan of a schools strengths and weaknesses.
• The concept of needs assessment is to build on
  the schools strengths and improve areas of
  weakness.
• Educators are expected to use data from the needs
• assessment to write a school improvement plan
  that has specific SMART Goals and SMART
  objectives designed by the school stakeholders to
  improve the school and student achievement.
• This process is more important than the data
  collection.

53
Program Design, Evaluation, Needs Assessment

• If we know that a schools goals are its
  programs, and we know that programs should be
  based on worth, value, merit, and solutions to
  problems, then it makes sense to establish school
  needs assessments as the critical factor to be
  used in determining program design, program
  inputs, program processes, program outputs,
  program outcomes, and program evaluation.

54
Questions?