New England Common Assessment Program

1
New England Common Assessment Program
Guide to Using the October, 2006 NECAP
Reports Companion PowerPoint Presentation
February 2007
2
Welcome and Introductions

• Mary Ann Snider
• Director of Assessment and Accountability
• RI Department of Education

3
Welcome and Introductions

• Measured Progress Service Center 1-877-632-7774

4
Purpose of the Workshop

• To assist schools and districts in using the
  variety of reports and support material issued by
  the NECAP states in conjunction with the release
  of results
• To provide school and district personnel an
  opportunity to ask questions regarding the
  reports and support materials

5
Involvement of Local Educators

• Development of Grade Level Expectations
• Test Item Review Committees
• Bias and Sensitivity Review Committees
• Classroom Teacher Judgment data
• Standard Setting Panelists
• Technical Advisory Committee

6
Basics of Test Design

• Fall test previous years GLEs
• Reading and Mathematics grades 3-8
• Writing grades 5 and 8
• Variety of Item Types
• Reading multiple choice and constructed response
• Mathematics multiple choice, short answer (one
  point), short answer (two points), and
  constructed response
• Writing multiple choice, constructed response,
  and extended response
• Common and Matrix Items

7
Two Basic Types of Educational Tests

• Norm-Referenced What is the students status or
  rank in a designated group?
• Criterion or Standards-Referenced Has the
  student learned the skills or achieved the
  standard?
• NECAP is a Standards-Referenced Test

8
Family Educational Rights and Privacy Act

• The Family Educational Rights and Privacy Act
  (FERPA)
• Access to individual student results is
  restricted to
• the student
• the students parents/guardians
• authorized school personnel
• Superintendents and principals are responsible
  for maintaining the privacy and security of all
  student records.
• Authorized school personnel shall have access to
  the records of students to whom they are
  providing services when such access is required
  in the performance of their official duties.
• FERPA website http//www.ed.gov/policy/gen/guid/f
  pco/ferpa/index.html

9
National Council on Measurement in Education
www.ncme.org
10
Questions and Answers

• Questions about the purpose of the
• workshop and/or NECAP background?

11
Rhode Island Results
Suburban
Urban Fringe
Urban
12
Rhode Island Results
Indicates that gain exceeds the gain made by
all students
13
Types of Reports

• Student Report
• Item Analysis Report
• School/District Results Report
• School/District Summary Report
• District Student Level Data Files

14
Student Report

• Two copies mailed to the school
• color copy for parents/guardians
• black and white copy for local use
• Dual sided text side and data side
• Spanish version of the student report available
  on the RIDE website for the Office of Assessment
  and Accountability

15
(No Transcript)
16
(No Transcript)
17
(No Transcript)
18
(No Transcript)
19
Fun with Student Level Reports!

• Review to be sure you have a student report for
  every student tested in October, 2006.
• Prepare a letter to provide your interpretation
  of the results for your school and/or district.
• Be sure teachers are prepared and comfortable
  discussing individual student results with
  families including any instructional
  interventions.
• Use your district data file to analyze growth
  between students you had last year and this year.

20
Questions and Answers

• Questions about the Student Report?

21
Teaching School vs. Testing School

• Rationale
• Inclusion of students in Teaching Year reports

22
Teaching School vs. Testing School
Schools Can View Reports for Testing Year
(2006-07)
23
Teaching School vs. Testing School
Or Teaching Year(2005-06)
24
Teaching Year (accountability)
Testing Year(released this month)
25
Item Analysis Report

• Available to schools and districts on-line
• These reports are confidential and the website is
  password-protected.
• One per content area
• Use in conjunction with Released Items documents

26
(No Transcript)
27
(No Transcript)
28
Released Item Documents
29
Grade 5 2006 Released Mathematics Item
DOK 3 GLE DSP 4.1
30
Grade 5 Practice Test Item (page 4)
DOK 2 GLE DSP 4.1
31
Fun with Released Support Materials!

• Build practice tests
• Share writing samples as models
• Mock scoring sessions to calibrate teacher
  judgments
• Compare released items (GLE and DOK) with
  classroom instruction and assessments and student
  work.
• Map instruction and scaffolding practices to
  items based on student performance.

32
Released Item Documents

• RIDE website at http//www.ride.ri.gov/assessment
  /NECAP.aspx
• New information provided in the Released Item
  Support Materials documents includes
• For all content areas, providing the
  multiple-choice items and the GLE code and text
  that each item is linked to
• For math, indicating which items were included in
  session 1 of the math test by adding a no
  calculator symbol above the items
• For reading, listing the passage name and type of
  text
• For writing, listing passage name and mode of
  writing

33
Questions and Answers

• Questions about the Item Analysis Report or the
  Released Item documents?

34
School/District Results Report

• Available on-line at the Measured Progress
  website
• Separate report for each grade level tested
• District report contains no individual school
  data

35
http//iservices.measuredprogress.org
36
New this year
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
Grade Level Summary Report

• Page 2 of the NECAP School/District Results
  Report
• Summary of participation and results

43
(No Transcript)
44
Fun with Grade Level Summaries!

• Review last years grade level summary report
  with this years
• If gains were made, identify what strategies,
  interventions, and programs may have worked
• If no progress was made, hypothesize why and what
  can be done at the school or district level to
  help students make progress
• Identify where your results differ from the
  district and state results. Hypothesize why they
  may differ

45
Questions and Answers

• Questions about the Grade Level
• Summary Page?

46
Historical Data

• Data is included for two years and is combined in
  the Cumulative Total row
• In the Cumulative Total row for the achievement
  level columns
• The N columns are summed
• The columns are found by dividing the total
  number of students in the level by the total
  number of students tested
• The Mean Scaled Score column is a weighted average

47
(No Transcript)
48
Comparisons of NECAP Scores Across Years

• School and District-Level Scaled Scores and
  Achievement Levels
• Student-Level Scaled Scores and Achievement
  Levels
• Content Area Subscores

49
Content Area Results

• Pages 3, 5, and 7 of the NECAP School/District
  Results Report
• Provides information about achievement in
  subtopics of tested content areas

50
(No Transcript)
51
Number of Scaled Score Points Denoting Minimally
Statistically Significant Difference for Average
Group Results
Standard error of the mean difference
with assumption n1n2 and s1s2
52
Percentage Difference in Student Achievement
Level Classification Denoting Minimally
Statistically Significant Differences for Group
Results
0.05 level of statistical significance
53
Questions and Answers

• Questions About the Content Area
• Results Page?

54
Disaggregated Content Area Results

• Pages 4, 6, and 8 of the NECAP School/District
  Results Report
• Provides information about achievement for
  different groups in a school/district

55
(No Transcript)
56
Fun with Disaggregated Results!

• Compare last years Disaggregated Report with
  this year
• Highlight in yellow, groups that had significant
  positive increases
• Highlight in blue, groups that remained stable or
  had decreases
• Establish cross-grade and content groups to
  hypothesize what the school can do to close gaps
  and maintain increases
• Access to rigorous curriculum?
• Professional development in a specific area?
• Adequate support programs?
• Adequate family involvement?

57
Questions and Answers

• Questions About the Disaggregated Content Area
  Results Page?

58
School/District Summary Report

• Separate report for each grade level tested
• Provides details about student achievement by
  content area

59
School/District Summary Report

• The first line designates the year the test was
  administered
• The second line is the name of the report
• The third line differentiates between teaching
  year and testing year

60
(No Transcript)
61
Fun with School Level Summary Data!

• Are there consistent patterns of achievement
  across grades? Across content areas?
• Compare results to last years school summary
• Highlight in yellow, areas where there has been
  significant growth. Highlight in blue, areas
  where there has been no growth or decline.
• Establish cross-grade and content groups to
  hypothesize what the school can do to close gaps
  and maintain increases
• Develop a school/district plan to address areas
  of weakness and continue areas of growth

62
Student Level Data Files

• All demographic information for each student that
  was provided by the state
• The scaled score, achievement level, and
  subscores earned by each student in all content
  areas tested
• Also contain
• Performance on released items
• Student questionnaire responses
• Optional reports fields data

63
(No Transcript)
64
Supporting Materials

• Guide to Using the 2006 NECAP Reports and
  companion PowerPoint
• Grade Level Expectations
• Test Specifications documents
• Release Item documents
• Preparing Students for NECAP Tips for Teachers
  to Share with Students
• Technical Report