Guide to Using the 2006

1
New Hampshire Statewide Assessment

• Guide to Using the 2006
• NECAP NH-Alternate
• Reports
• Companion PowerPoint Presentation
• February 2007

2
Agenda

• Introductions
• NHDoE Updates
• Background
• NH-Alternate Assessment Reporting
• Short break
• NECAP Reporting

3
Welcome Introductions
4
Welcome Introductions

• Measured Progress Service Center
  1-877-632-7774
• Measured Progress Offices 1-800-431-8901

5
NHDoE Contacts
http//www.ed.state.nh.us/assessment

• Linda Stimson - ELA Coordinator
• Phone [phone]
• E-mail LStimson_at_ed.state.nh.us
• Rich Andrusiak - Mathematics Coordinator
• Phone [phone]
• E-Mail RAndrusiak_at_ed.state.nh.us
• Jan McLaughlin - Science Coordinator
• Phone [phone]
• E-Mail JMcLaughlin_at_ed.state.nh.us
• Ken Relihan - Social Studies Coordinator
• Phone [phone]
• E-mail KRelihan_at_ed.state.nh.us
• Deb Fleurant - Bias and Sensitivity and Title I
• Phone [phone]
• E-mail DFleurant_at_ed.state.nh.us

• Tim Kurtz
• Director of Curriculum and Assessment
• Phone [phone]
• E-Mail TKurtz_at_ed.state.nh.us
• Gaye Fedorchak
• Alternate Assessment Coordinator
• Phone [phone]
• E-Mail GFedorchak_at_ed.state.nh.us
• Susan Morgan - ELL Access Specialist
• Phone [phone]
• E-Mail SMorgan_at_ed.state.nh.us
• Carol Angowski
• Assessment Program Specialist
• Phone [phone]
• E-mail CAngowski_at_ed.state.nh.us

6
Update High School Assessment

• Assessment
• NECAP
• Piloted Reading, Math, and Writing in November
• No student, school, or district scores will be
  produced
• First operational test October 2007
• NH-Alt
• 2006-2007 Grade 10 portfolios in Reading,
  Mathematics, and Writing
• 2007-2008 In addition to above, Grade 11
  portfolios in science
• Accountability
• Schools AYP based on Graduation rate only
• Districts AYP based on May 2006 NHEIAP and
  2005-2006 Alt

7
Update Science Assessment

• Assessment
• NECAP
• Pilot Science May 14-24, 2007
• Two Sessions about 90 minutes each
• No student, school, or district scores will be
  produced
• First operational test (three sessions) May
  2008
• Workshops during the week of April 30 May 4
• NH-Alt
• Grade 11 Science Portfolios will be added in
  school year 2007-2008
• Accountability
• None at this time, but NCLB reauthorization
  proposals include it starting in school year
  2007-2008.

8
Update Student Demographic Assessment Data

• i.4.see Workbench
• Hub for district submitted student level data
• Many new build-it checks on submitted data
• i.4.see Dashboard
• Check summative student demographic files that
  affect AYP
• Ability to change incorrect data
• Notice will go to Superintendents, Special
  Education Directors, i.4.see coordinators, and
  Principals
• Hope to go live on Monday, February 12, 2007
• Deadline on changes has not been set yet
• Follow the Child Assistance Center
• Phase I April/May
• Phase 2 early summer

New
9
Updates

• Any questions about Updates?

10
Purpose of the Workshop

• To assist schools and districts in using
• the variety of NH Statewide Assessment
• reports and support material in conjunction
  
• with the release of results
• To provide school and district personnel an
• opportunity to ask questions regarding the
• reports and support materials

11
Local Educator Involvement

• Development of Grade Level Expectations (NECAP)
• Development of Alternate Achievement Standards
  Linked to Grade Level Expectations (NH-Alt)
• Test Item Review Committees (NECAP)
• Bias and Sensitivity Review Committees (NECAP)
• Classroom Teacher Judgment Data (NECAP)
• Standard Setting Panelists (NECAP and NH-Alt)
• Technical Advisory Committee (NECAP and NH-Alt)
• NH-Alt Advisory Task Force (NH-Alt)

12
Test Basics

• When
• NECAP Oct 2006
• NH-Alternate School year 2005-2006
• What
• Reading and Mathematics
• NECAP Grades 3-8 (tests previous years GLEs)
• NH-Alt Grades 2-7, 10 (tested during
  instruction last year)
• Writing
• NECAP Grades 5 8 (tests previous years GLEs)
• NH-Alt Grades 4 7 (tested during instruction
  last year)

13
Basics of Test Design

• NECAP
• Variety of Item Types
• Reading multiple choice and constructed response
• Mathematics multiple choice, short answer 1
  point, short answer 2 points, and constructed
  response
• Writing multiple choice, constructed response,
  and extended response
• Common and Matrix Items
• Common items used to generate student scores
• Matrix items used for equating and field testing

14
Basics of Test Design

• NH Alternate Portfolio Assessment
• What is an Item in NH-Alt?
• In each content area, two standards-based
  targeted skills are identified that address
  student learning needs
• Baseline data are collected for each targeted
  skill
• Work samples are gathered over the course of the
  year to show how the student is progressing on
  these two targeted skills
• The collection of work samples and data presented
  in the portfolio are scored as a single item
• Progress data and student work samples are
  collected across the school year (Sept April)
  on each targeted skill as the student is learning
  the skill.
• Reading 2 Targeted Skills
• Mathematics 2 Targeted Skills,
• Writing 2 Targeted Skills

15
Basics of Test Design

• NH Alternate Portfolio Assessment
• Targeted skills are represented by a variety of
  work samples collected over the course of the
  year.
• Collections include multiple samples of actual
  student work, which typically show the student
  solving problems in many ways, i.e. multiple
  choice responses, short answer responses, and
  longer responses that look similar to constructed
  response problems. Responses often involve use
  of manipulatives, communication systems, or other
  instructional technology.
• Authentic classroom constructed response and
  short answer tasks provide the best methods for
  allowing students with severe disabilities to
  show their work in the way they are able to
  produce it.
• Provides richer evidence of access to
  standards-based content and student progress.

16
Types of Educational Tests

• Norm-Referenced What is the students status or
  rank in a designated group?
• Criterion-Referenced (Standards-Based) Has the
  student learned the skills or achieved the
  standard?
• Both NECAP and NH Alternate Assessments are
  Criterion-Referenced Tests.

17
Questions and Answers

• Questions about
• the purpose of the workshop
• or
• NECAP or NH-Alternate background?

18
Basics of Test Design

• Student Report (Confidential) with Information
  for Parents and Report Interpretation Guide
• Student Roster Report (Confidential) School and
  District levels by student
• District Student Level Data Files (Confidential)
• Disaggregated Results by Content Area (Public)
  School, District, State reported separately for
  each grade
• State Summary Reports (Public)
• NH-Alt Students included in NECAP reports as
  NT-Approved

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
NH-Alt Achievement Levels

• Names of Achievement Levels are the same as in
  NECAP -
• Level 4 Proficient with Distinction
• Level 3 Proficient
• Level 2 Partially Proficient
• Level 1 Substantially Below Proficient
• But - describe performance of grade-linked
  standards-based skills reduced in depth, breadth,
  and complexity for the student.

24
NH-Alt Two Scoring Dimensions

• 1. The Performance Dimension
• Most heavily weighted in score and includes
• Student Progress (from start of year baseline)
• Connections and Access to the General Curriculum
  (Student work samples must show linkage to
  grade-level content at reduced level of depth,
  breadth, complexity)

25
NH-Alt Two Scoring Dimensions

• The Program Dimension -
• Less heavily weighted in scoring, and includes
• Generalized Performance (Student uses skill
  across different settings and situations)
• Self-Determination (Student attempts to self
  direct and monitor work)
• Supports (Does level of assistance provided
  support growth of independence and match student
  need?)

26
Sample Portfolio Score Calculation
27
Raw Score To Achievement Level Conversion Chart
28
Questions and Answers

• Questions about
• NH-Alternate Reporting?

29
NECAP Types of Reports

• Student Report
• Item Analysis Report
• School/District Results Report
• School/District Summary Report
• District Student Level Data Files

30

• The Family Educational Rights and Privacy Act
  (FERPA)
• Access to individual student results is
  restricted to
• the student
• the students parents/guardians
• authorized school personnel
• Superintendents and principals are responsible
  for maintaining the privacy and security of all
  student records.
• Authorized school personnel shall have access to
  the records of students to whom they are
  providing services when such access is required
  in the performance of their official duties.
• FERPA website http//www.ed.gov/policy/gen/guid/f
  pco/ferpa/index.html

31
National Council on Measurement in Education
www.ncme.org
32
Student Report

• Two copies mailed to the school (February 7)
• color copy for parents/guardians
• black and white copy for local use
• Dual sided Parent Letter Student Report
• Spanish version of Parent Letter and Student
  Report shell available for NECAP
• (available at http//www.ed.state.nh.us/NECAP)

33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
Questions and Answers

• Questions about the Student Report?

38
Teaching School vs. Testing School

• Rationale
• Inclusion of students in Teaching Year reports

39
Teaching School vs. Testing School
Schools Can View Reports for Testing Year
(2006-07)
40
Teaching School vs. Testing School
Or Teaching Year(2005-06)
41
Teaching Year
Testing Year
42
http//iservices.measuredprogress.org
43
New this year
44
(No Transcript)
45
(No Transcript)
46
(No Transcript)
47
Item Analysis Report

• Available at the school level via password
  protected website (User ID and passwords sent to
  Principals and Superintendents in January)
• One report for each content area/grade level
  combination
• Issued in both Testing Year and Teaching Year
  formats
• Use in conjunction with Released Items documents

48
(No Transcript)
49
(No Transcript)
50
(No Transcript)
51
Released Item Documents
52
Released Item Documents

• New information provided in the Released Item
  Support Materials - documents include
• For all content areas, providing the
  multiple-choice items and the GLE code and text
  that each item is linked to
• For math, indicating which items were included in
  session 1 of the math test by adding a no
  calculator symbol above the items
• For reading, listing the passage name and type of
  text
• For writing, listing passage name and mode of
  writing

53
Questions and Answers

• Questions about the Item Analysis Report or the
  Released Item documents?

54
School/District Results

• Available on-line (January 30) at
• http//reporting.measuredprogress.org/NECAPpublicN
  H/
• Grade Level Report
• One for each grade in each school and district
• Issued in both testing year and teaching year
  formats
• Summary Reports
• One for each school or district
• Issued in both testing year and teaching year
  formats

55
(No Transcript)
56
Grade Level Results Report

• Page 1 Title Page
• Page 2 Grade Level Summary Report
• Page 3 Reading Results
• Page 4 Disaggregated Reading Results
• Page 5 Mathematics Results
• Page 6 Disaggregated Mathematics Results
• Page 7 Writing Results (Grades 5 and 8 only)
• Page 8 Disaggregated Writing Results (ditto)

57
(No Transcript)
58
Questions and Answers

• Questions about the Grade Level
• Summary Report page?

59
Content Area Results

• Pages 3, 5, and 7 of the NECAP School/District
  Results Report
• Provides current and historical summative
  information about achievement in tested content
  areas
• Provides information about achievement in
  subtopics of tested content areas

60
Content Area Results - Historical Data

• Data is included for two years and is combined in
  the Cumulative Total row
• In the Cumulative Total row for the achievement
  level columns
• The N columns are summed
• The columns are found by dividing the total
  number of students in the level by the total
  number of students tested
• The Mean Scaled Score column is a weighted average

61
(No Transcript)
62
Content Area Results Subscore Results

• Results based on both common matrix equating
  items
• Many Reading items are included twice once
  under Type of Text, and once under Level of
  Comprehension
• Standard Error Bars are included to help
  determine if differences between scores are
  statistically significant

63
(No Transcript)
64
Comparisons of NECAP Scores Across Years

• School and District-Level Scaled Scores and
  Achievement Levels
• Student-Level Scaled Scores and Achievement
  Levels
• Content Area Subscores

65
Number of Scaled Score Points Denoting Minimally
Statistically Significant Difference for Average
Group Results
Standard error of the mean difference
with assumption n1n2 and s1s2
66
Percentage Difference in Student Achievement
Level Classification Denoting Minimally
Statistically Significant Differences for Group
Results
0.05 level of statistical significance
67
Questions and Answers

• Questions About the Content Area
• Results Page?

68
Disaggregated Content Area Results

• Pages 4, 6, and 8 of the NECAP School/District
  Results Report
• Provides information about achievement for
  different groups in a school/district
• Testing year reports have detailed information
  about all students enrolled in October 2006
• Teaching year reports include only those students
  with NECAP scores from Oct 2006.
• Detailed information for districts and the state
  may be found in the district and state reports.
• CSV files containing all this data will be posted
  to the NHDoE website in mid February.

69
(No Transcript)
70
Questions and Answers

• Questions about the Disaggregated Content Area
  Results page?

71
School/District Summary Report

• Separate report for each school and district
• Provides summative information for each grade
  level within the school or district
• Provides summative achievement level data for
  whole school or district
• Provided in both teaching year and testing year
  formats
• CSV files containing all this data will be posted
  to the NHDoE website in mid February.

72
Identifying the Report

• The first line designates the year the test was
  administered
• The second line is the name of the report
• The third line differentiates between teaching
  year and testing year

73
(No Transcript)
74
Questions and Answers

• Questions about School/District Summary Reports?

75
Student Level Data Files

• CSV files available at the district level via
  password protected website (User ID and passwords
  sent to Superintendents in January)
• All demographic information for each student that
  was provided by the state
• Provided in both teaching year and testing year
  formats
• The scaled score, achievement level, and
  subscores earned by each student in all content
  areas tested
• Also contain
• Performance on released items
• Student questionnaire responses
• Optional reports fields data

76
(No Transcript)
77
Questions and Answers

• Questions about Student Level Data Files?

78
Supporting Materials

• Materials available now
• Guide to Using the 2006 NECAP Reports
• 2006 Reporting Workshop Ppt
• Grade Level Expectations
• Released Item documents
• GLE Support Materials

• Materials available soon
• Roster of school and district results (teaching
  year and testing year formats) (csv)
• Disaggregated Data for school districts by
  grade level (teaching year and testing year
  formats) (csv)
• Student work samples without scores or
  annotations (pdf)
• Release Item distracter stats (pdf)
• 2006 Technical Manual