State of Wisconsin Department of Public Instruction Elizabeth Burmaster, State Superintendent

1
State of WisconsinDepartment of Public
InstructionElizabeth Burmaster, State
Superintendent
2
WKCE-CRT Spring 2006 Post-Test Workshop

• Session 2 Introduction to the 2005-06 Wisconsin
  Student Assessment System (WSAS) Reports

3
Session 2 Topic Overview

• Part A Understanding Assessment Results
  General Guidelines (25 minutes followed by QA)
• Part B Using Individual Student Data and the
  Online Reporting System (ORS) to examine
  performance (45 minutes followed by QA)

4
Part A Understanding Assessment Results

• Interpreting Test Results General Guidelines,
  Considerations, and Limitations
• Looking at Student Performance by Groups Grade
  Levels, Schools, and Districts
• Explaining Assessment Results to Others (school
  board members, the media, etc.)

5
Part A Understanding Assessment Results

• 1. Interpreting Test Results General
  Guidelines, Considerations, and Limitations

6
Interpreting Test Results General Guidelines,
Considerations, and Limitations

• Single test administration single piece of
  evidence and a snapshot in time
• Tests serve many purposes WKCE-CRT is primarily
  an accountability test

7
Interpreting Test Results General Guidelines,
Considerations, and Limitations
8
Interpreting Test Results General Guidelines,
Considerations, and Limitations

• Know limitations of each measuresome are
  comparable across years and others arent
• Fall 2005 WKCE-CRT scale is different, but
  Proficiency means the same
• Fall 2002 Proficiency re-defined, but scale
  remained the same

9
Interpreting Test Results General Guidelines,
Considerations, and Limitations

• Some measures are not comparable across years
• A Standards Performance Index (SPI) is not
  designed to be compared across years, since the
  items change from year to year.

10
Part A Understanding Assessment Results

• 2. Looking at Student Performance by Groups
  Grade Levels, Schools, and Districts

11
Looking at Student Performance by Groups Grade
Levels, Schools, and Districts

• BEFORE you print reams of paper
• What do you want to compare?

12
Looking at Student Performance by Groups Grade
Levels, Schools, and Districts

• BEFORE you print reams of paper
• 2. Why do you want to know?

13
Looking at Student Performance by Groups Grade
Levels, Schools, and Districts

• BEFORE you print reams of paper
• 3. How many years of data do you need?

14
Looking at Student Performance by Groups Grade
Levels, Schools, and Districts

• BEFORE you print reams of paper
• 4. Which is more important status or growth?

15
Looking at Student Performance by Groups Grade
Levels, Schools, and Districts

• BEFORE you print reams of paper
• 5. Who are you comparing your school/district to?

16
Part A Understanding Assessment Results

• 3. Explaining Assessment Results to Others
  (school board members, the media, etc.)

17
Explaining Assessment Results to Others

• Testing is a complex topic to explain clearly
• but now affects most students (and their
  parents/guardians)
• Budget constraints pressure to demonstrate
  results and program effectiveness

18
Explaining Assessment Results to Others

• When Results Change from Year to Year
• Be modest in claiming success
• One year of improvement does not indicate a major
  upward trend
• Variation occurs, especially with small groups
• What goes up can also come down

19
Explaining Assessment Results to Others

• When Results Change from Year to Year
• Is this an actual score decline, or lower
  comparative performance?
• Statistical significance isnt the same as
  practical significance
• Be the first to explain the data it should not
  be discovered

20
Explaining Assessment Results to Others

• When Results Change from Year to Year
• Why did the drop occur? Possible factors
• Change in attendance patterns?
• New program or curriculum implemented?
• Different demographics of tested students?

21
Questions?

22
Part B Using Individual Student Data and ORS to
Examine Performance

• Looking at Individual Student Performance the
  Individual Profile Report (IPR)
• Using the Online Reporting System (ORS) to
  examine performance

23
Part B Using Individual Student Data and ORS to
Examine Performance

• Looking at Individual Student Performance the
  Individual Profile Report (IPR)

24
Looking at Individual Student Performance the
Individual Profile Report (IPR)

• IPR is the only report printed on paper not
  available through ORS
• Districts receive 2 copies for each tested
  student 1 for the district and 1 for parents
• Students who take Wisconsin Alternate Assessments
  receive a separate IPR

25
Looking at Individual Student Performance the
Individual Profile Report (IPR)

• Re-designed 2005-06 IPR has both
  criterion-referenced and norm-referenced results
• CRT results scale score, proficiency level, and
  Standards Performance Index (SPI)
• NRT result students state percentile

26
Looking at Individual Student Performance the
Individual Profile Report (IPR)
27
Looking at Individual Student Performance the
Individual Profile Report (IPR)
28
Looking at Individual Student Performance the
Individual Profile Report (IPR)
NRT
CRT
29
Looking at Individual Student Performance the
Individual Profile Report (IPR)
CRT
30
Looking at Individual Student Performance the
Individual Profile Report (IPR)
31
Looking at Individual Student Performance the
Individual Profile Report (IPR)
32
Looking at Individual Student Performance the
Individual Profile Report (IPR)
33

• 2. Using the Online Reporting System (ORS) to
  examine performance

34

• The URL for the ORS site is https//wsasors.turnle
  af.com.

35
Overview

• ORS Topics
• ORS Overview/History
• System Design and Security
• ORS Reports Walk-Through

36
ORS Overview/History

• Original purpose (early 2005) reduction in paper
  reports
• Expanded purpose (late 2005) more flexible
  powerful reporting tool
• Proficiency Longitudinal Report (new for 2005-06)
  

37
System Design and Security

• ORS is a secure, password-protected site that
  reports private (non-suppressed) data NOT FOR
  PUBLIC USE (with exception of Public PDF files)
• Main site for public reporting is the Wisconsin
  Information Network for Successful Schools
  (WINSS) site, located at http//www.dpi.wi.gov/sig
  /index.html.

38
System Design and Security

• On-line reports (ORS and WINSS) are not intended
  for use in making Adequate Yearly Progress (AYP)
  calculationssee http//dpi.wi.gov/oea/accounty.ht
  ml for AYP information.

39
System Design Security

• Two levels of data district and school
• Two types of user roles
• assessment coordinators (with administrative
  rights)
• users (no administrative rights)

40
System Design Security

• District Assessment Coordinator (DAC) is the
  point of contact for each district
• School Assessment Coordinator (SAC) is the point
  of contact for each school

41
System Design Security

• Only one DAC and SAC for each district and school
  create user accounts for others
• New DAC? Notify DPI using DAC Update Form at
  http//www.dpi.wi.gov/oea/cds.html.

42
System Design Security

• How to create new users click on Manage Users
  and then Create New User

43
Creating a New User
1
2
44
System Design Security

• DACs SACs can view usernames and passwords by
  clicking on Manage Users and then Create User
  Reports

45
Viewing User Names Passwords
1
2
46
Help, FAQ, and Interpretive Info

• Interpretive information (including a searchable
  index of topics) is found under the Help and
  FAQ links at the top right-hand portion of the
  screen, as well as the Report Legend link on
  the left-hand side.

47
Interpretive Information
FAQ
Help
Report Legend appears here on specific ORS
reports
48
The Individual Student Data File

• The Individual Student Data File (formerly the
  GRT) contains all information about every tested
  child demographics and performance
• Opens easily as an Excel file
• Available only to District Assessment
  Coordinators (DACs) through the Online Reporting
  System (ORS)

49
The Individual Student Data File
Individual Student Data File DACs only
50
4. ORS Reports Walk-Through

• 10 ORS Reports (including one new for 2005-06)
• Proficiency Summary Report
• Standards Performance Summary
• Scale Score Summary Report
• Item Response Analysis Selected Response
• Item Response Analysis Constructed Response
• Item Analysis Summary by Standard
• Extended Writing Sample by Group
• Extended Writing Sample by Rubric
• School Record Sheet (redesigned for 2005-06)
• Proficiency Longitudinal Report (new for 2005-06

51
Common Questions and Where to Find the Data in ORS

• Where can I see the percentage of students in my
  school/district who were tested, took alternate
  assessments, and scored in each proficiency
  category?
• The Proficiency Summary Report

52
Proficiency Summary Report
Graph
Printing for PDF
Filters for Selecting year, school, grade,
subject, and demographic
Shaded boxes for a row add to 100
53
Changing Tables to Graphs
Shows WSAS, WKCE, or WAA
Change scale
Mouse-Over shows Number for each color
54
Common Questions and Where to Find the Data in ORS

• Where can I see a summary of how students in my
  district/school did in the various standards
  within a subject?
• Standards Performance Summary

55
Standards Performance Summary
56
Common Questions and Where to Find the Data in ORS

• How can I get a summary of how students in my
  district/school did in terms of scale scores in
  relation to other tested students statewide?
• Scale Score Summary

57
Scale Score Summary Report
State Percentile at the Mean NCE
58
Common Questions and Where to Find the Data in ORS

• How can I see how students in my district/school
  did on individual items measuring specific
  content standards?
• Item Response Analysis-Selected Response
• Item Response Analysis-Constructed Response
• Item Analysis Summary by Standard

59
Item Analysis Reports

• Change in reporting convention for Fall 2005
  WKCE-CRT Item Analysis Reports report to
  Assessment Frameworks rather than to Wisconsin
  Model Academic Standards

60
Item Response Analysis Selected Response
Shows district-state and school-district
differences
Correct answer bolded
Depth of Knowledge
61
Item Response Analysis Constructed Response
62
Item Analysis Summary by Standard
63
Common Questions and Where to Find the Data in ORS

• Where can I see results for the Writing
  assessment at grades 4, 8, and 10 by student
  subgroup and by rubric (Composition and
  Conventions)?
• Extended Writing Sample by Group
• Extended Writing Sample by Rubric

64
Extended Writing Sample by Student Group
65
Extended Writing Sample by Rubric
Writing is Descriptive at grade 4 and
Persuasive at grades 8 and 10
66
Common Questions and Where to Find the Data in ORS

• Where can I see a roster of all students tested
  in my school, with their results by subject?
• School Record Sheet

67
School Record Sheet - Live Version
Individual student names appear here in alpha
order, 15 names per page
68
School Record Sheet PDF Version
Individual student results here in alpha order
WAA students at the end
69
District Record Sheet
Individual schools appear here in alpha order
70
Proficiency Longitudinal - NEW
71
Proficiency Longitudinal - NEW
72
Contacts

• CTB Wisconsin Help Desk 1-800-282-2203
• Office of Educational Accountability
  608-267-1072
• Questions??