Scale Scoring

1
Scale Scoring

• A New Format for
• Provincial Assessment Reports

2
http//www.ella.ednet.ns.ca
3

• Beginning in the 2007-2008 school year, all
• provincial assessments will be reported on a
• common scale.

4
Why Use a Common Scale?
Imagine trying to accurately compare a students
or a schools achievement when given the
following raw scores

• letter grades
• numerical marks
• rubric levels
• percentage grades

5
What is a Scale?

• A scale is an arbitrarily established set of
  numbers used for measurement.
• Different scales may be used to measure the same
  thing, but they are calibrated differently. (e.g.
  rulers, thermometers, speedometers, bathroom
  scales)

6
What is a Scale?

• A scale, simply put, is a mathematical
• conversion of raw scores to a
• common scale or a derived score.

7

• A scaled score is a conversion of a student's raw
• score on a test to a common scale that allows for
  a
• numerical comparison between students.
• Provincial assessments use multiple versions of a
  test
• over the years. The scale is used to control
  slight
• variations from one version of a test to the
  next.
• Scaled scores, when equated, are particularly
  useful
• for comparing test scores over time.

8
Equating A Statistical Process

• Equating measures the difficulty of each
  assessment (content,
• cognition, difficulty) and adjusts the cut score
  to
• account for differences among assessment forms.
• The difficulty level of tests in different years
  can be
• equated and this enables us to compare assessment
• results from year to year.

9
Advantages of Scale Scores

• Scaled scores allow better comparison of
  assessment
• results
• from year to year
• from one grade level assessment to another
• of student achievement in different subject areas
  

10
Future Advantages

• Because each successive years assessments are
  statistically equated with a baseline reference
  year, the scaled scores can also be used to
  reflect growth.
• For example, if a future scaled score is higher
  than the reference year scale score (2007-2008),
  it can be interpreted as an improvement from that
  base year.

11
Cautions for Comparisons

• Many factors impact upon a schools results.
• No single assessment result tells the whole
  story of a students or schools achievement.
• Sample size does matter! Schools with fewer
  students will show more variability in scores
  from one year to the next.

12
Scale Scoring
Provincial Mean
Cut score

One standard deviation above
65 of students who wrote this
assessment
One standard deviation below
469
600
300
800
700
400
200
500
Scale Scores
13
To review

• Distribution of Scores
• Scale ranges from 200 to 800
• Provincial mean is set to 500
• Provincial standard deviation is set to 100
• On most provincial assessments, about 65 of
  students will place between 400 and 600
• The cut score is 469

14
(No Transcript)
15
Cut Score

• A cut score defines the point at which a
• certain required level of achievement has been
• demonstrated on the assessment.
• Cut scores are set by a panel of educators
• who look at the actual test questions to
• determine levels of acceptable performance.
• A cut score can be above or below the mean.

16
Cut Score

• A cut score differentiates between students who
  are
• meeting early stage expectations
• or
• meeting some early stage expectations
• not yet meeting early stage expectations

17
Interpreting Results Overall Score

• Cut Score
• How close or how far away from the cut score
  (469)
• did the student perform?
• Did the student meet expectations for this
  assessment or has
• the student not yet met expectations for this
  assessment?
• REMEMBER The cut score only applies to the
  overall score
• (section 1 and 2A) the student received.

18
(No Transcript)
19
Interpreting ResultsOverall Score

• Provincial Mean (500)
• How close or how far away from the mean did the
  student perform?
• How did the student perform relative to others
  that wrote the assessment?

20
Using the Results

• Students Performance in Silent Reading
• Provides information on student performance in
  realistic fiction and nonfiction/multimodal text
  and cognitive levels.
• Allows for comparison to the provincial mean of
  500.
• May help teachers isolate concerns and
• develop strategies to address these indicators.
• is intended to provide to teachers information
• for planning purposes only.

21
Using the Results Overall Score

• Students Meeting Expectations (470 and above)
• If the overall score is above the cut score (470)
  the student is meeting early stage expectations.
• If the overall score is within the 351-469 range,
  the student is meeting some early stage
  expectations.
• For every student who is meeting some early stage
  expectations teachers will be expected to
• Collect classroom assessment data
• Develop a Literacy Development Record (LDR)
• provide on-going, in-class support
• contact parents/guardians to discuss support
  options

22
Using the Results Overall Score

• Students Not Yet Meeting Expectations
• (Overall Score below 350)
• For every student who is not yet meeting early
  stage expectations (overall score is below 350)
  teachers will be expected to
• Develop a Literacy Development Record (LDR)
• Support student within the class structure
• Contact parents/guardians to discuss support
  options
• The LDR will be on-going for students, with
  transition meetings at the end of each grade.

23
(No Transcript)
24

• Students with the following performance in
  writing, will require an LDR
• Students who met some early stage expectations
• Students who did not yet meet early stage
  expectations
• Insufficient Evidence

25
Using the Results

• The Writing Performance section
• Provides information on student performance in
  the elements of writing ideas, organization,
  language use and conventions.
• May help teachers identify concerns and trends in
  student writing.

26
Determining Monitored Status

• In some instances, a student who met some of the
  early stage expectations may be placed on
  monitored status.
• If current classroom assessment indicates that a
  students literacy development matches with
  current curriculum expectations, he/she may be
  placed on monitored status.

27
Criteria for determining monitor status

• Student met some early stage expectations or has
  insufficient evidence on ELLA.
• Classroom/school/board assessment information
  collected within the past 4-6 weeks suggest that
  the student meets grade level expectations.
• Classroom assessment information must be attached
  to the LDR.

28
Classroom /school/ board assessment
requirements
Reading - monitoring status Reading - monitoring status Reading - monitoring status
Requirements Details Instructional Level of Expectation
At least 2 current Oral Reading Records (ORR) Unseen fiction/non-fiction texts reflecting curriculum expectations Accompanying comprehension questions and responses 90 accuracy and 80 with comprehension questions and retelling
At least 2 current silent reading passages Unseen fiction/non-fiction texts reflecting curriculum expectations Accompanying comprehension questions (completed in oral, visual or written presentation) 80 with comprehension questions and retelling
29
Writing - monitoring status Writing - monitoring status Writing - monitoring status
Requirements Details Instructional Level of Expectation
At least 2 current student writing samples Two different topics or genres Written , revised, edited and proofread independently Assessed using an analytic writing rubric Student performance must be consistent with current classroom expectations
30
Monitored status

• An LDR is developed.
• Complete to be monitored in Section C of the
  LDR.
• Classroom/school/board assessment data is to be
  attached to the LDR.
• Specific Instructional Practices/ Interventions
  need not be completed on the LDR when
  classroom/school/board assessment data
  demonstrates that the student meets current
  curriculum expectations.

31

• For more information on developing LDRs please
  view the Podcast
• Developing
• Literacy Development Records

32
Using the Results

• Schools are responsible for tracking the LDRs and
  for monitoring support.
• Teachers and Principals are expected to
• review LDRs and monitor support
• ensure parent/guardian signatures on the LDR
• ensure that the completed LDR is kept in the
  students cum file

33
Using Your Data Classroom Level

• Teachers are encouraged to use the student
  reports to identify trends and patterns in
  literacy development.
• Class / grade profiles may be used for
  classroom instructional decision making and for
  targeting support to individual students.

34
Using the Results School Level

• In May, The Minister of Education will release
  the provincial, board and school results in the
  Ministers Report to Parents.

35
Next Steps

• Print Report for School Administration and
  Teachers and place them in the cum file.
• Print Report for Parents/ Guardians and send them
  home by May 9th.
• Gather classroom based assessment information to
  support LDR writing.
• Commence LDR writing using the electronic
  template.
• Arrange a meeting with parents/guardians of
  students who require an LDR to review results and
  support.
• EXPECTED IMPLEMENTATION DATE Mid May 2008
• Late May / June complete the Student Progress
  and Transition sections of the LDR.
• Print a hard copy of the updated LDR and place it
  in the cum file.

36
http//hrsbstaff.ednet.ns.ca/lsp/
37

• For more information contact
• Emilie Lively
• HRSB literacy department
• elively_at_hrsb.ns.ca
• 464-2000 (4431)