Scale Scoring

1
Scale Scoring

• A New Format for
• Provincial Assessment Reports

2
http//www.ela.ednet.ns.ca
3
Agenda

• 900 - 1015 Introduction to Scaled Scores
• 1015 -1030 Break
• 1030 -1200 Scaled Scores continued
• LSP website
• Using the Results
• 1200 -100 Lunch
• 100 - 300 LSP writing

4

• Beginning in the 2007-2008 school year, all
• provincial assessments will be reported on a
• common scale.

5
Why Use a Common Scale?
Imagine trying to accurately compare a students
or a schools achievement when given the
following raw scores

• letter grades
• numerical marks
• rubric levels
• percentage grades

6
What is a Scale?

• A scale is an arbitrarily established set of
  numbers used for measurement.
• Different scales may be used to measure the same
  thing, but they are calibrated differently. (e.g.
  rulers, thermometers, speedometers, bathroom
  scales)

7
What is a Scale?

• A scale, simply put, is a mathematical
• conversion of raw scores to a
• common scale or a derived score.

8

• A scaled score is a conversion of a student's raw
• score on a test to a common scale that allows for
  a
• numerical comparison between students.
• Provincial assessments use multiple versions of a
  test
• over the years. The scale is used to control
  slight
• variations from one version of a test to the
  next.
• Scaled scores, when equated, are particularly
  useful
• for comparing test scores over time.

9
Equating A Statistical Process

• Equating measures the difficulty of each
  assessment (content,
• cognition, difficulty) and adjusts the cut score
  to
• account for differences among assessment forms.
• The difficulty level of tests in different years
  can be
• equated and this enables us to compare assessment
• results from year to year.

10
Advantages of Scale Scores

• Scaled scores allow better comparison of
  assessment
• results
• from year to year
• from one grade level assessment to another
• of student achievement in different subject areas
  

11
Future Advantages

• Because each successive years assessments are
  statistically equated with a baseline reference
  year, the scaled scores can also be used to
  reflect growth.
• For example, if a future scaled score is higher
  than the reference year scale score (2007-2008),
  it can be interpreted as an improvement from that
  base year.

12
Cautions for Comparisons

• Many factors impact upon a schools results.
• No single assessment result tells the whole
  story of a students or schools achievement.
• Sample size does matter! Schools with fewer
  students will show more variability in scores
  from one year to the next.

13
Scale Scoring
Provincial Mean
Cut score

One standard deviation above
65 of students who wrote this
assessment
One standard deviation below
384
600
300
800
700
400
200
500
Scale Scores
14
Things to Consider

• Distribution of Scores
• Scale ranges from 200 to 800
• Provincial mean is set to 500
• Provincial standard deviation is set to 100
• On most provincial assessments, about 65 of
  students will place between 400 and 600

15
(No Transcript)
16
Cut Score

• A cut score defines the point at which a
• certain required level of achievement has been
• demonstrated on the assessment.
• Cut scores are set by a panel of educators
• who look at the actual test questions to
• determine levels of acceptable performance.
• A cut score can be above or below the mean.

17
Cut Score

• A cut score differentiates between students who
  are
• meeting expectations
• or
• not yet meeting expectations.

18
Things to Consider

• The cut score for the ELA was set at 384
• Higher scores mean stronger performance relative
  to
• other students who wrote the same assessment.

19
Interpreting Results Overall Score

• Cut Score
• How close or how far away from the cut score
  (384)
• did the student perform?
• Did the student meet expectations for this
  assessment or has
• the student not yet met expectations for this
  assessment?
• REMEMBER The cut score only applies to the
  overall score
• (section 1) the student received.

20
Using the Results Overall Score

• Students Meeting Expectations (385 and above)
• If the overall score is above the cut score (384)
  the student is
• meeting expectations.
• Benchmarks have been established to help
  determine the
• level of performance.
• If a students overall score falls between 385 -
  600, his/her
• performance is considered to be satisfactory.
• If a students overall score falls between 601-
  800, his/her
• performance is considered to be stronger.

21
Using the Results Overall Score

• Students Not Yet Meeting Expectations
• (Overall Score below 384)
• For every student who is not yet meeting
  expectations (overall score is
• below 384) teachers will be expected to
• provide on-going, in-class support
• contact parents/guardians to discuss support
  options

22
Using the Results Overall Score

• Students Not Yet Meeting Expectations (Overall
  Score below 384)
• If a students overall score falls below 384,
  his/her score is more than one standard
• deviation from the mean. Therefore, he/she will
  require the most support and formal
• tracking of that support.
• For every student who is not yet meeting
  expectations and whose score is below 384,
• the teacher is expected to
• Develop a Literacy Support (LS) document
• Support student within the class structure
• The LS document will be on-going for students,
  with transition meetings at the end of each
  grade.

23
(No Transcript)
24
Interpreting ResultsOverall Score

• Provincial Mean (500)
• How close or how far away from the mean did the
  student perform?
• How did the student perform relative to others
  that wrote the assessment?
• How does the students performance relative to
  others compare from subject to subject?
• How did the student perform this time compared
  to the time before?

25
Using the Results

• The Reading Performance section
• Provides information on student performance in
  the various genres, cognitive levels and some
  specific outcomes .
• Allows for comparison to the provincial mean of
  500.
• May help teachers isolate concerns and
• develop strategies to address these indicators.
• was not factored into a students overall score
• is intended to provide to teachers information
• for planning purposes only.

26
(No Transcript)
27
Using the Results

• The Writing Performance section
• Provides information on student performance in
  the elements of writing ideas, organization,
  matters of correctness
• May help teachers identify concerns and trends.

28
Using the Results

• Schools are responsible for tracking Literacy
  Support (LS) and for monitoring support.
• Teachers and Principals are expected to
• review LS document and monitor support
• ensure parent/guardian signatures on the LS
  document
• ensure that the completed LS document is kept in
  the students cum file

29
Using Your Data Classroom Level

• Teachers are encouraged to use the student
  reports to identify trends and patterns in
  literacy development.
• Class / grade profiles may be used for
  classroom instructional decision making and for
  targeting support to individual students.

30
Using the Results School Level

• In April, The Minister of Education will release
  the provincial, board and school results in the
  Ministers Report to Parents.

31
Using the Results School Level
32
Next Steps

• Principals have been asked to verify with the DOE
  that the information sent to them is accurate.
• NOTE Assessment information is now sent to
  principals electronically via the Student Records
  website. The blue forms will no longer be sent
  to schools.
• Print Report for School Administration and
  Teachers. Put results for students who do not
  meet expectations in the blue pocket folders and
  place them in the cum file.

33

• Parent Reports will be sent to schools once
  verification with the DOE is complete. Please do
  not send home the school reports.
• Gather classroom based assessment information to
  support LSP writing.
• Commence LSP writing using the electronic
  template.
• IMPLEMENTATION DEADLINE APRIL 30, 2008

34

• In May, there will be a random Audit of LSPs at
  the grade 6 level.
• Late May / June complete the Student Progress
  and Transition sections of the LSP and meet with
  Junior High teachers.
• Print a hard copy of the updated LSP and place it
  in the blue pocket folder in the cum file.
• Forward the electronic templates to the Junior
  High school.

35
http//hrsbstaff.ednet.ns.ca/lsp/
36

• Lets walk through
• the set up of
• a Literacy
• Support Plan

37
www.hrsbstaff.ednet.ns.ca/lsp
38
For more information

• Program of Learning Assessment for Nova Scotia
  (PLANS), Nova Scotia Department of Education,
  Provincial Assessment Reports A New Format
  (2007)
• http//www.ela.ednet.ns.ca