Introduction to NormReferenced Assessment

1
Introduction to Norm-Referenced Assessment

• Assessment in Special Education
• Kent State University
• Erich Merkle, M.A., M.Ed.

2
Multi-Method Pillars of Assessment

• Multi-method approach to assessment
• Multiple Sources student, family, teachers
• Multiple Areas intelligence, achievement,
  vision, hearing, motor, language, memory,
  adaptation, social/emotional, etc.
• Multiple Means tests, interviews, observations,
  informal procedures.
• Sattlers Four Pillars of Assessment
• Norm-referenced tests
• Interviews
• Observations
• Informal assessment procedures

3
Norm-Referenced Tests

• Designed to provide professional with capacity to
  compare performance of one client/student to
  others nationwide at same age/grade level.
• Since cannot administer test to everyone, tests
  are administered to representative samples
• Interpolation is used when data sets from sample
  are incomplete

4
Construction

• Multi-step process
• Domain, theoretical basis defined
• Exploration of item pool
• Developmental version of test or subtests
• Research on developmental versions analyzed
• Developmental version changed based on results
• Standardization version prepared
• Sampling procedures determine how/when sample
  will be recruited
• Standardization research test administered at
  test sites
• Data collected, analyzed for establishing
  psychometric properties
• Test prepared for final version w/packaging,
  protocols, manual
• Test now available for purchase

5
Basic Steps in Test Administration

• Read and fully understand test manual contains
  critical instructions for administration and
  normative tables
• Review the tests protocol response sheet or
  record form used by the examiner to record the
  students answers
• Experiment and learn the various subtests and
  nuances of the test materials before
  administering the test
• Legal ethical standard is typically
  administering a test 25 times to demonstrate your
  ability to use an instrument

6
Examiner Skills

• Boy Scout!
• Be prepared
• Standardization without this, your results are
  invalid!
• Confidentiality
• Flexibility
• Awareness of Individual Child
• Controversy - Arbitrary cutoffs, demographic
  differences, labeling

7
Examiner Skills Part 2

• Vigilance
• Self-awareness
• Eye contact, body language, appearance
• Expectancy effects (e.g. halo effect)
• Empathy, not sympathy

8
Norm-Referenced Assessment Rules

• Minimal distractions
• Control your materials
• Make sure student is comfortable
• Adhere to test instructions exactly if you want
  to use norm data
• Avoid cues (facial, tone of voice) that let
  student know how s/he is doing.

9
Norm-Referenced Assessment Rules Part 2

• Do not tell student if responses are correct or
  incorrect, unless the instructions require it.
• Phrase requests as mild commands - dont ask
  questions that provide an option when you need a
  response.
• Dont tell a student how much time s/he has.

10
Norm-Referenced Assessment Rules Part 3

• I DK ? Probe for clarification
• Avoid giving the impression that youve stopped a
  test due to repeated failures.
• Document everything! (Q, DK, NR)
• Be able to score on the fly
• Testing the Limits

11
Calculating Chronological Age

• Many tests require the correct calculation of
  chronological age to determine starting points
  and to consult appropriate norm tables
• Calculated by writing the test date first then
  subtracting the date of birth
• Dates are written in order of year, month, and
  day.
• Remember each column represents a different
  numerical system days 30, months 12
• Test Yourself
• A child born on August 13, 1980 is how old today?
• A child born on December 25, 1995 is how old
  today?

12
Raw Scores Basal/Ceilings

• First score usually obtained in a test,
  represents the number of items counted as correct
• Usually indicated as 1 or 0
• Starting and stopping points on a test are
  typically determined by basal and ceiling rules
• Basal Thought to represent the point where
  student would answer all previous items correctly
• Ceiling Point where student would answer all
  subsequent items incorrectly

13
Basal Considerations

• Basals prevent the student from taking more items
  than necessary allows student to feel success
  for answering some items correct
• Generally expressed as one of the following
• _____ consecutive correct OR
• _____ out of _____ correct

14
Ceiling Considerations

• Ceilings prevent the child from experiencing too
  much failure and responding to items beyond skill
  level
• Generally expressed as one of the following
• _____ consecutive incorrect OR
• Fails _____ of _____ attemps

15
Derived Scores

• Raw scores in themselves are not meaningful
• Missing 3 items on a math tests doesnt say much
  about a childs performance to others nationwide
  at his/her grade or age level
• Derived scores allow a tests results to be
  understood regardless of the test itself because
  they are universally ascribed quantitative and
  qualitative desciptions
• Derived scores allow a tests results to be
  compared to other tests in other domains,
  assuming they provide the same derived scores
• Raw scores on any norm-referenced test are
  converted to various standard scores through the
  use of a norm table and manual, based upon
  chronological age or grade calculations
• Important to determine which standard/derived
  scores are most helpful in explaining the student
  as a learner

16
Derived Scores Percentile Rank

• Percentile Ranks
• Compare an individuals result relative to the
  standardization sample.
• A point at which a certain percentage of scores
  are at or below.
• They are ordinal - cant assume same degree of
  separation.
• Writing Example Harvey, age 10-3, scored at the
  32nd percentile (ile) on Test X when compared
  with age (grade) peers. This means that Harveys
  performance was equal to or better than 32 of
  the 10-year-old students in Test Xs norm group.

17
Derived Scores Stanines

• Divides the normal curve into 9 parts and produce
  a single digit score
• Ordinal, not equal intervals
• 4th,5th, and 6th stanines are in normal range
• 1st, 2nd, 3rd, are below average
• 7th, 8th, and 9th are above average

18
Derived Scores Standard Scores

• Raw score has been transferred to this score for
  comparative purposes. Includes
• Operates on the interval scale
• z-scores x0, sd1
• Normal-Curve Equivalents (x100, sd21.06)
• Deviation Scores x100, sd15
• Scaled Scores x10, sd3
• T-scores x50, sd10
• Mean and standard deviation are always known
• Writing Example
• Barbara, a 7th grader, received a SS of 115 on
  Test X when compared with grade peers. Because
  the mean standard score on this test is 100 and
  the standard deviation is 15, Barbaras scored
  one S.D. above the mean. This indicates that she
  scored in the high average range.

19
Derived Scores Standard Scores (2)

• Each test gives specific qualitative descriptions
  for a given standard score value, however all
  standard scores can be generically interpreted
• Very Low SS lt73, PR 1-4, T-Score lt32, ScS 1-4
• Low SS 74 to 81, PR 4-11, T-Score 33-37, ScS
  5-6
• Below Average SS 82 to 88, PR 11-23, T-Score
  38-42, ScS 7
• Low Average SS 89 to 96, PR 23-40, T-Score
  43-47, ScS 8-9
• Average SS 97 to 103, PR 40-60, T-Score 48-52,
  ScS 10
• High Average SS 104-111, PR 60-77, T-Score
  53-57, ScS 11-12
• Above Average SS 112-118, PR 77-89, T-Score
  58-62, ScS 13
• High SS 119-126, PR 89-96, T-Score 63-67, ScS
  14-15
• Very High SS gt127, PR 96, T-Score 68, ScS
  16-19

20
Derived Scores Age/Grade Equivalents

• The score is the average score obtained by
  individuals in the sample of a specific age or
  grade.
• Age scores - years and months
• Grade scores - tenths of a grade
• The scores do not necessarily represent equal
  units - in many cases, ordinal.

21
Derived Scores Age/Grade Equivalents (2)

• They encourage inappropriate comparison.
• They mean different things on different tests.
• Have little meaning for older students with core
  skills that peaked at an earlier age.
• Exaggerate small differences.
• Writing Example
• John, a 7 year-old second-grader, earned a grade
  equivalent score or 6.3 on Test X. This many mean
  that Johns raw score was equal to or
  approximated the average raw score of grade 6.3
  students in the norm group.

22
Derived Scores Conversions

• All derived scores can be converted between one
  another may help in comparing various test
  scores across different instruments or providing
  a unified standard of comparison
• For example a Deviation Score of 115 is equal to
  a T-Score of 60, and a z-score of 1.0
• Desired Score (new SD)(z score) (new mean)

23
Group Norm-Referenced Testing

• Group achievement tests are increasingly used to
  assess accountability of individual students and
  school systems (esp. with NCLB 01)
• High stakes tests are used by at least 26 states
  to determine if a child can graduate, and at
  least 6 use for grade promotion
• IDEA 97 requires that students with disabilities
  are included in statewide and district-wide
  assessments

24
Group Norm-Referenced Testing (2)

• Group assessments under IDEA 97 may be given
  with accommodations
• Changes in format, response mode, setting, or
  scheduling that will enable a child with
  disabilities to complete general curriculum or
  test.
• Determined by IEP team
• May include assistive technology or alternative
  assessment
• High stakes testing is controversial for students
  with disabilities comparing students who differ
  from norms with typically developing student
  norms may be inappropriate?