Building Effective Assessments

1
Building Effective Assessments
2
Agenda

• Brief overview of Assess2Know content
  development
• Assessment building pre-planning
• Cognitive factors
• Building the assessment

3
Building Effective Assessments Using
4
Formative Assessment Content

• Assess2Know
• Reading grades 2 11
• Mathematics grades 2 11
• Science grades 3 11
• Social Studies grades 9 11

NEW
5
Formative Assessment Content

• High Quality Content
• Rigorous development process
• Tight alignment with Michigan State Standards
• Highly trained and experienced assessment
  development staff

6
Formative Assessment Content

• Rigorous Requirements
• Clearly and simply written no trick questions
• Appropriate vocabulary for the grade level being
  tested
• Functions independently does not provide
  information that may be used to answer other
  items
• Requires students to read the passage in order to
  answer the question
• Does not use all of the above or none of the
  above
• Presents answer options in a logical order

7
Formative Assessment Content

• Reading
• Passages are commissioned works by well-known
  authors
• Lexiles and Flesch-Kincaid
• Genres
• Blooms Taxonomy, Cognitive Difficulty Levels
• Maps, Charts, Graphs, Images, and Primary Sources

8
Flesch-Kincaid Formula

• FKRS (0.39 x ASL) (11.8 x ASW) - 15.59
• FKRS Flesch-Kincaid Readability Score
• ASL average sentence length in words or average
  number of words in sentence (number of words
  divided by the number of sentences)
• ASW average number of syllables per word (the
  number of syllables divided by the number of
  words)

9
Assess2Know Lexiles

• Lexile measures are considered to be the most
  accurate measure of readability because they are
  the only formula which applies a psychometric
  formula to the assignation for establishing
  readability. 
• Since 2005, Metametrics assigns each A2K passage
  a Certified Lexile Measure.

10
Assess2Know Reading Items

• To determine grade level placement of passages,
  Riverside uses readability guidelines to place
  passages at each grade level. 
• In addition, passage structure, interest level,
  topic, and word count are among some of the
  things used to evaluate where a passage should be
  placed for grade level benchmarking. 
• District level committees review passages and
  items and if they had felt a passage was not
  appropriate, Riverside replaces it and the items
  at the review. 
• Advantage to onsite committee reviews because
  educators in the field are affirming what is
  appropriate for each benchmark.

11
Assess2Know Readability Limits Assess2Know Readability Limits Assess2Know Readability Limits Assess2Know Readability Limits Assess2Know Readability Limits
Grade Flesh-Kincaid Flesh-Kincaid Lexile Lexile
  minimum maximum minimum maximum
3 2.0 4.0 300L 850L
4 3.0 5.0 500L 950L
5 4.0 6.0 650L 1050L
6 5.0 7.0 750L 1075L
7 6.0 8.0 850L 1100L
8 7.0 9.0 950L 1150L
9 8.0 10.0 1000L 1200L
10 9.0 11.0 1050L 1300L
11 10.0 12.0 1100L 1300L
12 11.0 13.0 1100L 1300L
12
Formative Assessment Content

• Mathematics, Science, and Social Studies
• Real world context
• Unique
• Scenario Based Science Items
• Blooms Taxonomy, Difficulty levels
• Use of Tools
• Maps, Charts, Graphs, Images, and Primary Sources

13
Research Efforts

• The Research and Measurement Services Department
  analyzes data from multiple users of the
  Assess2Know Benchmark item bank from across the
  country to monitor the performance of the items
  in the bank.
• Goal is to ensure that the item bank provides
  users with strong quality items with a range of
  difficulty.
• Results look at average ranges for statistics
  such as p-values, discrimination statistics, and
  point-biserials for each item.
• This information is used to plan for refresh and
  future development of our item banks.

14
High Quality Assessment
15
Why do we give assessments?
16
Importance of Designing High Quality Assessments

• Assessments measure what is being taught in the
  classroom
• Align to the district curriculum
• Curriculum Maps
• Scope and Sequence
• Results of the assessment are used to report on
  student achievement
• Used to make decisions about student growth,
  interventions and program placement

17
Characteristics of High Quality Assessments

• Design curriculum based assessments that measure
  student achievement to the state standards
• Report assessment results at the individual
  student, class, school and district level
• Provide achievement information that can be
  compared from year to year
• Construct assessments that are valid and reliable

18
Characteristics of High Quality Assessments

• Assessment design allows for a variety of
  assessment types selected response, open ended
  items, constructed response and rubrics
• Delivered in a timely manner
• Easy to administer

19
Purpose of Assessment

• Evaluate students overall achievement and growth
  in a content domain
• Diagnose students strengths and weaknesses in and
  across content domain
• Plan educational interventions and to design
  individual instruction

20
Purpose of Assessment

• Place students in appropriate educational program
• Select applicants into programs with limited
  enrollment
• Certify individual achievement or qualifications
  from Standards for Educational and Psychological
  Testing

21
What Do We Ask Ourselves Before Building an
Assessment?

• What do we want to measure?
• What information do we want to gather?
• How long do we want the assessment to take?
• How will we determine which standards we want to
  assess?
• How will we use the results?

22
Guiding Principles to Keep in Mind When Building
Formative Assessments

• A minimum of three items for every standard you
  want to measure.
• Spread items measuring an individual standard
  across the assessment.
• Range of overall assessment difficulty and for
  each standard measured.
• Begin assessment with easier items, build to more
  difficult and conclude with easier items.

23
Guiding Principles to Keep in Mind When Building
Formative Assessments

• How many passages will be used for reading
  assessments?
• What item types will be used?
• How many items per passage?
• Total number of items? Can this be completed in
  the allotted time?

24
Guiding Principles to Keep in Mind When Building
Formative Assessments

• Spread items with graphics across test.
• Readability levels for reading passages genres
  needed.
• A minimum of five items per passage.
• Even spread of answer choices across assessment
  when possible spread them out across the form.
• Check items for appropriateness to use online,
  paper, or both.
• Check for clueing

25
Formative Assessment Content
Cognitive Difficulty Levels

• Level 1 Basic Skills
• recall of information such as fact, definition,
  term, or simple one-step procedure.
• Level 2 Conceptual Understanding
• includes the engagement of some cognitive
  processing beyond recalling or reproducing a
  response. A conceptual understanding item
  requires students to make some decisions as to
  how to approach the problem or activity and may
  imply more than a single step.
• Level 3 Extended Reasoning
• requires problem solving, planning, and/or using
  evidence. Items require students to develop a
  strategy to connect and relate ideas in order to
  solve the problem while using multiple steps and
  drawing upon a variety of skills.

26
Importance of Content Complexity

• Vastness of Content
• Alignment
• Validity
• Clarity
• Teacher Guidance
• Truth in Advertising

27
Level One Basic Skills

• Recall of information such as fact, definition,
  term, or simple one-step procedure.
• Support ideas by reference to details in text
• Use dictionary to find meaning
• Identify figurative language in passage
• Solve a one step word problem
• Perform a specified procedure

28
Level 2 Conceptual Understanding

• Includes the engagement of some cognitive
  processing beyond recalling or reproducing a
  response. A conceptual understanding item
  requires students to make some decisions as to
  how to approach the problem or activity and may
  imply more than a single step.
• Predict logical outcome
• Identify and summarize main points
• Represent a situation mathematically in more than
  one way.
• Interpret a visual representation

29
Level 3 Extended Reasoning

• Requires problem solving, planning, and/or using
  evidence. Items require students to develop a
  strategy to connect and relate ideas in order to
  solve the problem while using multiple steps and
  drawing upon a variety of skills.
• Determine effect of authors purpose on text
  elements
• Summarize information from multiple sources
• Provide a mathematical justification
• Describe, compare and contrast solution methods

30
Building Effective Assessments

• Questions