OPEN RESPONSE QUESTIONS Tutorial

1
OPEN RESPONSE QUESTIONS Tutorial

• Developed by
• Gail Rhodes, Fulton County ATC
• Edited by
• Myra Wilson, OCTE, Central Office
• Sheree Koppel, Jefferson County Schools
• Meredith Reed, Eastside Technical Center
• Education Cabinet
• Equal Education and Employment Opportunities M/F/D

2
OPEN RESPONSE QUESTIONS

• Are used for student assessment and
  demonstration of learning.

3
OPEN RESPONSETutorial Outline

• Definition of an Open Response Question (1-7)
• Steps in Preparing an Open Response Question
• Plan the Question. (9)
• Write the Question. (10-24)
• Design the Directions (15-22)
• Scaffold (17)
• Single dimension ((19)
• Two or more relatively independent components
  (19)
• Student choice (19)
• Power Verbs (22)
• Develop a Scoring Guide. (25-30)
• Help! (31-32)

4
AN OPEN RESPONSE QUESTION IS

• a way to assess student progress
• a criterion-based assessment
• a question with a right answer but with more than
  one way to arrive at the answer
• an application of knowledge to real-world
  situation
• a way to see higher-order thinking skills in
  action and
• an example of writing to demonstrate learning.

5
AN OPEN RESPONSE QUESTION IS NOT

• a grammar and/or spelling test
• a norm-referenced test
• task based only on recall skills
• a question with only one way to arrive at the
  answer or
• an assessment of a students writing style.

6
OPEN RESPONSE

• Requires students to combine content knowledge
  and application of process skills in order to
  communicate an answer.

7
OPEN RESPONSE QUESTIONS

• are effective for applying content within a
  real-life context
• Geometry in construction
• Reading skills in recipes or directions
• Math skills in machining
• Scientific skills in information technology
• evaluate knowledge and
• increase interest by actively engaging students.

8
OPEN RESPONSE Steps

• Plan the question (prompt).
• Write the question (prompt).
• Develop a scoring guide.

9
PLANNING THE QUESTION

• Choose a topic that you need to assess to see if
  the students really got it.
• Use open response questions to assess content
  skills within a unit of studynot isolated
  activities.
• Use the curriculum map and/or core content to
  identify target components.
• Review the unit plan to identify target
  components.
• Focus on the big ideas of the unit.

10
PLANNING THE QUESTION (Cont.)

• Make sure the task is actually achievable.
• If the item asks students to read a passage or
  examine a graphic and then give three ways or
  explain two reasons based on the material
  given, make sure there are three ways or two
  reasons found in the material provided.

11
OPEN RESPONSE QUESTIONS

• Set up a situation in which the student applies
  knowledge.

12
WRITING THE QUESTION

• The question should
• reliably measure what you want to evaluate,
• measure what students can do with content
  knowledge, and
• target application of skills and content.

13
WRITING THE QUESTION A Two-Fold Step

• Establish the situation.
• Design the directions.

14
WRITING THE QUESTION Establish the Situation

• Develop the prompt or entry into the question.
• Prepare the student to answer a specific question
  by
• creating interest, and
• providing background information.
• Connect context (real-life) application from unit
  of study.

15
WRITING THE QUESTION Establish the Situation
(Cont.)

• Use entry statements that catch interest.
• Incorporate situations of interest to the
  students.
• Provide reading passages, if used, that are
  interesting.
• Offer appropriate graphics that are clear and
  readable.

16
DESIGN THE DIRECTIONS

• It is important that the question is clear and
  concise, so students know exactly what is
  expected.

17
DESIGN THE DIRECTIONS

• State in the directions if students should
  provide examples and how many.
• If you want the students to identify information,
  do not ask them to discuss, describe, or explain.
• There should be at least twice the number of
  possible answers compared to the number asked of
  students. The possible answers will be in the
  scoring guide.

18
DESIGN THE DIRECTIONS

• Make sure that each item has an item prompt
  (i.e., one or more introductory sentences) that
  precedes the directions. Even if the item has a
  graphic, there should be a prompt that describes
  or provides information related to the graphic
  and/or item directions.

19
DESIGN THE DIRECTIONSFive Types of Open Response
Items

• Choose the appropriate question type for the
  specific focus of the question
• Scaffold questionsmultiple related parts
• Single dimension/componentstraight-forward
  question
• Two or more relatively independent
  componentsmultiple, nonrelated parts
• Student choice from topics (usually an on-demand
  question)
• Response to provided information

20
DESIGN THE DIRECTIONS Scaffold Questions

• Have multiple parts of the question or
  directions.
• Have parts presented and labeled separately (a,
  b, c).
• Are order-arranged so that successive questions
  depend on the response of the previous answer.
• Become increasingly more difficult or complex.

21
DESIGN THE DIRECTIONS Scaffold Questions Math
Example

• Jim was investigating information about natural
  wonders of the world.
• He found that Mt. Everest is the highest mountain
  in the world at 29,028 feet above sea level.
• He found that the Marianas Trench in the Pacific
  Ocean is the lowest point of Earth at 35,840 feet
  below sea level.

22
DESIGN THE DIRECTIONS Scaffold Questions Math
Example (Cont.)

• If Jim could throw a rock from the top of Mt.
  Everest to the bottom of the Marianas Trench, how
  many feet would it fall?
• Draw a diagram and explain your answer for part A.

23
DESIGN THE DIRECTIONS Single Dimension/Component

• Is a straight-forward question.
• Requires explanation, examples, description, or
  evidence of support.

24
DESIGN THE DIRECTIONS Single Dimension/ComponentS
ocial Studies Example

• Many Kentucky cities are located near large
  rivers.
• Describe three important advantages that the
  rivers provide these cities.
• Explain why each advantage is important.

25
DESIGN THE DIRECTIONS Two or More Relatively
Independent Components

• Signaled by a, b, c.
• Parts about the same prompt have little
  relationship to each other.
• Correct response to one question is not dependent
  upon the response of other questions.

26
DESIGN THE DIRECTIONS Two or More Relatively
Independent Components Practical Living Example

• Bob has been told that regular physical activity
  is important for good health throughout a
  lifetime.
• Describe THREE ways physical activity helps the
  body to be healthy throughout a lifetime.
• Give THREE physical activities Bob could do that
  would help keep him healthy throughout his
  lifetime.

27
DESIGN THE DIRECTIONS Student Choice (seldom used)

• Forces students to choose from provided topics.
• Offers more opportunities to demonstrate
  individual learning.
• May be more difficult to score because there are
  more correct answers.

28
DESIGN THE DIRECTIONS Student Choice Science
Example

• Answer either A or B using some of the Earths
  materials listed below
• Soil
• Water
• Gases of the atmosphere
• Choose TWO materials from the list above.
  Explain how a PLANT uses these materials to live.
• Choose TWO materials from the list above.
  Explain how an ANIMAL uses these materials to
  live.

29
DESIGN THE DIRECTIONS Response to Provided
Information

• Respond to provided information, such as data,
  readings, or graphs.

30
DESIGN THE DIRECTIONS Response to Provided
Information Reading Example

• In the story, First Light, Matthew woke up in
  another time period--the 1850s.
• Describe FOUR things Mathew discovered that were
  different from what he was used to in his present
  life.
• Explain how each of those differences affected
  him. Use information from the story to support
  your answer.

31
DESIGN THE DIRECTIONS Designing the Question

• Choose the level of thinking you want to be
  reflected from the question
• Recall to synthesis and evaluation.
• Use skills of application or analysis.
• Use power verbs.

32
DESIGN THE DIRECTIONS Designing the
QuestionPower Verbs

• Explain
• List
• Compare
• Contrast
• Describe
• Tell

• Identify
• Criticize
• Discuss
• Define
• Analyze

33
WRITING THE QUESTIONSummary

• Carefully define what you want students to
  answer
• Be clear on both quality quantity indicators
  the scoring guide may need to be written before
  creating the question.
• Use simple and direct language.
• Use language both age- and grade-level
  appropriate.

34
WRITING THE QUESTION Ask Yourself

• Will my students know exactly what I am asking
  them to do?
• Is this the most effective way to phrase this
  question?
• Have I used the strongest verb?
• Is the question specific in quantities needed for
  a quality response?
• Is the language appropriate for my students?

35
DEVELOPING A SCORING GUIDE (RUBRIC)

• A rubric is sets of criteria which describe the
  characteristics of responses at each identified
  level.

36
SCORING GUIDE

• Is a necessary tool to accurately and fairly
  evaluate student success.
• Should be written prior to administering the
  activity.
• Sometimes indicates problems with the question.
• Ensures the question is rich enough to support
  various levels of student responses.

37
SCORING GUIDE (Cont.)

• Includes a clear explanation of what is expected
  in a quality response.
• Defines the various levels of possible responses
  and places a value on each.
• Enables scoring to be unbiased, consistent, and
  accurate.
• Uses simple, straight-forward language and
  repeats significant words used in the question.
• Ensures what is required is clearly indicated.

38
SCORING GUIDE Scoring Levels

• Kentuckys general scoring guide uses four
  levels.
• Fewer than four may be used.
• The number of levels is dictated by how narrow
  you want to define the characteristics of the
  response.
• The goal is to meet the needs and objectives of
  your course.

39
SCORING GUIDEDescriptors of Level Responses

• Top (Level 4)effective, thorough, complete,
  successful, insightful, in-depth, efficient
• Level 3adequate, satisfactory, understanding of
  major concepts, mostly complete, etc.
• Level 2having gaps or leaps, incomplete, some
  important points, basic understanding
• Level 1minimal, only small part, little
  understanding, not logical, unclear, major errors

40
REMEMBER

• If you cannot easily write a response of
  top-level quality, you are probably expecting too
  much of your students.

41
OPEN RESPONSEHelp! Websites

• KDE Open Response Planning Guide
• KDE Released Items
• KDE Core Content
• KDE Learning Goals and Academic Expectations
• KDE Suggestions for Preparing Students for Open
  Response Questions
• KDE Teachers Writing Handbook

42
OPEN RESPONSEHelp! Staff

• Call your district or high school contact.
• Cherry Boyles, KDE, Writing Team, 564-2106,
  cboyles_at_kde.state.ky.us.
• Myra Wilson, Department for Technical Education,
  502-564-4286, MyraH.Wilson_at_ky.gov.

43
OPEN RESPONSE

• Happy writing!