Testing%20and%20Grading

1
Testing and Grading

• Ensuring that we meet the objectives

2
BA 9200 Course Overview
Teaching Philosophy
Students Abilities
Course Level and Scope
Desired Learning Levels
Write Course and Topic Objectives
Select Effective Teaching Methods
How Students Learn
The Lecture
Testing and Grading
Active Learning
3
Testing Overview
A. Preparing Exams Type of Tests Number of Tests Questions per Test, and Table of Specification
B. Writing Questions Effective Stems and Distracters Structured Essay Questions and Answers
C. Grading Scoring Key and Partial Credit for Multiple Choice
D. Improving Exams Survey and Difficulty and Discrimination Indices
4
A. Preparing Exams1. Type of Tests
Norm Based Criterion Based
Goal Sort and Classify Students Assess Student Competency
Scores Must Have Large Spread in Scores Spread in Scores Not Relevant
Difficulty Questions Must Range from Easy to Difficult Questions Must Reflect Demands of Profession
Timing All Students May Not Finish Exam All (or Most) Should Finish Exam
5
A. Preparing Exams 2. Number of Tests

• Not Effective One Exam at the end
• Effective Unit Tests/Quizzes followed by a final
  exam.

6
A. Preparing Exams 3. Criterion-Based Tests
Number of Questions per Test
Rough guideline for time per question at each
level.
Rote Multiple choice Short Answer 2 minutes 4 minutes
MI Level Multiple Choice Short Answer 4 minutes 8-10 minutes
Critical Thinking Multiple Choice Short Answer 10-15 minutes 15-30 minutes
7
A. Preparing Exams 4. Table of Specification
Rote Exercise MI
Measures of Center 15 25
Measures of Spread 10 15 10
Shape of Distribution 10 15
8
Testing Overview
A. Preparing Exams Type of Tests Number of Tests Questions per Test, and Table of Specification
B. Writing Questions Effective Stems and Distracters Structured Essay Questions and Answers
C. Grading Scoring Key and Partial Credit for Multiple Choice
D. Improving Exams Survey and Difficulty and Discrimination Indices
9
B. Writing Questions1. Multiple Choice Questions

• Essay Questions
• a. Require the Student to Remember Information.
• b. Are Scored More Reliably.
• c. Are More Difficult to Construct.
• d. Are Easier to Construct.
• The Median is
• a. the Average Value.
• b. the 25 Percentile Value.
• c. the 75 Percentile Value.
• d. that Number such that Approximately 50 of
  the Numbers Are Higher and 50 are Lower than
  the Median Value.

10
B. Writing Questions Multiple Choice Questions

• The Term Used in Connection with Sound that
  Corresponds to Color with Reference to Light
  is
• a. Pitch.
• b. Hue.
• c. Timbre.
• d. Intensity.
• One Basic Advantage of Using Short Answer Essay
  Questions as Opposed to Long Answer Ones is that
  Short Answer Essay Items
• a. Generating Questions is Easier.
• b. Allow for a Broader Sample of Questions.
• c. Students Prefer Them.
• d. They Relate More Closely to the Course
  Objectives.

11
B. Writing Questions Writing Effective Stems

• Ideal Stem Answerable Without Seeing Choices.
• Questions Address Important Concepts.
• Relate Each Question to One or More Topic
  Objectives.
• Question Stem Followed by Cognitive Objectives
  that Question AddressesO12, O15
• Cite Authority in Stem where applicaple

12
B. Writing Questions Writing Effective Stems

• Solve Math Problems to Ensure All Data in Stem.
• State Stem in Positive Language.
• Same Phrase Appears in Each Answer, Shift It to
  Stem.
• Use Complete Sentence Format.

13
B. Writing Questions Writing Effective
Distractors

• Use Best-Answer Format.
• Write Best or Correct Answer First.
• For Concept Question
• Correct Answer to Similar Concept.
• Have Correct Elements, But Also Incorrect
  Elements
• From Students' Answers to Essay Version of
  Question.

14
B. Writing Questions Writing Effective
Distractors

• For Math Question
• Reflect Mistakes That Students Make.
• Use "Not Sufficient Information to Solve Problem"
  Option.
• Avoid None of Above for Math Problems

15
B. Writing Questions Writing Effective
Distracters

• Correct Answer Should Not Be Longest Answer.
• All Distractors Should Form Complete and
  Grammatically-Correct Sentences With Stem.
• Avoid Using Never or Always.
• Avoid an All of the Above Option.
• Avoid Repeating Words From Stem.
• Vary Correct Answer Among Four Possible Answer
  Positions.

16
B. Writing Questions 2. Writing Short-Answer
Questions

• Base on Objectives.
• Use Structured Essay Format.
• Uses Precise Verbs.
• Helps Students Frame Answer.
• Use Iterative Procedure.
• Write Ideal Answer. Could Students Achieve Ideal
  Given Question Wording?
• Questions Address Important Concepts.
• Solve Math Problems.

17
B. Writing Questions Unstructured Short-Answer
Questions

• Compare and Contrast Amadeus and The Shawshank
  Redemption.
• John Locke The Key to David Hume? Discuss.
• Is the Central Limit Theorem Critical to
  Constructing a Confidence Interval on a
  Population Mean? Discuss and Defend.

18
B. Writing Questions The All-Bluff Answer and
the Need for the Structured Format

• Is Central Limit Theorem Critical to Constructing
  a Confidence Interval on a Population Mean?
• Dr. Deming Noted that the Central Limit Theorem
  is Essential.
• As You Said in Class, The Central Limit Theorem
  is a Critical Topic.
• The Central Limit Theorem Says that ..
• Finally, Hypothesis Testing is Similar to
  Confidence Intervals in that ...

19
B. Writing Questions Structured Answer Format

• Compare and Contrast Amadeus and The Shawshank
  Redemption.
• Basis 1 is _________
• Basis 1 is Important Because _________
• Basis 2 is __________
• Basis 2 is Important Because __________

20
B. Writing Questions Structured Answer Format

• Students then Complete the Table.

21
B. Writing Questions Structured Answer Format
Can the Bias in a forecast ever be greater than
the MAD value? O15 10 Points Select One
Response Below and Complete.
Yes Because No Because
22
B. Writing Questions Structured Answer Format
Suppose the Central Limit Theorem was not
true. Would It be Possible to Construct a
Confidence Interval on a Population Mean? O23
15 Points Select One of the Responses Below
and Complete. Would Never be Possible
Because Would Always be Possible Because Would
Only be Possible When
23
B. Writing Questions Structured Answer Format
Suggest a modification to the analogy method that
would increase the likelihood that a
Sensing-dominant manager would use the Method.
O45, O56 25 Points Describe the
Modification. This Modification Increases the
Likelihood that a Sensing-Dominant Manager Would
Use the Analogy Method Because
24
Testing Overview
A. Preparing Exams Type of Tests Number of Tests Questions per Test, and Table of Specification
B. Writing Questions Effective Stems and Distracters Structured Essay Questions and Answers
C. Grading Scoring Key and Partial Credit for Multiple Choice
D. Improving Exams Survey and Difficulty and Discrimination Indices
25
C. GradingGrading Issues for All Types of Exams

• Written Comments Are Highly Motivating.
• Excellent Improvement
• Sorry Did So Poorly. Please See Me.
• Return Within One Class (MC) or One Week (Short
  Answer).
• Provide Key After Quiz, if Possible.

26
C. GradingGrading Issues on Multiple Choice Exams

• Partial Credit for Multiple Choice?
• On Numerical Problems Provide Optional White
  Space and Only Check if Wrong Option Selected.
• Defend Concept Answer to Complex Question
  Provide Optional White Space and Only Check if
  Wrong Option Selected.

27
C. GradingGrading Issues on Short Answer Exams

• For Small Classes, Provide Audio Feedback.
• Students Hand In Audio Cassette to Instructor.
• Write Annotated Comments on Exam.
• Use a Detailed Scoring Key.
• Grade One Question at a Time.
• Shuffle Papers After Each Question.

28
Testing Overview
A. Preparing Exams Type of Tests Number of Tests Questions per Test, and Table of Specification
B. Writing Questions Effective Stems and Distracters Structured Essay Questions and Answers
C. Grading Scoring Key and Partial Credit for Multiple Choice
D. Improving Exams Survey and Difficulty and Discrimination Indices
29
D. Improving Exams

• Use Test Assessment Instrument During Term.
• 1. Have exams reflected topic objectives?
• 2. Have exams reflected important subject
  content?
• 3. Has there been sufficient time to complete
  exams?
• 4. Has instructor reacted positively to student
  concerns on questions, grading,
  etc.? __________________________________________
• 1 2 3 4 5
• Not at All Moderately Very Much
• 5. How could the exams be improved?

30
Additional Readings

• Bloom, B., Hastings, T., and Madaus, G. Handbook
  of Formative and Summative Evaluation for Student
  Learning. New York McGraw Hill, 1971.
• Jacobs, L and Chase, C. Developing and Using
  Tests Effectively A Guide for Faculty. San
  Francisco, CA. Jossey-Bass, 1992.

31
BA 9200 Course Overview
Teaching Philosophy
Students Abilities
Course Level and Scope
Desired Learning Levels
Write Course and Topic Objectives
Select Effective Teaching Methods
How Students Learn
The Lecture
Testing and Grading
Active Learning