An Introduction to Test Construction

1
An Introduction to Test Construction

• By the end of this module, you will
• Know what should be the basis of all exam
  questions.
• Be able to connect assessment methods with your
  goals for students learning.
• Know how to write effective multiple choice
  questions.
• Know how to write and grade essays.

2
Learning Outcomes The basis of all exam
questions

• The basis of all exam questions should
• Be learning outcomes.
• Using Blooms Taxomony can help you
• clarify what you want students to
• learn.
• (Be sure to close the word document after reading
  it to return to this screen)

3
Learning Outcomes Using Blooms Taxonomy

• By setting learning outcomes at
• different levels of Blooms
• Taxonomy, you will target exactly
• what you want students to know,
• be able to do, or have experienced
• as a result of learning activities in
• class.

4
An Example of Using Blooms Taxonomy

• Heres an example. Suppose you want
• your students to be able tell you the
• causes that led up to the Civil War in the
• U.S.
• If you want them to tell you in their own
• words, they are at the Comprehension
• level of the taxonomy.

5
Another Example of Using Blooms Taxonomy

• You might want your students to tell you
• which were the causes of the Civil War
• that had the most negative economic
• impact on both North and South based
• on criteria they derive from original
• documents.
• This learning outcome is at the
• Evaluation level of the taxonomy.

6
Using a Test Blueprint is Effective

• Using Blooms Taxonomy, you can make
• a test blueprint that helps you tie
• concepts to the level of questions you
• should ask.

Click here to see an example of a test blueprint
for a statistics class.
7
Writing Effective Multiple Choice Questions

• Many tests are multiple choice because
• those are fairly quick and simple to
• grade.
• BUT, a good multiple choice question is
• very difficult to write.
• Why?

8
Multiple Choice Exam Questions are Difficult to
Write

• The most common mistakes you can
• make when writing multiple choice
• questions are
• Asking vague questions
• Asking two questions at once
• Asking questions only at the Knowledge level
• Making more than one plausible answer

9
Asking Vague Questions

• Heres a typical vague multiple choice question
• Which of the following was the primary
• economic problems from the last recession?
• Job losses
• Strict monetary policy
• Weakness of the dollar
• Consumer confidence loss

10
Fixing Vague Questions

• Why was that a vague question?
• When was the last recession?
• Was there just one primary problem?
• Where was the recession? In the U.S. or
  elsewhere?
• QuizAt which level on Blooms Taxonomy is this
  question?

11
Fixing Vague Questions

• Answer At the knowledge level IF you or
• the text specifically said this
• information. At the comprehension level
• if the student had to make a conclusion.
• How could you make the question less
• vague?

12
Fixing Vague Questions

• According to class lecture, the economic
• problem that contributed most to the
• recession in the U.S. in 2001 was
• Job losses
• Strict monetary policy
• Weakness of the dollar
• Consumer confidence loss

13
Asking Two Questions at Once

• Sometimes a test writer asks two questions at
• once. This confuses the student. Heres an
• example
• When exercising, what is the initial source of
• energy and where is it stored?
• Glycogen
• Fat
• Carbohydrates
• Gatorade

14
Asking Two Questions at Once

• You COULD put the answers to both questions
• in your answer list, but it is still confusing
  for
• the students
• When exercising, what is the initial source of
• energy and where is it stored?
• Glycogen, muscle
• Fat, thighs
• Carbohydrates, muscle
• Gatorade, stomach

15
Asking Two Questions at Once

• A better solution would be to ask two
• separate questions
• When exercising, what is the initial
• source of energy?
• Where is the initial source of energy
• used when exercising stored?

16
Questions at the Knowledge Level

• It is easiest to write multiple choice
• questions at the knowledge level
• because you can take a fact and convert
• into a question.
• BUT, most instructors want to test at higher
• levels of learning than just memory.
• What can you do?

17
Questions at Other Levels

• To write multiple choice questions at
• higher levels, you can
• Have students read a passage and answer questions
  requiring them to comprehend, analyze, or
  synthesize information (remember the SAT?)
• Set up a hypothetical situation and ask students
  to answer several questions about it.

18
Example Question at a Higher Level

• Wilma and Fred were arguing over the furniture
• while they were settling their divorce
• agreement. Wilma said to Fred, I never liked
• that chair anyway. At which of Knapps
• relational conversation levels is this
• statement?
• Integrating
• Differentiating
• Sustaining
• Initiating

19
Example Question at a Higher Level

• This question was at the Application
• level.
• Why?
• Because it required a student to
• understand the relationship levels and
• apply a statement to that typology.

20
Making More than Two Plausible Answers

• One of the worst mistakes in writing
• multiple choice questions is having more
• than one plausible answer.
• Students will continue to try to get
• points from instructors who have more than one
• plausible answer to questions.
• How do you avoid this problem?

21
Making More than Two Plausible Answers

• When writing answers, have one that is clearly
• correct. The others need to sound plausible
• but definitely be WRONG.
• Sometimes faculty try to trick students with
• answers that seem very similar to correct ones.
• Dont fall into that trap or you will have
• students arguing with you.
• Make sure all wrong answers are defensible
• and have real reasons for being wrong.

22
Essay Questions

• While they are easier to write, essay
• questions are not easier to grade than
• multiple choice questions.
• When should you use essay questions?
• When you want to test at the analysis,
• synthesis, and evaluation levels of the
• taxonomy.

23
Mistakes in Writing Essay Questions

• The main mistakes instructors make
• when writing essay questions are
• Writing vague, non-specific questions.
• Asking for too many parts in an answer.
• Asking questions over material that
• students did not read or cover in class.
• Asking questions that are difficult to grade
• (e.g. asking for students opinions).

24
Tips for Writing an Effective Essay Question

• To write an effective essay question
• Decide exactly what you want to know about the
  students learning. Do you want to know that
  they can tell you how a complex economic system
  forms? If so, ask that.
• Cue students to the material from texts and class
  that forms the basis of your questions.

25
Tips for Writing an Effective Essay Question

• Put points in parentheses after each component of
  the question so students will know exactly which
  parts to answer and how much those parts will
  count.
• Use key words so students will understand exactly
  how you want them to answer.
• Do you want to see an example?

26
An Effective Essay Question

• You are going to design a quantitative research
• project to look at students motivation in
• classrooms. You want to study ways their past
• experiences, current expectations, and self-
• efficacy influence their study behaviors this
• semester. Using what you learned from your
• textbook, list your independent and dependent
• variables (2 points each) and explain for each
• how you would operationalize that variable (5
• points each). Finally, discuss at least two ways
  
• you would analyze the data that you collect (10
• points for each way).
• What are the reasons this is an effective
• question?

27
An Effective Essay Question

• The example is an effective essay question
• because
• The students have all of the information they
  need to provide you with the answer you expect.
• Students know where the material was (their
  textbook).
• Students know exactly what you want them to
  answer and how many points each part will be
  worth.
• You will be able to grade the question easily
  because you specified parts and point values for
  each one.

28
Summary

• In this module you learned
• How to use Blooms Taxonomy as the basis of all
  exam questions.
• How to use an exam blueprint to make sure youre
  testing at appropriate levels to connect
  assessment methods with your goals for students
  learning.
• How to write effective multiple choice
• questions and avoid common pitfalls.
• How to write and grade essays effectively.
• Click here to check what youre learned.

29
Quiz your knowledge

• According to this module, which of the
• following is a common mistake instructors
• make when writing multiple choice questions?
• Writing questions only at the Analysis level.
• Making all answers but one be very silly.
• Making more than one plausible answer.
• Check your answer

30
Quiz Your Knowledge

• According to this module, which of the
• following is a common mistake
• instructors make when writing multiple
• choice questions?
• Writing questions only at the Analysis level.
• Making all answers but one be very silly.
• Making more than one plausible answer.

31
Quiz Your Knowledge

• You want to test your students at the analysis
• and synthesis levels. According to this module,
• which of the following would be the BEST way to
• assign them a grade based on their learning at
• these levels?
• A multiple choice exam.
• An essay exam.
• An ungraded in-class exercise.
• Is is impossible to test knowledge at these
  levels.
• Check your answer

32
Quiz Your Knowledge

• You want to test your students at the analysis
• and synthesis levels. According to this module,
• which of the following would be the BEST way
• to assign them a grade based on their learning
• at these levels?
• A multiple choice exam.
• An essay exam.
• An ungraded in-class exercise.
• Is is impossible to test knowledge at these
  levels.
• Please press ltESCgt to exit this module.