MCQ

1
MCQs 3Aligning with Course Objectives
2
MCQ Check-list

• This presentation will focus on
• Why we write them
• What type is best
• What flaws are to be avoided
• What content to test
• How to construct it

3
Why do we test?

• Formative assessment
• Communicate to the students what material is
  important
• Identify areas of deficiency in need of
  remediation or further learning
• Summative assessment
• Determine final grades
• Make promotion decisions
• Both
• Motivate students to study
• Identify areas where the course/curriculum is
  weak

4
What Should Be Tested?

• Exam content should match course objectives
• Important topics should be weighted more heavily
  than less important topics
• The testing time devoted to each topic should
  reflect the relative importance of the topics

5
What Should Be Tested?

• Sampling - Performance on the sample provides a
  basis for estimating achievement in the broader
  domain that is actually of interest
• Questions sample
• TOPICS eg MCC presentations
• SKILLS eg Course Objectives determining
  diagnosis, next step in management, etc.

6

• Term 4A, Jan 2012

7
(No Transcript)
8
(No Transcript)
9
(No Transcript)
10
Course Blueprint Alignment
11
Types of MCQs

• True/False family
• Assess recall of isolated facts
• Stems must be clear and unambiguous
• Options must be absolutely true or false or
  examinee must decide how true options are
• Generally not recommended usually have subtle
  flaws

12
Types of MCQs True/False

• Which of the following statements is true
  regarding contraception?
• oral contraceptives should not be initiated until
  six weeks after therapeutic abortion to avoid the
  risk of hypercoagulability
• an intrauterine device can provide good
  contraceptive protection 5 years after insertion
• oral contraceptives should not be prescribed to
  individuals under the age of 15 without parental
  consent
• lesbian women do not require counseling about
  contraception or sexually transmitted diseases
• oral contraceptives should not be prescribed for
  women over the age of 35

13
Types of MCQs

• One-Best-Answer family (A type)
• Options may be diagrammed
• D C A E
  B
• Correct answer is the most likely correct
• Better application of knowledge, integration,
  synthesis and judgment

Least correct
Most correct
14
Types of MCQs One-Best-Answer family

• A 24 year old university student came to see
  you because she is missing several days of school
  each month because of severe dysmenorrhea
  unresponsive to the OCP and NSAIDS. She also
  complains of deep dyspareunia and has a tender
  uterosacral nodule on pelvic examination. Which
  one of the following conditions is most likely
  responsible for her symptoms?
• Chronic pelvic inflammatory disease
• Endometriosis
• Adenomyosis
• Fibroids

15
MCQ Check-list

• Why we write them
• What type is best
• What flaws are to be avoided
• What content to test
• How to construct it

16
What to test?

• Items can be classified by cognitive processes
  required to answer the question
• Recall (Memory) tests knowledge of isolated
  facts
• Interpretation (Comprehension) review
  information to reach conclusion, eg diagnosis
• Problem solving (Reasoning) situation requires
  action, eg next step in management
• Process depends on background of trainee

17
What to test?

• Simpler classification based on task
• Application of knowledge item
• Reach conclusions
• Make predictions
• Select course of action
• Recall item
• Test knowledge of isolated facts

18
What to test?

• Test application of knowledge using clinical
  vignettes
• Adds face-validity
• Usually selects more important, less trivial
• Identifies student whove memorized but do not
  understand
• BUT experience in patient care should not be
  necessary

19
What to test - summary

• Focus item on key concepts and principles that
  are essential information (without access to
  references) for all examinees to understand
• Test material that is relevant to learning in
  pre-clinical courses and beyond
• Avoid items that only require recall of isolated
  facts
• Avoid esoteric or interesting topics that are not
  essential

20
What to test - summary

• Test application of knowledge using clinical
  vignettes to pose medical decisions in patient
  care situations
• Focus items on common or potentially catastrophic
  problems
• Pose clinical decisions that would be expected of
  a successful examinee
• ie Avoid clinical situations that would be
  handled by a (sub)specialist

21
Writing the Questions

• Construct stem include all necessary information
  to arrive at the right answer
• Choose lead-in pose a clear question
• Choose distractors

22
Stem

• Usually a clinical scenario
• Clear, unambiguous
• Should be long relative to options
• Includes all pertinent information
• Patients age/gender
• Clinical setting
• Complaint
• Other important info history, physical
  findings, test results

23
Lead-in

• Follows stem
• In the form of a question
• Should relate to the stem
• Should be answerable without looking at options
• Cover the Options test
• Should clearly communicate the learning objective

24
Distractors

• Each should be selected by some, therefore all
  plausible, none obviously incorrect
• 3 or 4 choices
• Common misconceptions
• Faulty reasoning

25
Distractors

• Homogeneous in content to the correct answer
• Should be in same category as correct answer
  diagnosis, treatment, diagnostic test
• Clearly incorrect or inferior to the correct
  answer
• Plausible and attractive to the uninformed
• Similar to the correct answer in construction and
  length
• Grammatically consistent and logically compatible
  with the stem

26
General Guidelines for Item Construction

• Include as much of the item as possible in the
  stems Long stems, short options
• Avoid negatively phrased items (except or not in
  the lead-in). If you must, use only with short
  options
• Avoid writing questions of the form
• Which of the following statements is correct?
• Each of the following statements is correct
  EXCEPT
• Unfocussed
• Heterogeneous options

27
Top-down versus Bottom-up

• Top-Down
• Knowledge or recall question
• Cite a disease and then ask what patient findings
  are expected
• Structured like textbooks
• Clinically backward

28
Top-down versus Bottom-up

• Bottom-up
• Application of knowledge
• Gives findings and asks examinee to indicate the
  disease
• Examinees need to be able to synthesize
  information from several pages of a textbook (or
  notes) to answer these questions
• Clinically realistic

29
Basic Rules for MCQs (A type)

• Focus item on important concept
• Item should assess application of knowledge, not
  recall of an isolated fact
• Stem must pose clear question Cover the
  options test
• All distractors should be homogeneous
• Avoid technical item flaws

30
Reference

• Constructing Written Test Questions For the Basic
  and Clinical Sciences
• Case Swanson, NBME 1996

31
Clinical Presentation

• A 28 year old female presents to the ER with a 2
  hour history of severe right sided abdominal pain
  and vomiting.

32
(No Transcript)
33
Clinical Presentation

• A 28 year old female presents to the ER with a 2
  hour history of severe right sided abdominal pain
  and vomiting.

34
Clinical presentation

• A 28 year old female presents to the ER with a 2
  hour history of severe right sided abdominal pain
  and vomiting. This was preceded by a 1 week
  history of urinary frequency and urgency. On
  physical examination, VSS are stable and she is
  afebrile. Abdomen is non-tender. She is noted to
  have mild Right CVA tenderness.

35
(No Transcript)
36
Clinical Presentation

• 1) A 28 year old female presents to the ER with a
  2 hour history of severe right sided abdominal
  pain and vomiting. This was preceded by a 1 week
  history of urinary frequency and urgency. On
  physical examination, VSS are stable and she is
  afebrile. Abdomen is non-tender. She is noted to
  have mild Right CVA tenderness. Which of the
  following investigations would be the next step
  to make a diagnosis?
• a) abdominal X-ray
• b) abdo/pelvic USS
• c) urinalysis
• d) CBC

37
(No Transcript)
38
Presentation

• 2) A 28 year old female presents to the ER with a
  2 hour history of severe right sided abdominal
  pain and vomiting. This was preceded by a 1 week
  history of urinary frequency and urgency. On
  physical examination, VSS are stable and she is
  afebrile. Abdomen is non-tender. She is noted to
  have mild Right CVA tenderness. Urinalysis
  reveals blood, WBC and nitrites neg. CBC is
  normal. Which of the following is the most likely
  diagnosis?
• a) appendicitis
• b) pyelonephritis
• c) nephrolithiasis
• d) cystitis

39
(No Transcript)
40
Presentation

• 3) A 28 year old female presents to the ER with a
  2 hour history of severe right sided abdominal
  pain, preceded by urinary frequency and urgency.
  VSS are stable and she is afebrile. She is noted
  to have mild Right CVA tenderness. Urinalysis
  reveals blood, WBC, nitrites neg. You
  suspect nephrolithiasis. Which of the following
  investigations is the best choice to confirm the
  diagnosis?
• a) Abdominal and pelvic USS
• b) abdominal x-ray
• c) CT scan
• d) IVP

41
(No Transcript)
42
Presentation

• 4) A 28 year old female presents to the ER with a
  2 hour history of severe ride sided abdominal
  pain with R CVA tenderness. A CT scan confirms
  the presence of a R UVJ stone measuring
  approximately 3 mm with complete ureteric
  obstruction. There are no signs of Urinary Tract
  infection. You provide analgesia. Which of the
  following is the next step in managing this
  problem?
• a) Hydration
• b) Percutaneous nephrostomy
• c) Cystoscopy and basket retrieval of stone
• d) Lithotripsy

43
(No Transcript)
44
Presentation

• 5) A 28 year old female has recently recovered
  from her third bout of renal colic. She is
  otherwise healthy. Which of the following
  investigations would be most helpful in finding
  the cause of her recurrent nephrolithiasis?
• a) IVP
• b) CT scan
• c) serum Calcium
• d) 24 hour urine calcium

45
(No Transcript)
46
Presentation

• 6) A 28 year old female has recently recovered
  from her third bout of renal colic. She is
  otherwise healthy. Investigations reveal no
  evidence of hyperparathyroidism, urinary tract
  anomaly or other underlying condition. Which of
  the following would be most helpful in preventing
  this from recurring?
• a) hydrochlorothiazide
• b) Flomax
• c) cranberry juice
• d) 8 glasses of water daily

47
Basic Rules for MCQs (A type)

• Focus item on important concept
• Item should assess application of knowledge, not
  recall of an isolated fact
• Stem must pose clear question Cover the
  options test
• All distractors should be homogeneous
• Avoid technical item flaws

48
One last step

• Get a colleague to read over your questions
• Detects more than one right answer
• Discovers the what am I thinking? question
• Identifies not enough information in the stem