Assessment

1
Assessment

• David Taylor
• Liverpool

2
Assessing knowledge

• There are many ways of assessing a students
  knowledge
• Coursework
• Essays
• Multiple choice
• Matching items
• others

3
MCQ vs EMI

• These are popular, since they are easy to mark
• But actually very difficult to write,
• they describe a continuum.

4
true/false

• Also known as guessing
• boys are taller than girls t/f

5
Multiple true/false

• Read the statement carefully, and give the best
  answer for each of the numbered statements
• the cardiac action potential
• Is quicker than a nerve action potential
• Calcium prolongs the action potential
• The potassium channels control the length of the
  action potential

6
Multiple choice

• Which of the following characteristics of an
  axon is most dependent on its diameter?
• a.       Its resting potential
• b.       The duration of its refractory period
• c.       The conduction velocity of its action
  potential
• d.       The overshoot of its action potential
• e.       The activity of its sodium-potassium
  pump
•  
• a b c d e

7
More sophisticated

• Match an interval from the ECG tracing below with
  each of the events listed.
•  
• 47. Closure of the atrioventricular valve
•   a b c d e
•   48. repolarisation of the atria
•   a b c d e
•   49. depolarisation of the ventricles
•   a b c d e

a
b
c
d
e
8
Matching items

• Each of the patients described below has
  symptoms of urinary urgency, frequency, and
  dysuria. Each has a midstream urine culture
  showing fewer than 10 colony-forming units per
  milliliter. For each of the numbered patients
  (1-2), select the most likely cause from the list
  of conditions (A-E).
•   A. Low -Count bladder bacteriuria
• B. Chlamydial Urethritis
• C. Urethral instability
• D. Urethral diverticulum
• E. Bacterial infection
•  1. A 36-year-old woman with symptoms for 2
  years additional symptoms include postvoid
  dribbling and dyspareuria.
•  2. A 20-year-old college student with pyuria and
  the gradual onset of symptoms over a period of
  2-3 weeks. She acquired a new sexual partner one
  month ago. 

9
Advantages MIs

• efficient for testing comparisons and related
  concepts and principles across broad content
  topic areas.
• straightforward to write and edit.
• It is important that all options relate to some
  common concept or theme
• there are more stimuli than options, so that
  examinees cannot attain correct answers by
  eliminating options.

10
Disadvantages of MIs

• Matching items tend to measure lower-level
  cognitive knowledge, such as recall or
  recognition of facts.
• difficult to write traditional matching items
  that measure higher-order cognitive knowledge.
• there is a tendency for this item type to test
  recall-recognition type information.
• Over sampling of test content may also be
  problematic,
• If some areas are over sampled, other content
  areas must necessarily be under sampled, thus
  reducing the test's validity evidence.

11
Extended matching items

• Options
• Acute leukemia
• Anemia of chronic disease
• Congestive heart failure
• Depression
• Epstein-Barr virus infection
• Folate deficiency
• Glucose 6-phosphate dehydrogenase deficiency
• Hereditary spherocytosis
• Hypothyroidism
• Iron deficiency
• Lyme disease
• Microangiopathic hemolytic anemia
• Miliary tuberculosis
• Vitamin B12 (Cyanocobalamin) deficiency
•  Lead in
• For each patient with fatigue, select the most
  likely diagnosis. Each option can be used once,
  more than once, or not at all. 

12
And the cases

• 1. A 19-year-old woman has had fatigue, fever,
  and sore throat for the past week. She has a
  temperature of 38.3 C (101 F), cervical
  lymphadenopathy, and splenomegaly. Initial
  laboratory studies show a leukocyte count of
  5000/mm3 (80 lymphocytes, with many lymphocytes
  exhibiting atypical features). Serum aspartate
  aminotransferase (AST, GOT) activity is 200 U/L.
  Serum bilirubin concentration and serum alkaline
  phosphatase activity are within normal limits.
•  
• 2. A 15-year-old girl has a two-week history of
  fatigue and back pain. She has widespread
  bruising, pallor, and tenderness over the
  vertebrae and both femurs. Complete blood count
  shows hemoglobin concentration of 7.0 g/dL,
  leukocyte count of 2000/mm3, and platelet count
  of 15,000/mm3.

13
What is an EMI

• four essential components
• a common theme of related concepts,
• a (short) lead-in,
• a list of options which are usually lengthy
  (always more than 5) and
• two or more item stems

14
Advantages of EMIs

• item stems that provide more detail
• clinical scenarios
• usually contain a long option list.
• more difficult than multiple-choice questions,
• with higher item discrimination
• higher reliability estimates.
• they may measure higher levels of the cognitive
  domain than the traditional matching item.

15
Disadvantages of EMIs

• Over and under sampling of content  
• the longer option list may lead authors to create
  questions that make trivial distinctions.