TABLE OF SPECIFICATION OF MCQ COMPREHENSIVE EXAMINATION

1
?????????????????? MCQ
2
??????????????????????????
3
??????????????????????????

• ??????????????
• ????
• ???????????????
• ???????????
• ??????????
• ????????????????????
• ?????????????????????????????
• ?????????????????????????

????? ????????????
4
?????????????

• ??????????
• ??????????????????????????????(Content Validity)
• ????????????????????????(Objectivity)
• ????????????????(Objectivity)
• ??????????? (Difficulty)
• ?????????? (Comprehensiveness)

????? ????????????
5
?????????????

• 2. ??????????
• ??????
• ????????????????? ???????? (p)
• ???????????????????????????????????? (r)
• ????????????????????????????
• ????????
• ???????????????????????????????
• ?????????????????????????(Reliability)
• ??????????????????????????(Error of Measurement)

????? ????????????
6
?????????????????

• Validity
• Reliability
• Objectivity
• Comprehensiveness
• Practicability

• Good level of difficulty
• Good discrimination power

7
??????????????????

1. ??????????????????????????
2. ????????????????????????

8
??????????????????????????

• Mean, SD
• Median
• Mode
• Range Max - Min
• Standard error of measurement ( SEM )
• Reliability ( rtt KR-21, KR- 20 )
• Difficulty index ( p )
• Discrimination power ( r )
• MPL (AI )
• n

9
Reliability
10
Validity v.s Reliability
?
not Valid, Reliable
not Valid, not Reliable
Valid, Reliable
11
Reliability ( rtt )
12
??????????????????????????????????
???????????????
13
??????????? Reliability ( KR-20 )
National Board of Medical Examiner ( NBME )

• Reliability ??????? 0.70 ??????????????????????
  ?
• ???????????????????????
• ????????????????????? performance ????????
• Reliability ?????????????????? 0.70 ??????
• ??????????????????????????????????????
• Reliability ?????????????????? 0.90 ??????
  

?
?
14
Standard Error of Measurement ( SEM )
???????? ???????? MCQ ??????????????????
???????????? ???????? ????????
Mean 58, SD 6.67 ??? rtt 0.64
SEM SD ? 1 rtt 6.67 ? 1
- 0.64 4
???. ?. ?????? 48 ????? (Obtained Score)
15
(No Transcript)
16
2. ????????????????????????
( Item Analysis )
17
??????????????????? Criterion - referencedMPL
18
Pretest Analysis
The task should be done by the teachers to
evaluate the test to be used in term of

• Validity
• Difficulty factor (DF) / Acceptability index
  (AI)
• Minimum pass index (AI)

19
Angoff Method

• ??????????????????? performance ??????????????
• ?????????????????????????? (marginal)
• ?? probability ????????????????????????????????
• ???????????????
• ??????????? probability ?????????????????
• ??????????????????????? MPL

20
Modified Angoff Method
??????????????????????????????????????????????????
??????????????????????? 1.00
Question Probability for Correct Answer
1 .75
2 .75
3 .60
4 .60
5 .50
6 .40
7 .75
8 .75
9 .50
10 .40
?
Expected passing score 60 Sum 6.00
21
(No Transcript)
22
Item analysis

• base on student,
• at one point in the time,
• reflecting some characteristics
• of the item

23
???????????????????????? (Item Analysis)

• ???????????? (Difficulty Index, p)
• ?????????? (Discrimination Index , r)
• ???????????????????? (Distracters)

24
?????????
???
???
????? ????????????
25
??????????? p ??? r
Item 41
A B C D E
High 13 ( n 53 ) 3 4 3 3 40
Low ( n 53 ) 6 1 9 8 29
P H L 2n 0.08 0.04 0.11 0.10 0.65
r H L n - 0.06 0.05 - 0.12 - 0.09 0.20
?
26
(No Transcript)
27
Item Discrimination Index Point BIserial
Correlation( rpbs )
What does the point biserial measure ? The
relationship between two variables, a categorical
variable (i.e., the student either answered the
test item correctly or incorrectly), with a
continuous variable(i.e., percent score on the
examination).
How do you interpret a point biserial
correlation? As with all correlation indices,
the point biserial ranges from 1.00 to 1.00.
A positive point biserial tells us that those
scoring higher on the exam were more likely
to have better item discriminates between
high-scoring and low-scoring students).
www.som.tulane.edu.
28
M p M q St
To calculate the rpbi for each item, use the
following formula
rpbi
rpbi point-biserial correlation coefficient
Mp whole-test mean for students answering item correctly (i.e., those coded as 1s)
Mq whole-test mean for students answering item incorrectly (i.e., those coded as 0s)
St standard deviation for whole test
p proportion of students answering correctly (i.e., those coded as 1s)
q proportion of students answering incorrectly (i.e., those coded as 0s)
29
Uses of Indices Review of Question
Easy
Difficult
30
Uses of Indices Review of Question
Discrimination Index (r) H-L x 100
N
31
??????????????????????????
32
II. Mechanisms of Disease
???????? 50 ?? ?????????? ??????????????????????
?????? ????????????????????????????
????????????????? ? ???????????????? ????????? 7
??. ?????????????????????????????????
??????? ???
A. Toxocara canii B. Trichinella
spiralis C. Strongyloides stercoralis D. Gnathos
toma spinigerum E. Angiostrongylus cantonensis
?????? ?????? ???????????
33
??????????????????????????????????????? (r)

• 1.???????????????????(r) ???????????? item
  validity
• 2.??????????? r ????????????????????????????????
• ??????
• ?????????????
• ????????????????????
• ??????????????
• 3. ??????????????????????????????????????????? ??
  ????????????

????? ????????????
34
????

1. ?????????????????????????????????????????
   ????????????????
2. ??? p ??????????????????? ???????????
   ????????????????????? ?? ???? ???
3. ??? p ???????????????? ???????????????????????????
   ?????
4. ?????????????? p ????????? r ??? p ????????
5. ?????????????? r ????????? p ??? p ???????????
6. ?????????????????????????????? Validity

????? ????????????
35
???????????????????? MCQ
36
??????????? (MCQ)
???????

• ?????????????????????????
• ????????????????????????????
• ??????????????????????
• ????????????????????????
• ????????????????????
• ???????????????????????????????

37
??????????? (MCQ)
???????

• ????????????????????????????????????????
• ??????????????????????????????????????
• ??????????????????????????
• ?????????????????????
• ????????????????????????????????
• ?????????????????????????

38
Thank for your attention