PowerPoint Presentation Selecting Medical Students

1
(No Transcript)
2
Personal Qualities AssessmentPQA2004

• David Powis, Miles Bore, Don Munro
• The University of Newcastle
• New South Wales, Australia

3
Psychometric tests

• can measure skills and attributes difficult to
  assess by other methods
• objective
• efficient (in terms of numbers of applicants
  processed)
• can be made both reliable and valid

4
Psychometric tests

• can be designed to test for
• cognitive skills (abilities)
• personality factors
• traits
• states
• values and beliefs
• attitudes

5
PQA

• Personal Qualities Assessment
• test battery developed at University of
  Newcastle, NSW
• three components, MCQ format
• A cognitive skills battery
• A personality inventory
• An ethical sensitivity / moral orientation test

6
Rationale for PQA

• to quantify cognitive skills
• to provide a profile of personal qualities and
  attributes, in the domains of
• ethics and morality
• interpersonal communication
• in applicants for Health Professional courses

7
PQA

• Cognitive Skills Test
• (Mental Agility Test - MAT Drs Don Munro Miles
  Bore)
• Personality Inventory
• (NACE Dr Don Munro)
• Ethical sensitivity / Moral orientation
• (Mojac Dr Miles Bore)

8
PQA

• Mental Agility Test
• Cognitive Skills
• verbal, numerical and spatial reasoning skills

9
(No Transcript)
10
MAT
45 items mean score 24.8 5.8 S.D range 5 - 41
11
Rasch analysis of MAT N 747
12
MAT
13
MAT
14
PQA

• Mental Agility Test
• Personality Inventory (NACE)
• Ethical sensitivity / Moral orientation (Mojac)

15
Ethics and Personality

• 1997 Academic/clinical staff asked to identify
  unethical students/doctors and to describe them
• the descriptors formed the basis for development
  of Mojac and NACE

16
Common descriptors of ethically
inappropriatebehaviours and attitudes in medical
personnel

• arrogant
• power-seeking
• inflexible
• defensive
• dishonest
• patronising
• brash
• egocentric
• isolated
• insensitive
• self-centred
• uncaring

• indifferent
• selfish
• antisocial
• amoral
• devious
• prejudiced
• flippant
• rude
• aggressive
• condescending
• rigid attitudes
• judgemental

17
NACE and Mojac

• validated with more than 16,000 individuals
• applicants to medical schools and health
  professional courses
• students in medicine, psychology, nursing, other
  health professional courses
• practising doctors
• Australia, NZ, Fiji, Israel, UK, SE Asia

18
NACE
Example
This test contains 100 statements about the way
you might behave and think in certain situations,
and general statements about people. Read each
of them quickly and decide whether you think each
statement is A definitely false OR
B false on the whole OR C true on the
whole OR D definitely true
1. When things get routine and boring I like to
stir up some excitement 2. I am quite
affectionate towards people 3. When I am a
member of a committee I like to be the
chairperson 4. I love to see two people who are
obviously happy and in love 5. Even when I am
willing to help, people seem to find it difficult
to confide in me 6. At parties I tend to
sit with just one other person rather than join
the crowd
19
NACE

• Factor analysis indicated 4 factors
• construct validity established against standard
  measures
• Four factors identified as
• Narcissism
• Aloofness
• Confidence
• Empathy

20
NACE a second-order factor

• Involved (Empathy, Self-Confidence)
• with others
• Detached (Aloofness, Narcissism)
• from others

21
NACE
22
NACE
23
NACE
24
NACE
25
NACE


Sample
n
Narcissism
Aloofness
Confidence
Empathy












School Leaver Applicants
1273
51.6 (7.6)
46.0 (6.7)
70.8 (7.4)
74.1 (6.6)







Graduate Applicants
838
50.3 (7.9)
42.7 (6.8)
75.2 (7.4)
74.9 (6.8)







1st Year BMed
280
53.4 (8.9)
47.1 (7.4)
67.5 (7.6)
73
.7 (7.3)







2nd Year BMed
286
55.1 (8.2)
49.2 (7.9)
65.2 (8.1)
72.2 (7.1)







3rd Year BMed
76
53.8 (7.0)
47.5 (6.3)
68.0 (6.4)
73.1 (6.5)







4th Year BMed
59
53.1 (7.4)
48.3 (6.3)
65.7 (7.7)
71.1 (6.8)







5th Year BMed
60
52.3 (7.5)
46.7 (6.7)
65.8 (
8.4)
71.7 (6.9)
Medical Graduates


166

45.0 (7.3)

41.6 (6.6)

70.1 (7.6)

74.6 (6.7)


26
Ethics

• ethical knowledge
• ethical beliefs
• ethical attitudes and sensitivity
• moral reasoning
• logic
• ethical development
• moral orientation
• moral decision making
• ethical behaviour

27
Ethics

• ethical knowledge
• ethical beliefs
• ethical attitudes and sensitivity
• moral reasoning
• logic
• ethical development
• moral orientation
• moral decision making
• ethical behaviour

Basis of Mojac
28
The problem with moral reasoning

• Dilemma
• Decision
• Reason

Post-decisional justifications
Kohlberg (MJI), Rest (DIT), Gibbs (SROM)
29

• Dilemma
• Decision
• Reason

Pre-decisional processes (Mojac)
Post-decisional justifications
Kohlberg (MJI), Rest (DIT), Gibbs (SROM)
30
Mojac

• (Mojac is just a name)
• developed by Miles Bore
• a dissonance model of moral decision making
• (moral cognitions in conflict)
• based on a series of moral dilemmas

31
Mojac
Example
32
Mojac
Example
What is your opinion? How do you feel about each
of the following statements? For each of these
statements mark A, B, C, D or E (A, Strongly
Agree B, Agree C, Not Sure D, Disagree E,
Strongly Disagree) 1. Mr D should steal the
money 2. People should not just take what they
want 3. A husband should save his wife's
life 4. Even in this situation stealing is
wrong 5. Mr D should maintain his trustworthy
reputation 6. Saving a person's life is more
important than upholding the law 7. Mr D should
steal the money only if he can do it without
getting caught 8. There must be something
wrong with our laws for Mr D to be in this
situation
33
(No Transcript)
34
(No Transcript)
35
(No Transcript)
36
(No Transcript)
37
(No Transcript)
38
(No Transcript)
39
(No Transcript)
40
(No Transcript)
41
(No Transcript)
42
(No Transcript)
43
Moral Orientation

• Mojac Scale (Bore, 1998, 2000, 2001)
• Responses to moral dilemmas can be expressed in
  following form

Duty to Society/Group
societys rules for family groups, class groups,
work groups humanity as a group
44
Individual
40
30
20
10
Group
-30
-20
-10
0
10
20
30
45
Distribution of LibCom Scores
46
three response patterns
47
Libertarian orientation
Dual orientation
Communitarian orientation
Values duty to the group much higher
than individual freedom
Values individual freedom much higher than
duty to the group
Values equally individual freedom and duty
to group
48
Moral Orientation Dimension
Libertarian
Dual
Communitarian
49
Mojac
50
Mojac
51
Mojac
52
Mojac
53
Mojac
54
Mojac
55
(No Transcript)
56
Gender Effects(n747 296 Males, 451 Females
24.8 4.7 years)
57
Gender Effects, MAT
58
Gender Effects, Mojac
59
Gender Effects, NACE
60
MAT
61
NACE
62
Mojac
63
Construct validity of Mojac and NACE

• Mojac
• NACE
• 16PF modified (Cattell, 1998)
• IPIP Big 5 (Goldberg, 1999)
• Right Wing Authoritarianism (Altemeyer, 1982)
• Emotional Intelligence (Schutte et al. 1998)
• Eysenck Personality Questionnaire (Eysenck, 1985)

64
(No Transcript)
65
NACE
66
Mojac correlates
Libertarian
Communitarian
67
(No Transcript)
68
(No Transcript)
69
Mojac NACE

• Mojac continuum is orthogonal to NACE

Involved
Libertarian
Communitarian
Detached
70
N A C E
M o j a c
71
Medical school applicants n
1273, 838
N A C E
M o j a c
72
(No Transcript)
73
Mojac and NACE Moral Types
(n 201 First Year Medical Students, New Zealand
Sample)
Libertarian
Dual
Communitarian
not narcissistic empathic trusting,
warm agreeable high soc. desirability reliant
on others
not authoritarian emotionally stable nonconforming
open minded lively
conscientious emotionally stable rule-conscious
open minded trusting
Involved
narcissistic, neurotic not conscientious antagoni
stic tolerates disorder reserved,
vigilant emotionally reactive non-conforming abs
tracted (creative)
not empathic antagonistic closed-minded vigilant
low soc. desirability reserved self-reliant
authoritarian closed-minded serious rule-consci
ous
Detached
Note all are plt 0.01 except p lt 0.001
74
Mojac and NACE Moral Types. Individual
differences in how we treat others
Libertarian
Dual
Communitarian
Overly caring
Permissive Autonomy
Paternalistic
Involved
MOST PEOPLE Concerned for the well-being of
individuals and groups
Intolerant of different others Authoritarian
Detached
Manipulative
Overly uncaring
75
Implications for selection

• Do these tests have validity?
• YES (Concurrent) Mojac and NACE have been
  validated against established and well documented
  measuring instruments, e.g., 16PF, IPIP Big Five
  etc.
• Can they be learnt?
• There is no right answer to the ethics and
  personality items.
• Are they stable
• Test-retest data suggest they do not change
  substantially, during medical school or with age.
  

76
Mojac test retest, after one year interval
n 59
77
Does moral orientation change over the course of
medical education?
Cross-sectional data
MOJAC LibCom Mean Scores for the Five Year Levels
of the BMed, 1999 Sample
BMed Year Level
n
Mean
S.D.
Year 1
65
60.9
7.6
Year 2
43
63.3
7.9
Year 3
59
66.6
8.7
Year 4
30
64.1
6.1
Year 5
37
65.3
6.5
78
Implications for selection II

• Do Mojac/NACE have predictive value?
• ethical clinicians performed as predicted
• predicts degree of discomfort/dissonance felt by
  respondents
• how people describe real-life experiences
  predicts their moral orientation (backward
  prediction)
• strongly predicts the decisions people make
• Clinical Interview assessment results correlate
  with NACE scores (NZ study)

79
Summary

• PQA can measure and quantify cognitive ability
• But do cognitive skills relate to success in
  medical studies and medical practice?

80
Summary

• PQA can measure and quantify cognitive ability
• But do cognitive skills relate to success in
  medical studies and medical practice?
• PQA can reliably identify desirable and
  undesirable personal qualities that can be
  labelled
• But do these labels describe actual behaviour?

81
Use of PQA

• A basis for eliminating unsuitable candidates
• poor cognitive skills
• extreme moral types
• extremes on involved/detached scale

82
Use of PQA

• A basis for eliminating unsuitable candidates
• poor cognitive skills
• (- 2 S.D. from mean)
• extreme moral types
• ( 2 S.D from mean)
• extremes on involved/detached scale ( 2 S.D
  from mean)

83
(No Transcript)
84
MAT
85
(No Transcript)
86
(No Transcript)
87
Medical school applicants n
1273, 838
N A C E
M o j a c
88
Use of PQA

• A basis for ranking candidates
• (after unsuitable applicants eliminated)
• good cognitive skills (average and above average
  scorers)

89
(No Transcript)
90
Do we need tests at all?
91
A recent report...

• Wright Tanner
• BMJ 324 2002, 1554 - 1555 (29 June)
• Medical students compliance with simple
• administrative tasks and success in final
• examinations

92
Photograph No Photograph
93
Photograph 366 No Photograph 27
94
Failed
Photograph 366 29 8 No Photograph 27
95
Failed
Photograph 366 29 8 No Photograph
27 13 48
96
2906 2862 234 16 52 133 72 84 236 510 83 67 986 58
6
N
97
New Zealand Year 1 Health Science students
N 236
N A C E
M o j a c
98
Scottish medical school applicants
N 491
N A C E
M o j a c
99
(No Transcript)
100
(No Transcript)
101
Mojac
102
(No Transcript)
103
(No Transcript)