Quality Assessment in Medical Education Need for Change

1
Quality Assessment in Medical Education Need for
Change
2
Introduction

• Primarily directed towards
• Measurement of the clinical competence the
  ability to gather data from the patient by
• History physical examination
• Diagnostic formulation
• Select appropriate investigations to confirm the
  diagnosis

3

• Institute efficacious management
• Faculty observations of clinical performance with
  rating scales
• Oral examinations of clinical competence
• Multiple choice examination of clinical
  competency
• Faculty and house staff observation of students
  clinical performance.
• The assessment solely based on factors above are
  crude and unrealistic because of lack of fixed
  yardsticks.

4
Material Methods

• Students
• A questionaery was devised for objective
  analysis of current teaching methods in five
  leading medical institutions of the country.
• 500 students studying in 4th year of their
  studies were randomly selected
• E-mail personal contact answers were obtained
  from only 400 students

5
Teachers Trainers

• Teachers and trainers were also randomly selected
  and objective analysis of their experiences was
  also collected
• Only 100 trainers examiners were selected due
  to their complete response to the questioners.

6
Patients

• Patients from
• Primary care BHU/RHU and
• Secondary care i.e.
• Tehsil District Headquarter
• Hospitals
• Teaching Hospitals
• A class private hospitals were interviewed

7

• The information manning these hospitals was
  obtained regarding the number of
• locally trained doctors
• Diploma holders
• MS/MD
• FCPS
• Produced locally as specialists.

8

• Similarly the number of specialists trained in
  developed countries like UK
• USA
• Canada
• Australia was collected.
• Comparison of the data was carried.

9
Analysis of Data

• The results obtained from the survey are analyzed.

10
Table-I shows response of fourth year students to
the parameters and relative frequency of teaching
and assessing methods at teaching institution.
11
Table-II shows response to question are regarding
basic sciences and clinical teaching.
12
Table-III shows the response to evaluation,
Assessment and Scoring methods employed by the
trainers.
13
Table-IV shows the patient response to the level
of care at different hospitals with different in
Local and Foreign Trained Doctors.
14
Analysis of the Data

• Need for improvements in our teaching system
• Ultimately what is taught is reflected as
  knowledge
• Translation through their practice
• Assess the knowledge which takes the center
  stage in assessment of quality

Continued
15
Analysis of the Data

• Professional education in our country continue to
  be imparted without developing any fair and
  uniform system for assessment.
• The basic requirements for training are based on
  obtaining knowledge assessed by
• MCQs
• PBLS
• oral assessments
• video or computer Simulations.

Continued
16
Analysis of the Data

• Any training program without a
• formally structured infrastructure
• training facilities
• protocols log books
• multimedia and computer labs
• and continuity of the training program by able
  teaching faculty and administration set up
• is bound to remain defective.
• Knowledge assessment as only tool is often
  misused.
• It remain valid if there is satisfactory
  completion of accredited training and successful
  examination performance.

Continued
17
Analysis of the Data

• Scores are often not related to the quality of
  education and professional performance
• Formal training set ups do better than the non
  formal jump ups or lateral programs.
• Better qualified training directors produce
  better trained graduates.
• Exam scores are sensitive to education but do not
  provide the quality assessment.

Continued
18
Analysis of the Data

• Written exams formats and essays, video and
  computer based simulations, clinical evaluation,
  ,orals and written exams correlation are higher
  when the tests assess similar aspects of
  compliance.
• Knowledge assessment is measure of quality and
  will continue to be used.
• Knowledge is only one aspect of training
• Assessment of competence and performance combined
  will provide better information on quality
• Work based assessment will make better basis for
  judgment by analyzing clinical record data,
  administrative data.

Continued
19
Analysis of the Data

• Diaries or logs of work maintained by the
  students and observed by the Trainers.
• Quality will also be assessable on
• Mortality
• Morbidity
• Patient satisfaction
• Functional status
• Cost effectiveness and
• Intermediate outcomes in Physiological functions
  of the patient.
• Patients students form a basis for deciding
  the work performance.
• Doing the right thing does not ensure a good
  outcome
• The relationship between volume and quality is
  also important
• Doing something often is not a guarantee of
  doing it right

Continued
20
Analysis of the Data

• Volume is a necessary but not sufficient
  condition for quality.
• It was abundantly clear from the data collected
  that there is clear difference in the care being
  provided at primary, secondary and Tertiary care
  hospitals.
• Strangely private hospitals were managed by
  foreign trained health personals
• Public seemed to be more satisfied with the level
  of care provided at such hospitals

Continued
21
Analysis of the Data

• Government has created a mechanism for quality
  and accreditation has been created for
  universities and autonomous colleges through
  PMDC.
• On the spot structural training assessment
  remain badly lacking.
• A quality assurance cell in the HEC can fill this
  gap.
• Mechanism for advice on a continuous basis, in
  connection with planning evaluation of
  professional education, at district, state or
  national level.

Continued
22
Analysis of the Data

• Self-audit external audit are both
  indispensable for quality assurance.
• The most difficult tasks facing medical education
  is to certify a level of achievement.
• The method of assessment has a powerful effect on
  how what students learn.
• Misinformation will be given back to students and
  poor decision will be made.
• Ultimately, inferior graduates will result in
  dissatisfied patients and compromise health care.

Continued
23
Analysis of the Data

• The board overseeing the United States Medical
  Licensing Examination (USMLE), a new clinical
  skills exam- a nontraditional performance
  assessment have proven more effective.
• A sound assessment modality must include
• a clear statement of purpose
• a detailed description of what is to be measured
• a set of instructions for feasible administration
• scoring
• and guideline for data interpretation.

Continued
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
Conclusion

• All assessments are imperfects measures of
  knowledge, skills, and performance.
• In the context of assessment, validity refers to
  the degree to which evidence and theory support
  the interpretation of test scores
• It is most important consideration in determining
  the quality of an assessment and refers to the
  appropriateness, meaningfulness, and usefulness
  of the specific inferences.

32
References

• Angoft W-H (1971)
• Elliot D. L. et al (1987)
• Barrows H.S. (1993)
• Colliver, J.A. (1994)
• Badger L.W. et al (1995)
• Bardes, C. L. (1996)
• Downing, S . M. et al (1997)
• American Educational Research Association, (1999)
• Adamo, G (2003)