Model for Monitoring and Evaluation of Overall Health System Performance for Comparison

1

• Model for Monitoring and Evaluation of Overall
  Health System Performance for Comparison
• Based on the Study Conducted for the Ministry of
  Health

2

• General Objective
• Comparison of overall health system
  performance against selected indicators between
  districts using one index district from each
  province.

3
Mandate

• The mandate was to look at routinely collected
  and readily generated data at district level to
  measure the selected indicators.

4
Do other countries use health system performance
indicators?
5
Cross-Country Comparison of Concepts of Health
System Performance
Dimensions and Subcategories of Health System Performance OECD Proposed framework WHO framework Australias Proposed framework Canadas Health System Performance framework UKs NHS High-level Performance Framework
Health Improvements/Outcomes X X X X X
Appropriateness X X X
Capacity/Competence X X
Safety X X X
Responsiveness X X
Patient Satisfaction Patient experience/ accessibility Acceptability X X X X X
Accessibility (in terms of timeliness of services X X X
Continuity X X
6
Cross-Country Comparison of Concepts of Health
System Performancecontd
Dimensions and Subcategories of Health System Performance OECD Proposed framework WHO framework Australias Proposed framework Canadas Health System Performance framework UKs NHS High-level Performance Framework
Equity X
Equity of Health Outcomes X X
Equity of access X X X X X
Equity of Finance X X
Efficiency X X
Macroeconomic efficiency X
Overall micro efficiency X X
Unit costs X X X
7
PROCESS
8

• Initially a steering committee was established
  consisting of ministry officials the AHF
  secretariat.
• Based on the literature review of national and
  global materials and the documents available with
  the MoH a draft conceptual framework for
  measurement of health system performance was
  presented to Steering Committee and modified
  taking into account the views of the committee.

9

• Concurrently the study team also looked at the
  indicators used world wide for performance
  measurements.
• After identifying the readily available
  indicators at provincial and national levels with
  the concurrence of the steering committee it was
  decided to place them before high level ministry
  officials from centre as well as the managers
  from the provinces.

10

• The following indexed districts one per each
  province were selected for comparison with the
  concurrence of the ministry officials PDHSs.
• The selected districts were
• - Gampaha Western Province
• - Ratnapura Sabaragamuwa -Province
• - Anuradhapura North Central-Province
• - Galle Southern Province
• - Matale Central Province
• - Trincomalee North East Province
• - Badulla Uva Province
• Kurunegala North Western Province

11
Sri Lankan Health Performance Framework-
A Model
Health Status and Outcome Tier 1
Health System Performance Framework Tier 3
HOW HEALTHY ARE THE CITIZENS? IS IT THE SAME FOR EVERYONE? WHERE IS THE MOST OPPORTUNITY FOR IMPROVEMENT? HOW HEALTHY ARE THE CITIZENS? IS IT THE SAME FOR EVERYONE? WHERE IS THE MOST OPPORTUNITY FOR IMPROVEMENT? HOW HEALTHY ARE THE CITIZENS? IS IT THE SAME FOR EVERYONE? WHERE IS THE MOST OPPORTUNITY FOR IMPROVEMENT?
Health condition Life expectancy and wellbeing Deaths
LBW Infectious diseases -incidence of malaria -incidence of dysentery -TB new cases Prevalence of anaemia among pregnant women Incidence of HIV/AIDS(NSACP) Life expectancy IMR MMR NMR Case fatality rate for dengue fever/DHF
Effectiveness Efficiency Sustainability Health Services
Measles coverage Tetanus toxoid (TT2 ) given to pregnant mothers of pregnant mothers tested for VDRL Pap smear screening rate Number of deaths within 48hrs of admission In pt /staff Hospital bed occupancy rate Average length of stay (MSU) expenditure for health vs. total budget of expenditure on drugs CS Rate DOTS treatment success rate Nurses or doctors/hospital bed (AHB)MSU Hospital beds /1,000 population (AHB)MSU Doctors /100,000 population (AHB)MSU Nurses/100,000 population PHM/ 100,000 population (AHB)MSU Patient Transfers No. of new cases of diabetics in hospital clinics for -diabetics -Hypertension -Cancer of medical audits done for -maternal deaths -still births No. of major surgical operations No. of minor surgical operations
Responsiveness Access Tier 2
IS THE SYSTEM GEARED TO MEET EXPECTATIONS THE FELT NEEDS OF THE CITIZENS? IS THE SYSTEM GEARED TO MEET EXPECTATIONS THE FELT NEEDS OF THE CITIZENS? IS THE SYSTEM GEARED TO MEET EXPECTATIONS THE FELT NEEDS OF THE CITIZENS?
Responsiveness Accessibility Equity Safety
These dimensions are not going to be measured during this project. These dimensions are not going to be measured during this project. These dimensions are not going to be measured during this project.
-Indicators not identified. Data has to be obtained from community surveys. Measuring equity in access to health services requires household survey data at district level. -Incidence of adverse drug reaction -incidence of nosocomial infection
12
Routine data was not available for the following
indicators

• Prevalence of anaemia among pregnant women
• Inpatient to Staff Ratio
• Percentage of expenditure for health vs. total
  budget
• Percentage of expenditure on drugs
• Patient Transfers
• Number of New cases of NCDs in hospital clinics
  forDiabetes HypertensionCancer
• Percentage of Medical Audits done for still
  Births

13
RESULTS - PILOT STUDY
14
Health Status and Outcome Tier 1 Health
Conditions
District
District

15
Deaths
Infant Mortality Rate (2005)
Definition Number of deaths to infants under one
year of age per 1,000 live births in a given year
District
Infant Mortality Rate per 1000 live births
16
Percentage of pregnant mothers tested for VDRL
(2005)
Health System Performance Tier 3
Effectiveness
Definition Number of mothers tested for VDRL as a
percentage of total number of deliveries reported.
Percentage of Pregnant mothers Tested for VDRL
17
Efficiency
Average length of stay(2005)
Definition The average length of stay a patient
spends in a government hospital. It is measured
by dividing the total number of days stayed by
all inpatients in government hospitals during a
year by the number of admissions
Average Length of Stay in Hospital
18
Caesarean Section Rate (2005)
Definition The number of caesareans per 100 live
births in government hospitals
Gampaha
28.6
Matale
23.9
Galle
23.2
District
Kurunegala
22.7
Badulla
22.5
Average
21.9
Anuradhapura
20.0
Ratnapura
19.5
Trincomalee
14.6
0
5
10
15
20
25
30
35
Rate
Caesarean Section Rate
19
Nurses per 100,000 population(2005)
Sustainability
20
Health Services Percentage of
Medical Audits done for Percentage of Medical
Audits done for Maternal deaths(2005)
Percentage of medical audits done for maternal
deaths
21
SUMMARY SHEETS FOR OVERALL HEALTH SECTOR
PERFORMANCE BY DISTRICT
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
(No Transcript)
27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
Direction for Establishing Using this Model for
Systematic Monitoring for Comparison of Health
System Performance between Districts in the
Future
31
POLICY ISSUES
32

• For future analysis the level of analysis of
  costing, whether provincial or district, needs to
  be identified. At present there is no provision
  to collect and compile the cost data at the
  district level as they are not accountable for
  them. Hence a policy decision needs to be taken
  whether performance comparison should be at
  district level or at provincial level.

33

• It is better to look at all inputs from both line
  ministry and the provincial ministry for an
  accurate comparison, as the health outcomes etc.
  will be dependant on all resource inputs to a
  particular province, rather than through
  provincial health sources only. This could best
  be done at the central level.

34

• It is also recommended to review the IMMR and
  hospital returns sent to the medical statistician
  and to modify them to include additional
  information which, at present, is already
  available at institution level but not collated
  and reported.

35

• Wherever national figures are available for the
  selected indicators these should be compiled from
  them as they are more accurate and reliable.
  Since the numerator and the denominators used
  will be same and for comparison across the
  districts.

36
RECOMMENDATIONS
37

• A system for regular monitoring of health system
  performance should be established.
• Provincial health authorities need to adopt this
  model for their monitoring purposes use this at
  review meetings with the district health
  authorities
• For this purpose planning cells of PDHS/DPDHS
  offices has to be strengthened.
• Timely accurate electronic data flow to the
  DPDHS offices from the periphery for compiling
  the selected indicators should be established.

38

• PDHSs should provide expenditure data to DPDHs
  for monitoring purposes.
• Expenditure on drugs by individual institutions
  should be monitored at DPDHS level.
• These data bases from the DPDHS offices should be
  linked with the PDHSs planning units the MDPU
  of the MoH
• For the present the responsibility of M E of
  Health System Performance could be a joint effort
  of both organizations MDPU and the AHF
  secretariat. This responsibility should be
  transferred to the D/I of the MDPU of the MoH
  after AHF secretariat cease to function after the
  project period.

39

• It is to be noted that in other countries too
  most of the health status indicators are
  calculated by a central organization for
  consistency.
• It is recommended that to get data for the
  second tier - responsiveness and access, as well
  as for NCDs, including the risk factor prevalence
  a national health survey is carried out every
  three to four years, depending on the resource
  availability, or alternatively, to look at the
  feasibility of combining this with DHS survey to
  cut down costs to the health ministry.