Post Introduction Evaluation

1

• Post Introduction Evaluation
• in Tanzania
• Presentation to the NESI Training Meeting,
  Nairobi, 9 11 December 2002
• Dr. Rose Macauley
• WHO/AFRO

2
Objectives

• To document the countrys experience with the
  process of introducing the new vaccine
• Review impact the introduction process has had on
  the routine program
• Identify constrains and challenges
• Propose recommendations as deemed necessary

3
Methodology

• 3 teams of national and international evaluators
  undertook the evaluation
• Standardized questionnaire was administered to
  key informants
• 2 regions (Mbeya and Killimanmjaro),
  Dar-es-Salaam Urban and Zanzibar were visited
• 2-3 districts in each region, 19 health
  facilities, 2 central medical stores and 5
  districts cold stores

4
Components of EPI evaluated

• Process
• Impact on routine program
• Immunization safety
• Vaccine management
• Cold chain
• Storage
• Reporting completeness and timeliness
• Advocacy, social mobilization
• Monitoring and Supervision

5
Findings Introduction Process

• Smooth transition (simple substitution for DTP
• Well accepted by the general public (no
  rejections or opposition)
• Health workers trained prior to introduction
• Health workers sometimes unclear what to do with
  children who have received 1st or 2nd dose of
  DTP
• High level advocacy through the media

6
Findings Vaccine management

• Adequate cold store/room available at all levels
• Vaccine properly arranged in all cold rooms
• No DTP found in the field
• All fridges set at 4 to 8 C and 2 to 8C
  respectively on the Mainland and Zanzibar
• No expired vaccines seen
• Daily temperature recording including weekends

7
Findings Vaccine management

• Good knowledge of VVM among health workers
• Multi-dose vial policy implemented nationwide
• Freeze watch monitors and monitoring cards
  available at all levels
• No guidelines for vaccine management
• No mechanism in place for monitoring and
  reporting vaccine wastage

8
Findings Introduction process

• DTP-Hep B (tetravalent) nationwide on the same
  day
• Immediate switch to new vaccine without catch-up
• National policy for remaining DTP stock was to be
  sent back to the regional stores
• No DTP vaccines were found at health facilities
  at the time of PIE
• Training for all health workers was completed
  prior to introduction
• Health workers felt well prepared
• Introduction carried out as planned

9
Findings Effects traditional EPI vaccines

• There has not been any significant effect on the
  traditional EPI vaccines delivery
• No perceived increase or decrease on service
  demand
• No negative impact on traditional vaccines
• Has provided opportunity for additional training
  for health workers

10
Findings Cold chain

• Cold chain capacity was adequate to accommodate
  new vaccines
• Refrigerators are needed for newly established
  fixed post and to replace aged ones
• Vaccines were stored at the correct temperature
  (4 to 8 on the mainland and 2 to 8 in
  Zanzibar)
• No frozen vaccines seen during evaluation
• In Zanzibar, vaccine is stored with all other
  medical supplies
• No cold chain monitors cards, no freeze watch at
  all levels

11
Findings Vaccine wastage

• Vaccine wastage is not recorded or reported,
  however vaccinators are conscious about vaccine
  wastage
• The HMIS reporting forms have no provisions for
  reporting vaccine wastage
• MDVP is implemented to minimize vaccine wastage
• Frequency of vaccination for BCG and measles are
  reduced to reduce wastage in some facilities

12
Findings Training

• All health workers were trained prior to the
  introduction, however some would have like more
  time for the training
• On-going training is in the context of
  supervision
• Reference materials EPI manual, DTP-HepB
  booklets and injection safety materials were
  available at all health facilities on the
  mainland, but not in Zanzibar

13
Findings Monitoring AEFI

• No systematic method to document AEFI
• Some kind of forms exist at some health
  facilities for reporting AEFI, although reporting
  is not mandatory
• No AEFI reported including abscesses since the
  introduction of the new vaccines
• Perception is that side effects are seen much
  less since the introduction
• Health workers confident that they would be
  informed had AEFI were occurred

14
Findings Immunization safety

• National training materials (posters and
  storybooks) material but not available at all
  facilities
• ADs are used for all immunization injections in
  Zanzibar but the ADs of BCG do not function on
  the mainland
• Adequate supply of safety boxes available at all
  health facilities
• Fair number of incinerators installed at district
  level (district hospital)
• No used syringes or needles were seen lying
  around health facilities
• Burning in pit is the most common method for
  disposal of used needles and syringes

15
Findings Supervision

• Supervision from regional to district and
  district to health facility levels is said to be
  carried out monthly,
• No supervisory record to allow appropriate
  follow-ups at all levels
• Supervision did not change on account of the
  introduction of the new vaccines
• Feedbacks to the lower levels are sent only after
  the annual evaluation

16
Findings Coverage

• Health workers feel that, the new vaccine has
  increased the overall attendance at the
  immunization clinics.
• At the National level no substantial increase has
  been noted.
• Health workers at the lower level are not keen in
  knowing their performance coverage.
• Health facility staff are not interested in
  knowing their monthly targets. Their interest is
  just to immunize the children.
• The monthly performance charts are not being used
  in most of the health facilities.

17
Findings Reporting

• Reports are due at the various levels quarterly
  on the mainland and monthly in Zanzibar
• Data not analyzed for action at the point of
  generation
• Feedback not provided from reports
• Computerized reporting begins at the regional
  level
• All health workers knew how to calculate coverage
  and performance monitor charts were displayed at
  all health facilities

18
Key recommendations

• Update immunization forms (tally sheets and MCH
  etc) to include DTP-Hep B
• A delicate transportation should be made
  available to ensue timely and regular delivery of
  vaccines
• The problem with BCG syringe should be address as
  matter of urgency

19
Key recommendations

• Ensure proper buffer stocks of vaccines are
  maintained at regional district and health
  facility levels
• Place refrigerators in new health facilities to
  enable them to provide immunization services
• Established a mechanism to monitor and report
  vaccine wastage
• Make IEC materials (posters, pamphlets) for
  immunization available targeting various
  audiences (health workers, caretakers, policy
  makers).
• A back-up generator should be provided for the
  district vaccine store in Dar-es-Salaam.

20
Thank you!