Managing for Results: a TOT for the LQAS Approach to Household Surveys

1
Managing for Results a TOT for the LQAS
Approach to Household Surveys

• Joseph J. Valadez, PhD, MPH, ScD
• La Rue K. Seims, MA, MPH
• Corey Leburg, MHS

2
Purpose of the LQAS TOT

• Train participants to train trainers of CORE PVO
  staff to conduct household surveys to collect
  data for establishing baselines and for regular
  monitoring of community programs
• Train participants to train trainers of CORE PVO
  staff to analyze LQAS data to identify priorities
  for improving program coverage

3
Skills to be Learned

• How to Use Tested and Proven Methods for LQAS
  Training
• Data Tabulation and Analysis for Program
  Improvement
• LQAS Sampling Methods and Statistics Behind the
  Method

4
What is LQAS?

• A sampling method that
• Can be used locally, at the level of a
  supervision area, to identify priority areas or
  indicators that are not reaching average coverage
  or an established benchmark
• Can provide an accurate measure of coverage or
  health system quality at a more aggregate level
  (e.g. program area)

5
A
Assume a program area that has 7 supervision
areas Each one is supervised by one person Each
one has between 25-35 promotors/communities to
supervise
B
C
D
E
F
G
6
A
Good
B
C
D
E
Below Average or Established Benchmark
F
G
7
Maintain the program at the current level
Good
Identify Supervisors and Health Workers that can
help other Health workers improve their
performance
Identify the reasons for program problems
Below Average or Established Benchmark
Develop targeted solutions
8
How LQAS Compares to Other Sampling Methods

• Simple Random Sampling
• LQAS provides a method for prioritizing local
  areas by indicator - unlike simple random
  sampling
• Both provide coverage proportion for program area
  similar
• Sample size requirements are similar for the
  program area
• Cluster Sampling
• LQAS provides a method for prioritizing local
  areas by indicator - unlike cluster sampling
• Both provide coverage proportions for a program
  area
• Sample size is smaller for LQAS (95 vs. 300)

9
(No Transcript)
10
Marble Exercises to Demonstrate

• Random Sampling
• Non-Random Sampling
• Using a Sample Size of 19

11
Why Sample?

• Sampling allows you to use the few to describe
  the whole, and this 
• Saves time  
• Saves money

12
(No Transcript)
13
(No Transcript)
14
(No Transcript)
15
What a Sample of 19 Can Tell Us

• Good for setting priorities within an SA
• Good for setting priorities among supervision
  areas with large differences in coverage
• Good for deciding what are the higher performing
  supervision areas to learn from
• Good for deciding what are the lower performing
  supervision areas
• Good for identifying knowledge/practices that
  have high coverage from those of low coverage

16
What a Sample of 19 Cannot Tell Us

• Not good for calculating exact coverage in an SA
  (but can be used to calculate coverage for an
  entire program)
• Not good for setting priorities among supervision
  areas with little difference in coverage

17
Why Use a Sample of 19

• A sample of 19 provides an acceptable level of
  error for making management decisions at least
  92 percent of the time, it identifies whether a
  coverage benchmark has been reached or whether an
  SA is below the average coverage of a program
  area
• Samples larger than 19 have practically the same
  statistical precision as 19. They do not result
  in better information, and they cost more.

18
Identifying Locations for Interviews

• Step 1. List communities and total population
• Step 2. Calculate the cumulative population
• Step 3. Calculate the sampling interval
• Step 4. Choose a random number
• Step 5. Beginning with the random number, use the
  sampling interval to identify communities for the
  19 sets of interviews

19
(No Transcript)
20
(No Transcript)
21
(No Transcript)
22
(No Transcript)
23
(No Transcript)
24
(No Transcript)
25
(No Transcript)
26
Module ThreeWhom Should I Interview?

• Session 1 Selecting Households
• Sesssion 2 Selecting Informants
• Session 3 Field Practical for Numbering and
  Selecting Households

27
(No Transcript)
28
(No Transcript)
29
(No Transcript)
30
(No Transcript)
31
(No Transcript)
32
Process for Field Practical

• 1. Meet with community leader.
• 2. Revise and/or create community map.
• 3. Subdivide com. into sections lt30 households.
• 4. Give each section a number.
• 5. Select a section using a random number.
• 6. Repeat 3-5 if the selected section still too
  large.
• 7. Assign numbers to households in selected
  section.
• 8. Select a starting household using random
  number.
• 9. Identify the next nearest household.

33
Common Respondent Types

• Women, 15-49, non-pregnant
• Men, 15-49
• Mothers with children 0-11 months
• Mothers with children 12-23 months

34
(No Transcript)
35
Parallel Sampling

• Separate questionnaires are used for different
  respondent types in the same households
• The same person can be interviewed if they fit
  the criteria for different respondent types
  except when data are aggregated for different
  respondent types in the analysis

36
(No Transcript)
37
Household Composition Scenarios - Examples

• Household 1
• Mother 18 yrs. Old with child 24 months
• Father 26 years
• Household 2
• Man 65 years
• Mother of 15 month old absent in field nearby
  might be pregnant
• 15 month old baby
• Father in city

38
Module FourWhat Questions Do I Ask and How
Should I Ask Them?

• Session 1 Reviewing the Survey Questionnaires
• Session 2 Interviewing Skills
• Session 3 Field Practical for Interviewing
• Session 4 Planning for the Data Collection/Survey

39
GO TO TABULATION TABLES IN EXCEL

• Alt-Tab

40
After the Baseline Define Program Goals and
Annual Targets
BASELINE
Yr. 1
Yr. 2
Yr. 3
Yr. 4
10
30
50
70
80
PROGRAM GOALS FROM BASELINE UNTIL YEAR 4 OF THE
PROJECT
IMPROVEMENT
41
Example Using LQAS Data for Supervision Area
Decision Making
42
(No Transcript)
43
(No Transcript)
44
Number of Mothers with Children 0-11 Months With
Substandard Knowledge or Health Practices
According to LQAS Thresholds and Decision Rules
13
13
14
10/17/00
45
Questions to Consider When Monitoring Programs

• How often should I monitor?
• Once Every 6 Months
• Once a Year
• End of Project only
• How should I collect the data
• Should I stop work for 3 days and collect
  monitoring data?
• Should I visit a household while I am in the
  village doing normal work? At the end of 1 month
  or so I have enough data.

46
Baseline Survey Report Format

• Summary
• Program Overview (location, objectives,
  activities, beneficiaries, etc.)
• Purpose of Baseline Survey and Methodology
• Main Findings Priorities by SA Program Area
• Action Plans and Goals/Benchmarks for Key
  Indicators
• Conclusions and Recommendations
• Appendix Summary Tabulation Tables

47
How LQAS Works

• LQAS uses binomials to determine whether a
  supervision area has reached a performance
  benchmark
• Pa n! paq n-a
• a! (n-a)!

48
n! a! (n-a)!
Number of Times an Even Occurs in a Sample
3! 3x2x1 3 2! (3-2)! 2x1x1
H H H H H T H T H T H H
T T T T T H T H T H T T
49
Using Cumulative ProbabilitiesUpper end of
Triage
50
Using Cumulative ProbabilitiesLower end of
Triage
51
(No Transcript)
52
Producer Risk
Consumer Risk
53
(No Transcript)
54
(No Transcript)
55
(No Transcript)
56
Identifying Pockets of Risk
Mean 71.5
80
90
85
80
55
85
95
55
90
50
50
95
75
45
45
65
57
(No Transcript)
58
Why Use a Sample of 19 ?
Larger samples do not provide that much more
information
Sample of 28 16 adequate municipalidades 16
sub-standard municipalidades X .039 error X
.044 error 0.624 or 1 misclassified 0.704 or 1
misclassified as poor as adequate Sample of
19 16 adequate municipalidades 16 sub-standard
municipalidades X .068 error X .084
error 1.08 or 1 misclassified 1.344 or 1
misclassified as poor as adequate
59
How to Calculate Coverage Proportions with LQAS
Data

• How many Supervision Areas do I need to calculate
  accurate coverage estimates?
• Do I weight or do I not weight?
• How much error is acceptable?

60
(No Transcript)
61
Costs of LQAS
62
Traditional LQAS Compared with GPV Approach

• WHO approach main purpose
• population based survey with known confidence
  interval
• Supervision Area assessment is secondary
• Traditional LQAS main purpose
• Accurate assessment of Supervision Areas
• Catchment Area assessment is secondary

63
WHO Approach Example 1

• Choose the desired confidence interval 7
• Choose the desired level of confidence 95
• Calculate needed total sample size n196
• Count of SAs 5
• Divide sample by of SAs to determine lot sample
  size 196/5 39
• This is a lot of work in each SA (p80,
  alpha.03 .p50, beta.01)

64

WHO Approach Example 2

• Choose the desired confidence interval 10
• Choose the desired level of confidence 95
• Calculate needed total sample size n96
• Count of SAs 13
• Divide sample by of SAs to determine lot sample
  size 96/13 7
• The error to classify each SA is too high (p80,
  alpha.14 p50, beta.23)

65
Refer to BASICS New Publication

• See The Series of LOT Samples
• Review Alpha and Beta Errors