Title: Managing for Results: a TOT for the LQAS Approach to Household Surveys
1Managing for Results a TOT for the LQAS
Approach to Household Surveys
- Joseph J. Valadez, PhD, MPH, ScD
- La Rue K. Seims, MA, MPH
- Corey Leburg, MHS
2Purpose of the LQAS TOT
- Train participants to train trainers of CORE PVO
staff to conduct household surveys to collect
data for establishing baselines and for regular
monitoring of community programs - Train participants to train trainers of CORE PVO
staff to analyze LQAS data to identify priorities
for improving program coverage
3Skills to be Learned
- How to Use Tested and Proven Methods for LQAS
Training - Data Tabulation and Analysis for Program
Improvement - LQAS Sampling Methods and Statistics Behind the
Method
4What is LQAS?
- A sampling method that
- Can be used locally, at the level of a
supervision area, to identify priority areas or
indicators that are not reaching average coverage
or an established benchmark - Can provide an accurate measure of coverage or
health system quality at a more aggregate level
(e.g. program area)
5A
Assume a program area that has 7 supervision
areas Each one is supervised by one person Each
one has between 25-35 promotors/communities to
supervise
B
C
D
E
F
G
6A
Good
B
C
D
E
Below Average or Established Benchmark
F
G
7Maintain the program at the current level
Good
Identify Supervisors and Health Workers that can
help other Health workers improve their
performance
Identify the reasons for program problems
Below Average or Established Benchmark
Develop targeted solutions
8How LQAS Compares to Other Sampling Methods
- Simple Random Sampling
- LQAS provides a method for prioritizing local
areas by indicator - unlike simple random
sampling - Both provide coverage proportion for program area
similar - Sample size requirements are similar for the
program area - Cluster Sampling
- LQAS provides a method for prioritizing local
areas by indicator - unlike cluster sampling - Both provide coverage proportions for a program
area - Sample size is smaller for LQAS (95 vs. 300)
9(No Transcript)
10Marble Exercises to Demonstrate
- Random Sampling
- Non-Random Sampling
- Using a Sample Size of 19
11Why Sample?
- Sampling allows you to use the few to describe
the whole, and this - Saves time
- Saves money
12(No Transcript)
13(No Transcript)
14(No Transcript)
15What a Sample of 19 Can Tell Us
- Good for setting priorities within an SA
- Good for setting priorities among supervision
areas with large differences in coverage - Good for deciding what are the higher performing
supervision areas to learn from - Good for deciding what are the lower performing
supervision areas - Good for identifying knowledge/practices that
have high coverage from those of low coverage
16What a Sample of 19 Cannot Tell Us
- Not good for calculating exact coverage in an SA
(but can be used to calculate coverage for an
entire program) - Not good for setting priorities among supervision
areas with little difference in coverage
17Why Use a Sample of 19
- A sample of 19 provides an acceptable level of
error for making management decisions at least
92 percent of the time, it identifies whether a
coverage benchmark has been reached or whether an
SA is below the average coverage of a program
area - Samples larger than 19 have practically the same
statistical precision as 19. They do not result
in better information, and they cost more.
18Identifying Locations for Interviews
- Step 1. List communities and total population
- Step 2. Calculate the cumulative population
- Step 3. Calculate the sampling interval
- Step 4. Choose a random number
- Step 5. Beginning with the random number, use the
sampling interval to identify communities for the
19 sets of interviews
19(No Transcript)
20(No Transcript)
21(No Transcript)
22(No Transcript)
23(No Transcript)
24(No Transcript)
25(No Transcript)
26Module ThreeWhom Should I Interview?
- Session 1 Selecting Households
- Sesssion 2 Selecting Informants
- Session 3 Field Practical for Numbering and
Selecting Households
27(No Transcript)
28(No Transcript)
29(No Transcript)
30(No Transcript)
31(No Transcript)
32Process for Field Practical
- 1. Meet with community leader.
- 2. Revise and/or create community map.
- 3. Subdivide com. into sections lt30 households.
- 4. Give each section a number.
- 5. Select a section using a random number.
- 6. Repeat 3-5 if the selected section still too
large. - 7. Assign numbers to households in selected
section. - 8. Select a starting household using random
number. - 9. Identify the next nearest household.
33Common Respondent Types
- Women, 15-49, non-pregnant
- Men, 15-49
- Mothers with children 0-11 months
- Mothers with children 12-23 months
34(No Transcript)
35Parallel Sampling
- Separate questionnaires are used for different
respondent types in the same households - The same person can be interviewed if they fit
the criteria for different respondent types
except when data are aggregated for different
respondent types in the analysis
36(No Transcript)
37Household Composition Scenarios - Examples
- Household 1
- Mother 18 yrs. Old with child 24 months
- Father 26 years
- Household 2
- Man 65 years
- Mother of 15 month old absent in field nearby
might be pregnant - 15 month old baby
- Father in city
38Module FourWhat Questions Do I Ask and How
Should I Ask Them?
- Session 1 Reviewing the Survey Questionnaires
- Session 2 Interviewing Skills
- Session 3 Field Practical for Interviewing
- Session 4 Planning for the Data Collection/Survey
39GO TO TABULATION TABLES IN EXCEL
40After the Baseline Define Program Goals and
Annual Targets
BASELINE
Yr. 1
Yr. 2
Yr. 3
Yr. 4
10
30
50
70
80
PROGRAM GOALS FROM BASELINE UNTIL YEAR 4 OF THE
PROJECT
IMPROVEMENT
41Example Using LQAS Data for Supervision Area
Decision Making
42(No Transcript)
43(No Transcript)
44Number of Mothers with Children 0-11 Months With
Substandard Knowledge or Health Practices
According to LQAS Thresholds and Decision Rules
13
13
14
10/17/00
45Questions to Consider When Monitoring Programs
- How often should I monitor?
- Once Every 6 Months
- Once a Year
- End of Project only
- How should I collect the data
- Should I stop work for 3 days and collect
monitoring data? - Should I visit a household while I am in the
village doing normal work? At the end of 1 month
or so I have enough data.
46Baseline Survey Report Format
- Summary
- Program Overview (location, objectives,
activities, beneficiaries, etc.) - Purpose of Baseline Survey and Methodology
- Main Findings Priorities by SA Program Area
- Action Plans and Goals/Benchmarks for Key
Indicators - Conclusions and Recommendations
- Appendix Summary Tabulation Tables
47How LQAS Works
- LQAS uses binomials to determine whether a
supervision area has reached a performance
benchmark - Pa n! paq n-a
- a! (n-a)!
48 n! a! (n-a)!
Number of Times an Even Occurs in a Sample
3! 3x2x1 3 2! (3-2)! 2x1x1
H H H H H T H T H T H H
T T T T T H T H T H T T
49Using Cumulative ProbabilitiesUpper end of
Triage
50Using Cumulative ProbabilitiesLower end of
Triage
51(No Transcript)
52Producer Risk
Consumer Risk
53(No Transcript)
54(No Transcript)
55(No Transcript)
56Identifying Pockets of Risk
Mean 71.5
80
90
85
80
55
85
95
55
90
50
50
95
75
45
45
65
57(No Transcript)
58Why Use a Sample of 19 ?
Larger samples do not provide that much more
information
Sample of 28 16 adequate municipalidades 16
sub-standard municipalidades X .039 error X
.044 error 0.624 or 1 misclassified 0.704 or 1
misclassified as poor as adequate Sample of
19 16 adequate municipalidades 16 sub-standard
municipalidades X .068 error X .084
error 1.08 or 1 misclassified 1.344 or 1
misclassified as poor as adequate
59How to Calculate Coverage Proportions with LQAS
Data
- How many Supervision Areas do I need to calculate
accurate coverage estimates? - Do I weight or do I not weight?
- How much error is acceptable?
60(No Transcript)
61Costs of LQAS
62Traditional LQAS Compared with GPV Approach
- WHO approach main purpose
- population based survey with known confidence
interval - Supervision Area assessment is secondary
- Traditional LQAS main purpose
- Accurate assessment of Supervision Areas
- Catchment Area assessment is secondary
63WHO Approach Example 1
- Choose the desired confidence interval 7
- Choose the desired level of confidence 95
- Calculate needed total sample size n196
- Count of SAs 5
- Divide sample by of SAs to determine lot sample
size 196/5 39 - This is a lot of work in each SA (p80,
alpha.03 .p50, beta.01)
64 WHO Approach Example 2
- Choose the desired confidence interval 10
- Choose the desired level of confidence 95
- Calculate needed total sample size n96
- Count of SAs 13
- Divide sample by of SAs to determine lot sample
size 96/13 7 - The error to classify each SA is too high (p80,
alpha.14 p50, beta.23)
65Refer to BASICS New Publication
- See The Series of LOT Samples
- Review Alpha and Beta Errors