Sudarno Sumarto

1
Local Monitoring System in Indonesia With
Special Reference to the BKKBN System

• Sudarno Sumarto
• The SMERU Research Institute
• Jakarta - Indonesia
• www.smeru.or.id

2
Background

• Efforts to monitor social impacts and development
  projects in Indonesia have traditionally relied
  on the national socio-economic survey (SUSENAS )
  or village potential census (PODES).
• The economic crisis in 1997 forced Indonesia to
  establish a social safety net program (SSN) for
  the first time in its history.
• SUSENAS and PODES were unable to respond to the
  quickly-changing crisis environment. By the time
  survey results came out, they were already
  obsolete.

3
Background(continued)

• In addition, both SUSENAS PODES can not be used
  to identify eligible individual beneficiaries.
• To meet the need for micro (household) level
  data, the government turned to BKKBN data.

4
Areas and Major Programs of SSN
Area Program Description and Benefits Targeting FY 1998/99 FY 1999/00
Food Security OPK program sale of subsidized rice to targeted households. Eligible households can purchase 10-20 kg of rice at Rp. 1,000/kg (market price is Rp. 2,500 3,000/kg) Geographic None None
Food Security OPK program sale of subsidized rice to targeted households. Eligible households can purchase 10-20 kg of rice at Rp. 1,000/kg (market price is Rp. 2,500 3,000/kg) Household BKKBN list BKKBN list with flexibility
Community Empowerment PDM-DKE a community fund program that provides block grants directly to villages for either public works or revolving credit funds. Geographic Pre-crisis data Updated with regional data
Community Empowerment PDM-DKE a community fund program that provides block grants directly to villages for either public works or revolving credit funds. Household Local decision Making Local decision making
Employment Creation Padat karya a loose, uncoordinated collection of several labor intensive programs in various government departments. Geographic None, various Ministries Urban areas, based on employment
Employment Creation Padat karya a loose, uncoordinated collection of several labor intensive programs in various government departments. Household Weak self selection Self selection
Education Scholarships and block grants providing Scholarships of Rp. 10,000/month for elementary (SD) students, Rp. 20,000/month for lower secondary (SLTP) students, and Rp. 30,000/month for upper secondary (SMU) students Block grants to selected schools Geographic Old data on enrollment Poverty data updated to 1998
Education Scholarships and block grants providing Scholarships of Rp. 10,000/month for elementary (SD) students, Rp. 20,000/month for lower secondary (SLTP) students, and Rp. 30,000/month for upper secondary (SMU) students Block grants to selected schools Household School Committees applying criteria School committees applying criteria
Health JPS-BK a program providing subsidies for Medical services Operational support for health centers Medicine and imported medical equipment Family planning services Nutrition (supplementary food) Midwife services Geographic BKKBN pre- prosperous rates Pre-prosperous rates updated to 1999
Health JPS-BK a program providing subsidies for Medical services Operational support for health centers Medicine and imported medical equipment Family planning services Nutrition (supplementary food) Midwife services Household BKKBN list BKKBN list with flexibility
5
Inventory of Monitoring Efforts during the Crisis

• CRPs NGO Mapping
• SMERUs Kecamatan Crisis Impact Survey
• SMERUs Community-based Monitoring
• BPS SUSENAS
• BPS Unicefs 100 Village Survey
• RANDs IFLS2

6
BKKBN (National Family Planning Coordinating
Agency) Background

• It was established in 1970 to monitor the FP
  (Family Planning) program.
• The first national FP survey began in 1985 and
  has been conducted annually ever since.
• Family welfare and demographic characteristics
  sections were added starting in 1994.

7
BKKBN Family Welfare Section

• BKKBN maintains a list of every family in the
  country with an indicator of welfare status
  based on a mix of variables (able to eat twice a
  day change of clothing able to fulfill
  religious obligations).
• Overall there are 23 welfare indicators.
• The indicators are used to categorize families
  into 5 welfare status (pre-prosperous just
  prosperous prosperous 2 prosperous 3
  prosperous 3 plus).

8
BKKBN Family Welfare Section(continued)

• In addition to the SSN program, other programs
  that have used the data from the BKKBN family
  welfare
• Family Savings Program (Takesra)
• Family Credit Program (Kukesra)
• National Foster Parents Movement (GN-OTA)
• Medical assistance for the poor
• Housing assistance for the poor

9
BKKBN Data Gathering System
10
Data Collection Reporting Process
11
Strengths of BKKBN Data

• The only micro data in Indonesia that includes
  data from almost every household.
• As of 2002, the data cover 197.5 million
  individuals (total population 210 million).
• Collected by locals, which ensures its accuracy
  and comprehensiveness.
• In 2000, about 1 million cadre groups conducted
  the survey, assisted by 35,000 FP supervisors.

12
Weaknesses of BKKBN Data

• Even with computerization starting in 2001,
  mistakes are unavoidable due to the large amount
  of data.
• High variation in the ability of enumerators
  affects consistency and quality.
• Local subjectivity could play a role.
• The data do not capture transitory shocks to
  income.

13
Weaknesses of BKKBN Data (continued)

• Questionable subjective welfare criteria.
• Composition of data is susceptible to changes by
  local government officials.
• Suffers from lack of funding.

14
How Accurate is BKKBN Data?

• Comparison with consumption-based measures of
  household welfare at the district level in 3
  provinces

Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators Spearman Rank Correlations of Districts Poverty Headcount Based on SMERU Poverty Mapping Results with BKKBN Welfare Indicators
Province BKKBN BKKBN BKKBN BKKBN
Province Pre-prosperous (economic) Pre-prosperous Pre-Prosperous Prosperous 1 (economic) Pre-Prosperous Prosperous 1
East Java 0.6932 0.6188 0.7390 0.7394
N 37        
North Sumatra 0.4421 0.4018 0.7579 0.7491
N 19        
South Sulawesi 0.3992 0.5326 0.4279 0.5958
N 23        
Note Significant at 1 percent level Significant at 5 percent level Note Significant at 1 percent level Significant at 5 percent level Note Significant at 1 percent level Significant at 5 percent level Note Significant at 1 percent level Significant at 5 percent level Note Significant at 1 percent level Significant at 5 percent level
15
How Accurate is BKKBN Data? (continued)

• Comparison with consumption-based measures of
  household welfare at the household level using
  the 100 Village Survey (SSD)

Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (20 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (20 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (20 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (20 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (20 Poor)
SSD SSD SSD
Non-Poor Poor Total
BKKBN Welfare Category Prosperous 5,197 920 6,117
BKKBN Welfare Category Row 85 15 100
BKKBN Welfare Category Column 54.2 38.3 51
BKKBN Welfare Category Pre-Prosperous 4,400 1,480 5,880
BKKBN Welfare Category Row 74.8 25.2 100
BKKBN Welfare Category Column 45.9 61.7 49
BKKBN Welfare Category Total 9,597 2,400 11,997
BKKBN Welfare Category Row 80 20 100
BKKBN Welfare Category Column 100 100 100
16
How Accurate is BKKBN Data? (continued)
Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (49 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (49 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (49 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (49 Poor) Cross-tabulation between BKKBNs Welfare Indicator with SSDs Non-Poor/Poor (49 Poor)
SSD SSD SSD
Non-Poor Poor Total
BKKBN Welfare Category Prosperous 3,585 2,532 6,117
BKKBN Welfare Category Row 58.6 41.4 100
BKKBN Welfare Category Column 58.7 43 51
BKKBN Welfare Category Pre-Prosperous 2,523 3,357 5,880
BKKBN Welfare Category Row 42.9 57.1 100
BKKBN Welfare Category Column 41.3 57 49
BKKBN Welfare Category Total 6,108 5,889 11,997
BKKBN Welfare Category Row 51 49 100
BKKBN Welfare Category Column 100 100 100
17
How Accurate is BKKBN Data? (continued)

• The conclusion is that although BKKBN is quite
  accurate at the district-level, it misses the
  target at the household-level.
• There is quite a large discrepancy between BKBBN
  welfare classification and welfare measured by
  consumption at the household level.
• Since interventions are aimed at households, this
  means using BKKBN for such interventions is
  insufficient.

18
Concluding Remarks

• Targeting is not as easy as it is often
  suggested.
• The BKKBN monitoring system was the most widely
  used targeting tool as it is the most
  comprehensive CBM system in the country.
• At the household level, there is a large degree
  of discrepancy between BKKBN welfare
  classification and consumption-based welfare
  classification.
• In order to remedy this discrepancy, a new
  scoring system or a composite index needs to be
  explored by combining a number of BKKBNs 23
  welfare variables with other variables.