Familias en Accion

1
Familias en Accion The Introduction and Abolition
of a Conditional Cash Transfer David
Phillips EDePo and IFS
2
Outline

• the Familias en acción Program A focus on
  education.
• Evaluation methodology Introduction of the
  programme.
• 3. Results of the Policy
• 4. What if the programme were abolished?
• 5. Results and methological implications
• Lessons to be learned.

3
Familias en acción (FeA)

• Familias en Accion is a Conditional Cash Transfer
  program. Its primary aim is to foster the
  accumulation of human capital.
• It gives monetary incentives for families to
  invest in childrens human capital (education and
  health)
• Similar conditional cash transfers are being
  implemented in
• Mexico, Honduras, Nicaragua, Panama, Brazil,
  Argentina, Turkey, Bangladesh
• In Colombia, the program is targeted towards the
  poorest 20 of households. It was initially
  limited to those living in semi-rural
  municipalities with fewer than 100k people, with
  enough education and health infrastructure, and
  with a bank, but is now being extended to major
  urban areas too.

4
Familias en acción (FeA)

• The program has two main components Health and
  Education
• Nutrition and health
• Eligible families with children aged 0-5 receive
  about 30US per month conditional on registering
  the children on growth development checkups
• Mothers are also encouraged to attend some
  talks with health professionals
• Education
• Eligible families with children aged 6-17
  receive a subsidy per child conditional on school
  attendance.
• All subsidies are given to the mothers. Why?

5
Educational Subsidies

• The educational component of FeA consists of a
  subsidy of about 10 per month per child enrolled
  in primary school and 20 per month per child in
  secondary school, provided their attendance is at
  least 80.
• What is the economic case for this intervention?

6
Educational Subsidies
If credit constraints were the full story then
unconditional subsidies would suffice.
Conditionality implies externalities are seen as
a key issue. Redistribution Traditional
redistribution has efficiency costs in terms of
reduced human capital investment and work
incentives. This programme redistributes to the
poor in a way that counteracts this general
effect.

7
Targeting the Programme

• How do we define the poorest 20 for the
  programme when income is often volatile and
  income data absent?
• SISBEN is a socio-economic classification system
  designed by the Colombian government which takes
  account of physical living standards (e.g whether
  they have electricity, quality of walls/roof
  etc). Eligibility is currently defined on 1999
  status.
• So what are the eligible families like?
• Average family size 7
• Average monthly consumption 150 per month (inc
  consumption in-kind)
• Food consumption is about 60 of total
  consumption.

8
Evaluating the Programme
How do you evaluate the impact of this programme
on school enrolment? We use a treatment-control
methodology where the programme is implemented in
some areas but not others.
So whats the problem? We dont observe
because those in treatment areas are only
observed in period t1 having been treated.
9
Evaluation Methodology
The ideal way to solve this problem is to
randomly allocate treatment across individuals
or, in this case, municipalities. With random
sampling we have
But, politics intervened. The Colombian
government wanted to roll out the policy wherever
the necessary infrastructure was in place, so the
controls we had were a selected
sample Typically, without banks and somewhat
poorer health/education infrastructure. So what
did we do?
10
Evaluation Methodology
We still use the control municipalities as our
counterfactual, but account for the potential
pre-existing differences in school enrolment
prior to programme implementation. We combine two
standard evaluation techniques to accomplish
this 1) Selection on Observables 2)
Difference-in-Differences (Programme effect is
the difference between treatment and control
areas minus pre-existing differences)
Underlying the evaluation is comprehensive high
quality dataset with individual, household and
area information.
11
Selection on Observables
Y a bX cD u where X are observables, D a
dummy indicating treatment and u
unobserved. If EDuX 0 (i.e. orthogonal), we
can get an unbiased estimator of the effect of
treatment by the coefficient c. We can
generalise this approach with Propensity Score
Matching. This method defines P(D1X) P(X)
the probability of being treated given your
observable characteristics, termed the
propensity score. We then estimate the impact
of the programme by comparing people with similar
propensity scores but different treatment
statuses.
12
Difference-in-Differences
Up until now we have assumed that, conditional
upon observables X, treatment and control areas
are identical. But, what if we have the model Y
a bX cD u v with EXv / 0 In
this case, we get biased estimates of the impact
of D they are confounded by the impact of v. How
do we get around this? If v is constant over
time (a fixed effect) and we have pre and
post-programme data we can use difference-in-diffe
rences method.
13
Difference in difference effect
Y
treatment
effect
Control
t1
t0
We also call the fixed effect a Common Time
Trend Assumption
14
Example diff-in-diff effect

• Before the program, school enrolment rates
• In treatment towns 60
• In control towns 50
• After the program,
• in treatment towns 80
• in control towns 60
• What is the effect of the program ?

15
Answer
16
The Data

• Sample 11,500 households in 67 treatment and 62
  control municipalities
• Baseline collected between June and November 2002
• First follow up collected between July and
  December 2003
• Second follow up collected between November 2005
  and April 2006.
• Complete household survey. Info about all
  household members. Household survey is quite
  long about three hours and a half
• Questionnaires to household heads, mothers
  etc....
• Survey to major, health care centres, schools,
  nurseries
• Good quality of data attrition 6 between the
  baseline and first follow-up. About a further 9
  between first and second follow-up.

17
Table 1. Impact of FA on percentage of children
who attend school
Statistically significantly different from zero
at the 5 significance level.
18
Impact on School Enrolment

• Higher impacts in rural areas.
• Strong significant impacts upon older children.
• No significant impact upon younger children.
• Remaining issues
• Has it affected quality of education?
• How does the effect operate?
• - our approach doesnt allow us to investigate
  credit/price channels and might reflect things
  other than response to the monetary incentive
  only.

19
Other Impacts of the Programme

• Increased Consumption
• This is increase is mainly in high quality
  foodstuffs (meat and dairy) childrens clothes
  and shoes and educational goods and services.
• No increase in alcohol, tobacco and adults
  clothes.
• Health
• Increase in attendance of health classes
• Lower risk of disease in rural areas
  (diarrhoea!)
• Increased height-for-age.

20
Summary (part 1)

• Familias en Accion is an education subsidy in
  Colombia
• It was evaluated using a difference-in-difference
  methodology comparing treatment and control
  areas.
• It had a significant positive impact on school
  enrolment for older children

Looking Ahead

• What might happen if the programme were
  abolished?
• How would we go about evaluating this without
  actually testing abolition in some areas?
• What conclusions do we draw from this work?

21
Todays political context
The Colombian government is presently expanding
the programme to cover urban areas. Doing this
will be expensive and it plans to help fund this
by abolishing the payments to children aged 11 or
under. Why? Because the results of the
evaluation showed the programme had little impact
on this group. Is this a sensible argument? Not
really, because the programme may have had
dynamic effects.
22
Dynamic effects of Familias

• The outcome (school enrolment) can still be
  affected by past-exposure to the programme. For
  instance
• the subsidy has meant older children are more
  likely to remain in school. In order to protect
  their investment in these children, who may have
  demonstrated otherwise hidden abilities, parents
  may withdraw younger children (who can go back to
  school later).
• the subsidy may have changed preferences.
  Families may have got used to extra consumption
  (and withdraw more children to continue to fund
  this) or, alternatively may have a greater
  preference for education.

23
Measurement Issues

• How could we test the impact of abolishing FeA?
• We could withdraw the programme from a
  (preferably) random selection of areas and use
  the same methods as before
• But this would be politically difficult.

Instead, we could elicit information directly
from survey respondents by asking what they
do. This is called the stated response method and
is treated with scepticism by many economists.
24
The stated-response questions
We ask respondents the following
question Suppose the government withdraws the
subsidy. Would you withdraw your child(ren) from
school? Yes or no. If respondents answer Yes
they are then required to state whether all or
some, with those stating some then asked to
indicate which children. We use the same
methodology to ask what would happen if the
subsidy were made unconditional cash transfer to
families with children.
25
Why sceptical?
We want to use our stated response as
follows Y a bX u Y stated withdrawal
decision X household, village or child
characteristic With this in mind, what kind of
problems are the most serious?
26
Measurement Error
Y a bX u

• What happens when Y is measured with different
  kinds of error?
• (1) If Eu 0 and EXu 0 (i.e. white
  noise), our estimates of a and b are unbiased but
  have greater variance.
• (2) If Y is consistently stated too high or too
  low, our constant term a is biased.
• (3) If EXu / 0 (i.e. correlation between the
  reporting errors, u and explanatory variables,
  X) then our estimates of b are biased.
• Hence we should be mainly concerned if we believe
  we have correlated errors.

27
Withdrawal Decisions
Appear reasonable withdrawal rates not large
enough to indicate significant strategic response
and higher for older/rural.
28
Withdrawals by Age
29
Withdrawals by Age
30
Checking Reliability

• In applied evaluation work its important to
  investigate the plausibility and reliability of
  your results
• Is withdrawal correlated in the expected way
  with family characteristics?
• Are results robust to changing specifications or
  samples?

31
Withdrawal Decisions
Regression shows that withdrawal is related in
the expected manner with household
characteristics Younger children Living closer
to school in urban areas With educated
mothers And high household incomes Are less
likely to be withdrawn from their school! But, a
larger number of households than expected (almost
60) say that they would withdraw all of their
children, although this might reflect large fixed
costs in sending your family to school.
32
Why so many young withdrawn?

• Look back a few slides. Notice that although
  withdrawal isnt so high as to be implausible,
  MANY more young children are being reported to be
  withdrawn than were induced to start school by
  the programme. Why?
• One possibility is intra-household effects.
• By inducing older children to remain in school,
  maybe they have greater attachment and have shown
  otherwise hidden abilities.
• Houses that are credit-constrained may
  reallocate resources to the older ones because
  the young ones can, arguably catch up.
• We do find that young children with older
  siblings are more likely to be withdrawn!

33
Why so many young withdrawn?

• But this doesnt seem to be an intra-household
  story.
• Over 8/10 of the younger children withdrawn come
  from households where all of the children are
  being withdrawn and hence no substitution
  could possibly take place.
• What could be driving our results?
• The structure of the question parents answering
  all due to inability to decide on the spot, an
  unwillingness to reveal preferences over children
  to the interviewer (and in front of children), or
  just a choice of the easy option.

34
Why so many young withdrawn?
Statistically speaking, we have problem (3)
discussed above. The presence of older children
(who parents wish to withdraw from school) is
positively correlated with measurement error in
withdrawal, and it is this correlation we
observe, not any underlying intra-household
effect. Lesson The structure of a question is
very important in its reliability. Whilst our
question seems very reliably to ascertain which
HOUSEHOLDS withdraw children, less good at
finding which children WITHIN the households
because we make the mistake of letting parents
say ALL
35
An Unconditional Subsidy?
Few respondents say they would withdraw if it
were made unconditional this indicates that
credit-constraints play a key role in low school
attendance, with externalities less important.
36
An Unconditional Subsidy?

• But this does not make a convincing case for
  unconditionality
• it would significantly increase the cost of the
  programme.
• the impact in rural areas is not insignificant.
• it might undermine support for the policy.

37
Conclusions (1)

• Economists use both selection on observables
  and difference-in-differences methods when
  completely randomised trials are unavailable.
• Familias en Accion had a significantly positive
  effect on school enrolment in Colombia.
• This was most notable in rural areas, and for
  older children
• But our methodology does not allow investigation
  of the channels credit constraints
  externalities/prices or attitudes

38
Conclusions (2)

• The impact of withdrawing a programme is not
  necessarily equal and opposite of introducing
  it.
• Stated-response methods are a potential way of
  evaluating withdrawing a successful and
  politically sensitive problem.
• However, question regimes need to be designed
  very carefully to ensure reliable response. They
  are best seen as a complement to traditional
  experimental approaches.
• Responses to our questions
• - possibility of dynamic programme effects
• - the importance of credit-constraints as
  channel.
• Whilst not directly comparable, Colombia should
  be cautious about removing subsidies for young
  children.

39
References
Fitzsimons, Phillips and Vera-Hernandez (2007)
What would you do? An investigation of
stated-response data IFS/EDePo working paper
http//www.ifs.org.uk/edepo/wps/ewp0701.pdf Thoma
s, Beegle et al (2004) Education in a crisis,
Journal of Development Economics, Vol. 74, pages
53 - 85