Diapositiva 1

1
EVALUATING WELFARE REFORM UNDER PROGRAM
HETEROGENEITY AND ALTERNATIVE MEASURES OF SUCCESS
Luis Ayala

Universidad Rey Juan Carlos / Instituto de
Estudios Fiscales Magdalena Rodríguez

Instituto de Estudios Fiscales
NEW PERSPECTIVES ON THE STUDY OF INEQUALITY,
POVERTY AND REDISTRIBUTION
2
1. INTRODUCTION (I)
Lively debate on the success of welfare reforms
- Main aim ? fostering transitions from welfare
to work - Wide array of options work-first
strategies, long-term programs
Substantial empirical evidence - Policy
changes have been much discussed -
Significant effects on employment
Some caveats in the evaluation of welfare
reforms 1. Little insight into the effects
of programs heterogeneity ? limits of
the standard binary treatment model of only two
states 2. Limited evidence on different
dimensions of economic well-being
3
1. INTRODUCTION (II)
This paper aims to present an assessment of
welfare reforms under a framework of - Program
heterogeneity - Alternative measures of success
We focus on a specific welfare program Madrids
Ingreso Madrileño de Integración (IMI) -
Standard program in the Southern European
context - Alternative design ? simultaneous
and heterogeneous sub- programs of life
labor skills - Very detailed information ?
administrative records leavers survey -
Huge range of success indicators
4
1. INTRODUCTION (III)
Structure of the paper
1.- Background 2.- IMI program 3.-
Methodology4.- Results5.- Conclusion
5
2. BACKGROUND (I)
Two central questions 1.- Nature and
economic rationale of the program under study
2.- Programs outcomes
Complexity when welfare recipients can take part
simultaneously in heterogeneous sub-programs
6
2. BACKGROUND (II) THE ECONOMIC RATIONALE
OF WELFARE REFORM
MOVING WELFARE RECIPIENTS INTO THE LABOR
MARKET 1.- Work-first strategies ? push
recipients into the labor market as rapidly as
possible 2.- Human capital strategies ?
intensive training and educational opportunities
for recipients
NO CONCLUSIVE EVIDENCE - Some estimates put
into question the traditional view -
Sensitivity to the time period considered

• PROGRAM HETEROGENEITY?
• Very limited evidence
• - European case ? only ALMP (Sianesi, 2001
  Lechner, 2002)

7
2. BACKGROUND (III) ALTERNATIVE OUTCOMES
1.- ARE LABOR INDICATORS THE RELEVANT OUTCOME?
-Intermediate goal for achieving higher levels
of economic well-being -Sensitivity of the
results to the outcome variable (Cancian Meyer)

2.-INCOME POVERTY - Somewhat contrasting
evidence for the U.S. - Studies of welfare
leavers ? subject to data availability

• 3.- MATERIAL WELL-BEING
• Very limited correlation with income poverty
• - There is no clear evidence of improvements due
  to welfare reform

4.- WELFARE INDEPENDENCE - Very limited
empirical evidence ? lack of longitudinal data
5.- SOCIAL DIFFICULTIES - Serious problems for
defining the relevant conditions
8
3. THE IMI PROGRAM (I) INSTITUTIONAL FEATURES

• Institutional features of the IMI program
• - average program within the set of regional
  schemes
• - cash benefits / insertion measures
• - elegibility conditions ? all households (age
  limits)
• - empirical evidence on durations ? employability
  is one of the main determining factors
  (heterogeneity)

• Insertion activities
• Insertion contracts with welfare agencies ?
  every recipient
• should be engaged at least in one program
  (multiple states)
• - Overall actions developed to guarantee the
  basic pre-conditions of
• social participation
• - Measures aimed to improve the employment
  opportunities of recipients

9
3. THE IMI PROGRAM (II)INSTITUTIONAL FEATURES

• Insertion measures ? life skills
• general information - general counseling
• continuous individual support - psychological
  support
• legal support - children intervention
• family mediation - group activities
• assistance for other benefits

• Insertion measures ? labor skills
• access to specific employment offers
• general job search assistance
• training
• subsidized employment
• social enterprises

10
3. THE IMI PROGRAM (II) DATA

• We match administrative records (whole history,
  1990-2001) with a specific survey of IMI leavers
  (2001)
• very detailed information
• - no endogeneity bias

• ADMINISTRATIVE RECORDS
• - 40,000 households
• specific characteristics of
• welfare populations? e.g.,
• employability, social problems

• LEAVERS SURVEY
• - 2,300 households
• previous participation in
• different sub-programs (14)
• different well-being indicators
• - socioeconomic characteristics

11
4. METHODOLOGY (I)
Do insertion activities improve economic
well-being?

• We aggregate the 14 treatments into 4 different
  and mutually exclusive groups
• 1.- non-participation in specific work-related
  sub-programs overall actions developed only to
  guarantee the basic pre-conditions of social
  participation
• 2.- participation in general labor-oriented
  activities general labor services for recipients
  
• 3.- participation in labor-intensive
  sub-programs actions more specifically targeted
  to foster transitions from welfare to work
  (subsidized employment and social enterprises)
• 4. participation in both work-related strategies
  possibility of taking part both in the second
  and third groups

12
4. METHODOLOGY (II)
Do insertion activities improve economic
well-being?

• We focus on the relative effectiveness of each
  sub-program
• 1.- effects of participation in some work-related
  scheme as compared to participate only in
  life-skills activities
• 2.- effects of participation in each one of the
  specific work-related schemes compared to
  participation in general life skills activities
• 3.- relative effectiveness of labor-oriented
  treatments

13
4. METHODOLOGY (III)
Evaluation method (standard binary model)

• - Expected treatment effect for the treated
  population
• E(Y1 Yo ?D1) E(Y1?D1) E(Yo?D1)
• Y1 outcome for participants in a program
• Y0 outcome if these individuals were not exposed
  to the treatment
• Counterfactual E(Yo?D1) ? matching
• (pairing participants with non-participants
  recipients)
• A control group among non-participants (similar
  distribution of observed variables) ? Rosenbaum
  and Rubin (1983)
• E?E?Y1?D1, p(X)? E?Yo?D0, p(X) ?D1??
• where p(X) is the propensity score.
• - A probit model to estimate the score with the
  covariates predicting participation in the
  program
• Pr ?D1 ? X? ? ?h(X)?

14
4. METHODOLOGY (IV)
Evaluation method (multiple states)

• Multiple states framework ? can we use matching
  methods?
• Individuals decide to participate in one or
  other sub-program
• - Extended model ? Lechner (2001, 2002), Sianesi
  (2001)
• - Pair-wise comparisons

• (M1) mutually exclusive sub-programs
  (treatments)
• Every welfare leaver ? one observable outcome
  ?Y0, Y1,,YM?
• Participation in a sub-program is indicated by S
  ? ?0,1,, M?
• We are interested in the effects of
  participation in sub-program (a)
• compared to participation in other sub-program
  (b)
• ?a,b0 E(Ya Yb ?Sa) E(Ya?Sa) E(Yb?Sa)
  
• A counterfactual is needed to estimate E(Yb?Sa)
  ? if the standard properties of the binary model
  hold, matching can also work
• E(Yb?Sa) ?E(Yb?Sb, Pa/ab(X))?Sa?

15
4. METHODOLOGY (V)
Evaluation method
PROPENSITY SCORES IN A MULTIPLE STATES
FRAMEWORK Lechner (2002), two different
approaches 1.- multiple discrete-choice
model, such as multinomial logit or
probit model (structural approach) 2.-
estimating all conditional probabilities between
possible pairs of choices directly
(reduced-form approach) - Selection of Xs for
balancing the different sub-samples? ? resulting
quality of the matched samples (administrative
records and leavers survey)
16
Comparison 1
Comparison 2
Comparison 3
Comparison 4
Comparison 5
Comparison 6
Comparison 7
Horizontal axis cumulative units from lowest to
highest propensity scores Vertical axis
propensity score for treated and control units
17
4. METHODOLOGY (VI)

• EMPLOYMENT
• employment dummy (to be employed)
• legal contract

• SUBJECTIVE WELL-BEING
• subjective poverty
• living standards compared to ten years ago
• inter-generational comparisons (compared to
  their parents at present/at the same age)

Outcome variables

• MATERIAL WELL-BEING
• summary measure of material hardship
• summary measure of material well-being

• SOCIAL DIFFICULTIES
• Summary measure - Housing problems - Health
  problems
• Number of problems - Family/individual -
  Economic problems

18
5. RESULTS (I)

• Relative effectiveness of the sub-programs
• Average effects for each pair-wise comparison
• Two different questions
• 1.- Do work-related sub-programs perform
  better than
• general measures aimed at improving life
  skills?
• 2.- Which work-related sub-program works best?

• a) EMPLOYMENT
• Work-related sub-programs give rise to higher
  employment rates
• Intensive employment activities have the
  highest capacity to increase employment
  opportunities (better than mixed strategies)
• Legal contract ? very general labor measures are
  not enough to guarantee a stable position in the
  labor market

19
5. RESULTS (II)

• b) SUBJECTIVE WELL-BEING
• Subjective poverty ? work-related activities
  help to reduce income poverty (intensive
  employment most efficient policy)
• Intertemporal analysis ? somewhat contrasting
  evidence
• Intergenerational comparisons ? positive effects
  (not in all cases)

• c) MATERIAL HARDSHIP
• Small average effects
• Political inferences ? low effectiveness or
  Pareto improvement?
• Very different effects across treatments ?
  intensive employment works best

• d) SOCIAL DIFFICULTIES
• Very low incidence on social problems
• Health problems are clearly reduced
• Intensive employment strategies are the only
  alternative

20
5. RESULTS (III)
1. Non-participation in a work-related scheme.
2. Participation in a work-related scheme 3.
Participation in general work-related schemes 4.
Participation in intensive work-related schemes
5. Participation in mixed work-related schemes
21
6. CONCLUSION
1.- IMI ? challenging questions for the
evaluation of welfare reform 2.- Consistent
picture of results - sensitivity to the
outcome variables - good results for
work-related sub-programs in terms of
employment and subjective poverty, few changes
regarding other dimensions - intensive
employment higher effectiveness 3.- Useful
results for policy-making 4.- Cautions ?
heterogeneity of welfare populations