Welfare Reform Research Issues

1
Welfare Reform Research Issues

• National Association of Child Advocates
  Conference
• Atlanta, Georgia

2
Research Forum

• Created in January 1997
• Monitors current and future research developments
• Promotes additional rigorous policy relevant
  research
• Maintains Research Database
• www.researchforum.org

3
Issues Related to Research

• Relevance
• Timeliness
• Accessibility
• Quality

4
Issues Related to Research

• Relevance- how well hypotheses relate to
  advocates concerns regarding
• Income
• Health Care
• Child Care

5
Issues Related to Research

• Timeliness- how likely findings will be available
  when needed
• -time lags due to methodological issues
• -overlap with legislative/budget cycles
• -value of earlier studies

6
Issues Related to Research

• Accessibility- how well findings can be
  understood and used
• -quantity of findings
• -need for syntheses

7
Issues Related to Research

• Quality- how the research methods and design
  advance our knowledge
• -design
• -scale
• -validity
• -response rate/attrition
• -implementation

8
Types of Research Studies

• Impact Studies
• Randomized Experiment
• Quasi-Experiment
• Implementation/Process Study
• Descriptive/Analytical Study

9
Impact Study

• Aims to detect the effects of policies or
  programs on particular outcomes
• Compare one or more groups exposed to a
  particular policy with unexposed group

10
Impact Study with Controlled Experiment

• Random assignment of participants to different
  types of programs
• Strong internal validity
• Expensive to conduct
• May not reflect real world situations
• Ethical issues related to withholding certain
  services

11
Examples of Experimental Studies

• Minnesota Family Investment Program Evaluation
• Canada Self-Sufficiency Project
• New Hope Evaluation
• MDRC Next Generation Project- synthesis of
  findings from experimental studies

12
Impact Study with Quasi-Experimental Design

• Ability to compare groups exposed and not exposed
  to the intervention
• Non-equivalence of program participants and
  control group members limit internal validity
• Quality depends upon choice of control group and
  number of time periods observed

13
Examples of Quasi-Experimental Studies

• Many of the State Child Welfare Waiver Studies
• The National Survey of American Families

14
Implementation/Process Study

• Increasingly important in evaluating social
  policies and programs
• Track how well components of a program are put in
  place
• Few agreed upon standards for good implementation
  studies

15
Descriptive/Analytical Study

• Designed to report on variations in program
  content and outcomes
• May use qualitative methods to examine in-depth
  information about program participants
• Useful in understanding findings from larger
  experimental studies
• Often small scale and non-representative thus
  difficult to generalize

16
Types of Research Data Sources

• Administrative Data
• -Data originally collected for administrative
  purposes
• -Include caseload counts, child support records,
  unemployment insurance, etc.
• -Comparisons across states may be limited

17
Types of Research Data Sources

• Survey Data
• -Allow for the reliable measurement of variables
  and characteristics at a single point in time or
  across time
• -Indicator studies measure characteristics of a
  particular population

18
Quality of Survey Research

• Response Rate
• -Refers to proportion of eligible persons
  interviewed
• -Higher response rates for in-person interviews
  than for telephone or mail surveys
• -Many state welfare leaver studies had low
  response rates

19
Quality of Survey Research

• Attrition
• -Refers to loss of sample members in follow-up
  over time
• -Problematic in longitudinal research where
  participants are interviewed at multiple time
  points

20
Research Forum Database

• www.researchforum.org
• 55 large scale, multi-site, reviewed projects
• 174 smaller, unreviewed projects
• One page summaries of findings on key welfare
  research topics
• One-page abstracts of research projects

21
Issues in Methodology

• Role of advocates in helping to formulate
  research questions
• Need for more collaboration and
  cross-disciplinary exchange between researchers
• Need for development of methods to assess program
  implementation

22
Conclusion

• Relevance of particular research topics
• Timing of studies undertaken
• Quality of Research
• Accessibility of findings to a broader audience