OUTCOME BASED CHILD WELFARE PRACTICE

1
OUTCOME BASED CHILD WELFARE PRACTICE

• Module VI
• Training for Managers

Created by the Children and Family Research
Center at UIUC
Through a DHHS Child Welfare Training Grant,
Updated February 2005
2
AGENDA
Section 1 Welcome and Introductions Section
2 The New Child Welfare Environment Section
3 Theoretical Framework Section 4 Integration
of Data into Daily Practice Section
5 Performance Tools for Daily Practice Section
6 Development of Outcome Based Training Programs
3
TRAINING OBJECTIVES

• Identify the fundamental aspects of an
  organization that must be assessed when moving to
  an outcome based practice.
• 2. Identify the interrelationship between
  outcomes on the Child and Family Services Review
  (CFSR) and outcomes associated with other
  accrediting bodies.
• Explore how outcomes are impacted by the agencys
  theoretical framework.
• 4. Examine a framework for program evaluation
  that incorporates the development and monitoring
  of outcomes.
• 5. Identify tools that can help field staff
  better manage toward positive outcomes for
  children and families.

4

• The Children and Family Research Center (the
  Center) received a training grant from the
  Department of Health and Human Services. The
  primary purpose of the grant was to explore the
  most effective manner in which to bring outcomes
  to the field level. The Center trained a
  combined group of 8 public and private child
  welfare agencies over a 3 year period.
• This module identifies the findings from the
  training initiative.

5
Section 2
The New Child Welfare Environment
6
The Old and New Accountability in Child
WelfareAdapted from the Center for the Study of
Social Policy, Improving the Performance and
Outcomes of Child Welfare through the State
Program Improvement Plans (PIPs) May 2003
7
THE NEW CHILD WELFARE ENVIRONMENT

• A shared set of expectations and a vision about
  what child welfare should be achieving.
• Frontline practice is in the spotlight.
• Data is being used to inform practice.
• Broad base of stakeholders involved.
• Comprehensive and continuous system of
  improvement.

8
SYSTEMATIC CHANGE
Child welfare has moved from a compliance driven
system to an outcome based system.

• The challenge for child welfare managers is to
  expand their knowledge as well as their staffs
  knowledge to
• Learn how to integrate data into their daily work
  ultimately improving outcomes for children and
  families.
• Learn how to integrate outcome achievement with
  best practice.
• Develop an agency culture that incorporates and
  rewards outcomes in all agency systems.
• Learn how to prioritize case activities so
  outcomes can be achieved.

9
SELF EVALUATION TOOL
In order to be in compliance with the seven
outcomes on the CFSR, all agency systemic factors
need to promote outcome based management.

• The self evaluation tool is organized around the
  systemic factors that are reviewed through the
  CFSR. It is designed to help an agency examine
• Different agency systems that support outcome
  based practice.
• Whether outcome management is being supported at
  every level within the agency.
• Where changes need to occur within the agency to
  become more outcome focused.

10
IS YOUR AGENCY PREPARED FOR OUTCOME BASED
CHILD WELFARE MANAGEMENT?

• COMPLETE
• THE SELF EVALUATION
• TOOL

11
Interrelationship Between Different Outcome
Measures
Federal State Laws
Accreditation Requirements
Federal Review Process
OUTCOMES
Performance Based Contracting
State Child Welfare Agency Rules and Policies
Grants
12
Section 3
Theoretical Framework
13
OUTCOMES
PRACTICE
PERFORMANCE TOOLS
DATA
TRAINING
THEORETICAL FRAMEWORK
14
The theoretical framework is the foundation from
which an agency works. The framework is the
first element that needs to be assessed and
changed when moving toward outcome based
practice. The framework should

• Provide staff with a new paradigm that
  incorporates outcomes and best practice (includes
  preventative planning).
• Make all staff feel they are part of outcome
  achievement, highlighting how they are integral
  to the process.
• Focus on outcome management at all levels. It
  must be embraced at administrative levels in
  order to become important at the field level.
• Find ways to implement processes that support the
  most important tasks of caseworkers.
• Organize the workload based on outcomes.

15
Stephen Covey states in The 7 Habits of Highly
Effective People, paradigms are powerful because
they create the lens through which we see the
world. The power of a paradigm shift is the
essential power of quantum change.
Knowledge (what to, why to)
Skills (how to)
Desire (want to)
Habits
16

• Work habits at an agency are reflections of the
  theoretical framework. This framework has a
  direct result on the achievement of outcomes.

17

• In The 7 Habits of Highly Effective People
  Stephen Covey identifies four quadrants within
  which all work activity is divided. Each
  quadrant has a different level of urgency and
  importance.

18
URGENT I Activities Crises Pressing
problems Deadline-driven reports
NOT URGENT II Activities Prevention Relationship
building Recognizing new opportunities Planning
IMPORTANT
III Activities Interruptions, some calls Some
mail, some reports Some meetings Proximate,
pressing matters
IV Activities Trivia, busy work Some mail Some
phone calls Time wasters
NOT IMPORTANT
19

• How can agencies shift the greatest percentage of
  their workers daily activities to quadrant II?
• Create an environment which supports planning,
  emphasizes skill building, provides ongoing
  learning experiences, focuses on concrete
  outcomes, and evaluates and rewards success.

20
To change an agencys theoretical framework the
agency must
1) Have the ENTIRE agency committed to making
this change 2) Ensure that ALL levels of staff
can and will work in teams supporting one
another. 3) Ensure that ALL staff are aware of
the agencys outcomes.
21

Section 4
Integration of Data into Daily Practice
22

• Individuals at all levels should access and use
  data in their daily work. The importance of data
  must be emphasized throughout the agency in words
  and in deeds.

23
Developing and finding data sources is an
essential step in moving toward outcome based
practice. Some data can come from external
sources however the majority of data should come
from internal evaluation.
24
EXTERNAL DATA SOURCES

• External data can be obtained from literature
  reviews, bibliographies, monitoring agencies, and
  review of similar programs. In Illinois,
  examples of external data sources include
• The Children and Family Research Centers
  literature reviews, bibliographies and research
  papers on various child welfare topics. These
  documents can be obtained for free through their
  website (cfrcwww.social.uiuc.edu).
• The POS Agency Statistical Summary- a document
  that is produced by DCFS which provides profiles
  and data on private child welfare agencies
  throughout the state.
• The Outcome Based Child Welfare Practice Website
  (accessible through the best practice link at
  cfrcwww.social.uiuc.edu). This website provides
  data on specific performance outcomes that are
  broken out by agency and team level.

25
INTERNAL DATA SOURCES

• Internal data can come from
• Performance tools used by field staff
• Statistics gathered through internal data systems
• Program evaluations

26

• Continuous program evaluation will not only
  provide the agency data, but will also assist in
  the development of outcome indicators.

27

• The remainder of this section will summarize a
  program evaluation model described in Program
  Evaluation in Family Centered Services Workbook
  (August 2001). This workbook was developed by
  Dr. Susan Wells and Dr. Tamara Fuller at the
  School of Social Work, University of Illinois at
  Urbana Champaign. This workbook is accessible on
  the Outcome Based Website.

28

• The next few slides will introduce you to the
  necessary steps for conducting a program
  evaluation. This information is only intended to
  highlight the overall model. Agencies are
  encouraged to review the full model in the
  workbook and/or consult with a expert in program
  evaluation.

29
STEPS FOR PROGRAM EVALUATION
1. Involve Stakeholders
2. Establish Program Goals and Objectives
3. Link Best Practice to Goals Objectives
4. Establish Program Evaluation Method
5. Select Measures
6. Plan Data Collection
7. Finalize Plan
8. Collect Analyze Data
9. Report, Disseminate, Use What You Learn
Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
30
Step 1 Involve Stakeholders

• This step brings all of the players together
  and creates an environment of ownership. It is
  essential to get input from the stakeholders on
  program goals, desired outcomes, collection of
  information, and analysis of data. During this
  step the agency should
• Form a review board or committee to guide the
  entire process
• Form an evaluation work group (parties
  responsible for conducting the evaluation)
• Develop a working agreement (specifies the
  evaluation plan)

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
31
Step 2 Establish Program Goals and Objectives

• One of the most critical steps is establishing
  goals that accurately reflect what you hope to
  achieve and that lend themselves to being easily
  measured. There are four basic processes
  associated with this step
• Review and evaluate existing mission and goals
• Identify service areas
• Define service area objectives
• Identify indicators that measure progress on
  objectives

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
32
Step 2 Example

• Program Goal Provide foster
  families with support
• 
  services in order to reduce child maltreatment
  while in care.
• Service Area Traditional foster
  care placements at Agency R.
• Service Area Objective Reduce the maltreatment
  of children in traditional placements at Agency
  R by x percent
• toward
  a target of y percent, which is the DCFS CFSR
  PIP Goal for FY 2005.
• Outcome Indicator Maltreatment rate in
  traditional foster care during FY05.
• Service Outcome Percent of traditional
  foster care families that receive support
  services specific to the needs of children
  placed in their home.
• Child Outcome Children are not maltreated in
  placement by their foster families.

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
33
Step 3 Link Best Practice to Goals and
Objectives
The main elements of the program should make
sense in terms of goals and objectives. Each
program activity should lead, directly or
indirectly, to the achievement of program
improvements. This step involves the development
of a logic model. A logic model is a coherent
framework of the program. It shows the
relationships among activities, the path from
activities to outcomes, and finally to long term
goals.
Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
34
Step 4 Establish Program Evaluation Method

• Selection of an evaluation design includes
  consideration of cost, participants, and
  potential results. The following processes must
  be completed during this step
• Determine the goal of the evaluation- what you
  hope to achieve with the results.
• Review different evaluation designs and select
  the most appropriate.
• Identify information already available that will
  help with evaluation questions.
• Determine the additional information required to
  address the evaluation goal including resources
  required for data collection, ability of existing
  staff to carry out evaluation tasks, and time
  required for evaluation.
• Establish a design framework.

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
35
Step 5 Select Measures

• This step involves selecting appropriate measures
  for the program evaluation design. A good
  measure is
• Reliable- it is credible, scores or calculations
  are accurate and consistent over time.
• Valid- it represent the actual thing that it is
  purported to measure.
• Suitable- for the client population and setting.

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
36
Step 6 Plan Data Collection

• This step determines how an agency will collect
  the information used in the outcome measures.
  There are numerous ways to collect program
  evaluation data, ranging from the use of existing
  records to complex client observation coding
  systems. Examples of data collection include
• Population Data
• Administrative Data and Case Records
• Surveys
• Interviews and Focus Groups
• Observational Techniques

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
37
Step 6 Continued Plan Data Collection

• This step will also involve the creation of a
  data collection plan. The plan should contain
  the following information
• Indicators
• Data Sources (where will the data come from)
• Outcome Measures
• How often the data will be collected
• Who will collect the data
• How data will be tracked and managed

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
38
Step 7 Finalize Plan

• During this step the agency should re-visit the
  working agreement established by the review board
  in step 1. The evaluation plan should be
  specific. General information that should be
  included in the plan
• Evaluation framework
• -What are you going to evaluate
• -Questions to be addressed in the evaluation
• -Timeframe for the evaluation
• Procedures and methods for evaluating
  implementation objectives
• Procedures and methods for evaluating participant
  outcome objectives
• Procedures for managing and monitoring the
  evaluation

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
39
Step 8 Collect Analyze Data
Once all of the data has been collected an agency
can begin to analyze and interpret the data.
Before the data can be analyzed it needs to be
coded. Coding involves two steps 1) Assign a
numerical value to all variables (i.e. male is
1 and female is 2). Keep a written record
of how each variable was coded. 2) Enter the
values into a spreadsheet or database
Once all the data is entered into a spreadsheet
it can be analyzed and interpreted. Central to
an outcome evaluation is determining performance
standards. In child welfare, performance
standards have been set for many variables
through the Child and Family Services Reviews.
If the variable is not part of the Child and
Family Services Review, the agency will have to
make judgments about what is considered a
success.
Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
40
Step 9 Report, Disseminate, Use What You Learn

• A well organized evaluation report is critical.
  There are different ways to report evaluation
  findings. In general, an evaluation report
  should include
• Executive summary
• Background information on the program
• Program evaluation methods
• Program evaluation results
• Discussion of program evaluation results
• Conclusions and recommendations

Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
41
Step 9 Report, Disseminate, Use What You Learn

• Once the evaluation report is complete, the
  agency must decide
• How and to whom it can and will be disseminated
• How the information gained in the report will be
  used to improve relevant programs

The data gained from the evaluation can be used
to improve programs, develop specific outcomes,
set performance standards, and identify specific
areas that need to be improved. Evaluation that
emphasizes continuous improvement will keep
agencies focused on achieving outcomes.
Institutional Review Board (IRB) approval may be
required if reports are disseminated outside of
the agency.
Program Evaluation in Family Centered Services
Workbook (2001). Prepared by Susan Wells and
Tamara Fuller at the School of Social Work,
University of Illinois at Urbana-Champaign for
the Illinois Department of Children and Family
Services.
42
Section 5
Performance Tools for Daily Practice
43

• Performance Tools Serve Two Primary Purposes
• Help individuals meet a specific outcome by
  tracking essential information, showing trends
  over time, and/or providing a framework for
  outcome achievement.
• Gather and display data in a way that helps
  people manage toward outcomes.

44
Listed below are performance tools for workers
and supervisors that have been created by the
Center. These tools can be downloaded from the
Outcome Based Child Welfare Practice Website

• Caseload Service Plan (caseload/team/program
  level)
• Placement History Tool
• (caseload level)
• Safety Tool (caseload level)

45
CASELOAD SERVICE PLAN TOOL

• Divides permanency goals into objectives and
  action steps.
• Sets time frames for action steps, objectives,
  and permanency achievement.
• Provides building blocks for qualitative and
  quantitative measurement.
• Gathers information that can be utilized at the
  worker, supervisor, or program level.
• Identifies where policy and/or practice changes
  may need to occur within the agency in order to
  support progress toward permanency achievement.
• Provides the foundation for a comprehensive
  continuous quality improvement (CQI) process.

46
This tool allows a worker to compare the types of
children on his/her caseload to characteristics
that research has shown to be associated with the
achievement of each of the four following
outcomes
47
CASELOAD SERVICE PLAN TOOL Worker Level
Once workers have completed the caseload service
plan tool, they can use the information to

• Determine the permanency goal for each case and
  then assign objectives and action steps to meet
  the goal. Set an achievement date for each
  action step and objective.
• Estimate a date to achieve the permanency goal
  for each case based on the estimated dates for
  achievement of the objectives and action steps.
• Assign a Permanency Achievement Date level to
  each case (1 4) so that the worker can quickly
  prioritize permanency achievement dates for all
  cases on the caseload.
• Keep track of their progress toward achieving
  permanency goals on their caseload.
• Focus on cases that are close to achieving
  permanency.
• Identify trends on their caseload.

48
CASELOAD SERVICE PLAN TOOL Supervisor/Manager
Level Useage

• Once workers have completed the caseload service
  plan tool, supervisors can use the information
  to
• Quickly assess the number and types of cases a
  worker has across a number of variables (age,
  reason for placement, type of placement, etc.).
  This information can be used for case planning,
  feedback, as well as worker performance
  management.
• Quickly determine how many cases are expected to
  achieve permanency within a year and identify
  those cases that are close to permanency.
• Track a workers progress toward goal achievement
  and jointly find solutions to any barriers
  encountered.
• Identify worker and team level trends.
• Identify areas that worker needs additional
  training or resources.

49
PLACEMENT HISTORY TOOL

• Provides a chronological listing of placements
  for a child since he/she entered care.
• Provides data that shows different levels of
  stability caseworker, placement, city, living
  with/without siblings.
• Provides workers with a new perspective of their
  cases can see stability trends over time.
• Provides workers with information that is needed
  in various reports and settings.

50
SAFETY TOOL

• Uses information gathered from the Illinois Child
  Endangerment Risk Assessment Protocol.
• Shows trends with safety which can be used to
  make critical decisions relating to visitation,
  placement, and permanency goal achievement.
• Provides workers with a historical view of safety
  factors for the child/family.
• Maintains SAFETY as a key goal for children in
  placement.

51
Section 6
Development of Outcome Based Training Programs
52
Development of Outcome Based Training Programs

• Include a combination of data and best practice
  information.
• Incorporate outcomes into all trainings. Field
  staff must know what they are managing toward and
  why. Be clear about the outcomes that you want
  staff to measure and track.
• Avoid discussions about how the data is wrong.
  Instead focus on ways that practice can be
  improved.
• Develop exercises that allow staff to use data
  presented in the training.
• Guide staff to an awareness that they are part of
  the solution and that they have the ability to
  improve the outcomes.
• Ensure that individual staff members understand
  and encourage them to embrace the new paradigm.
  Provide training to each person in the agency on
  outcome management.

53
Emphasize the strengths associated with outcomes
management in training. These strengths include

• Provides a road map for child welfare workers
  that clearly states the priorities for practice.
• Helps workers stay focused on the most important
  components of serving children and families on
  their caseload.
• Helps workers track their own as well as their
  clients progress toward measurable goals.
• Helps workers prioritize the many demands that
  casework brings.
• Provides information that helps people with case
  planning.

54

• Agencies can create their own outcome based
  training or they can utilize free modules that
  the Center has developed through the Department
  of Health and Human Services grant. The
  following modules can be accessed through the
  best practice link at cfrcwww.social.uiuc.edu
  
• Understanding the Review Process
• Achieving Permanence for Children in Substitute
  Care
• Ensuring Safety of Children in Substitute Care
• Promoting Well Being of Children in Substitute
  Care
• Practice Issues related to the Child Well Being
  Study

55

• The modules on the Centers website have been
  developed for direct field level staff. All of
  the modules are accompanied by a
  train-the-trainer guide and handouts. The
  modules can be downloaded and adapted to meet an
  agencys specific needs.
• Additional outcome based training modules and
  tools can also be accessed at the following
  sites
• www.rom.ku.edu/otherlink.asp
• www.nrcitcw.org