Developing Meaningful

1
Developing Meaningful Sustainable Outcome
Measures Indicators in Child WelfareRecurrence
of Maltreatment as an Indicator of Child Safety

• Workshop1st International Society for Child
  Indicators Conference Chicago, ILJune 26-28,
  2007
• PresentersBrenda Moody, Amin Malik, Brad
  Bain,Yosi Derman Mark Kartusch

2
Overview

• Context
• Methodology challenges
• Small group activity
• Literature
• Lessons learned discussion

3
Ontario, Canada
4
Child Welfare in Ontario

• Child and Family Services Act
• 53 independent, non-profit agencies
• Total budget 1.2 billion
• Caseload
• 82,346 investigations
• 26,378 open protection cases
• 18,497 children in care

5
Child Welfare in the Central Zone

• Eight diverse agencies in the Greater Toronto
  Area
• Represent approximately 1/3 of expenditures and
  caseloads in Ontario child welfare
• Central Zone Quality Assurance Committee

6
Accountability Levels
7
Child Welfare National (CDN) Outcomes Indicator
Matrix
8
Approaches to Indicator Development
9
Safety of Children - Indicators

• Serious occurrences
• New investigations on open cases
• Recurrence of maltreatment

• Consistent process for data
• Defining
• Gathering
• Analyzing
• Reporting
• Interpreting

10
Methodology

• Determine how many of the total closed cases in a
  fiscal year were re-opened within 12 months.
• For cases that had multiple closings within the
  same fiscal year, use the first closing in the
  calculation.

11
Cases of Reopening

• Total number of unique families that had at least
  one reopening
• Calculate the number of cases reopened as a
  percentage of the number of cases closed

12
Cases of Reopening
13
Incidents of Reopening

• All incidents of reopening including more than
  one opening for the same family
• Calculate the number of incidents of re-openings
  as a percentage of the number of cases closed

14
Incidents of Reopening
15
Challenges

• Defining variables
• Data extraction
• Formulas
• Multiple databases
• Data analysis interpretation
• Limitations
• Who does it?
• Inability to drill down into the data
• Data sharing
• Culture of secrecy - Provincially
  organizationally
• What to do with the findings

16
A Tale of Two Counties

• This tale transpires in the Sunshine State of
  Idyllia. You are the CEO of Donjon County Child
  Protection Service (CPS). Idyllia requires each
  CPS to evaluate the safety of children on its
  caseload-by measuring the recurrence of
  maltreatment according to the Toronto model.
• The child welfare authority of neighboring
  Paradise County finds that its recurrence rate is
  25. A week later, your IT manager informs you
  that she has found that YOUR recurrence of
  maltreatment rate is 45.
• You have just received an e-mail from the Idyllia
  State Department of Childrens Services, wishing
  to discuss the performance of your Agency.
  Realizing that 45 looks quite bad compared with
  25, you have two choices
• Bite down on your cyanide pill (secreted for just
  such an occasion in a handy wisdom tooth
  filling).
• Take a closer look at your data to see if you can
  understandand help others understandfactors
  that may contribute to the measured differences
  between the agencies.
• If you choose Bwhat might you find that could
  potentially protect your Agencys reputation (and
  your career)?

17
Identifying Defining Key Variables

• Time frame
• Frequency
• Severity
• Type of maltreatment
• Referral source
• Verified / substantiated

• Childs age
• Perpetrator
• Subject child
• Across within jurisdiction
• Service type / level

18
Recurrence of maltreatment literature -
Definitions

• No accepted standard definition
• Reliant on research objectives availability of
  data
• Has been defined as
• any subsequent report of maltreatment
• any subsequent founded or verified report of
  maltreatment
• any subsequent maltreatment of the same child, of
  another child within the family, or by the same
  perpetrator
• recurrence of maltreatment without a prior
  report and
• a combination of these

19
Recurrence of maltreatment literature - Factors

• Child age
• History/prior referrals
• Frequency of previous incidents of maltreatment
• Neglect or parent absence
• Parental conflict
• Parental mental health

20
Recurrence of maltreatment literature - Patterns

• Rate of maltreatment recurrence
• 14.7 at 6 months
• 22.6 at 18 months
• Greatest risk soon after an index episode
• Intensity of intervention seems to reduce rate
• Risk of recurrent maltreatment increased after
  each maltreatment
• Time between episodes of maltreatment shortened
  as the number of episodes increased.
• Children with prior contacts tend to have gt 2
  contacts

21
Lessons Learned

• Developing outcome measures indicators
• Defining indicators
• Gathering data
• Analyzing reporting on indicators
• Interpretation

22
Where has our work led us?

• Pilot the data definitions
• Re-collect analyze child indicators
• Complete Ministry Baseline Project
• Institutionalize in the Single Information System
• Adopt consistent approaches to indicator
  development
• Select additional indicators to measure

23
Thank you