Evaluation of the MAC Milk

1
Evaluation of the MAC (Milk Cookies) Childrens
Program

• Collaborating Partners
• Assisting Families of Inmates
• VCU School of Social Work

2

• Collaborating Partners
• Cindy Freeman, M.S.W., Former Program Coordinator
• Dana Taylor, M.S.W. Student, Research Assistant
  Interim Program Coordinator
• Fran Bolin, M.S.W., Director, Assisting Families
  of Inmates
• Michael Sheridan, M.S.W., Ph.D., Program Evaluator

3
Overview of MAC Childrens Program

• Specialized services for children of incarcerated
  parents and their caregivers.
• Overall goal of serving 30 families within 2-year
  demonstration project period.
• Major funding sources Jackson Foundation, United
  Way, Memorial Foundation for Children, Robins
  Foundation.

4

• Major Program Activities
• Identification Recruitment
• Intake/Initial Assessment Families Children
• Ongoing Assessments/Relationship Building
• Individual Counseling Children and Caregivers
• Childrens Support Groups
• Childrens Recreation/Enrichment Activities
• Family Assistance with Concrete Needs
• Information Referral Services
• Community Education/Advocacy
• Program Staffing
• Full-time Program Coordinator/Case Manager
• Part-time Volunteer and Family Services Case
  Manager

5
Program Context

• Assisting Families of Inmates (AFOI)
• Began in 1978 as outreach mission of Second
  Presbyterian Church (formerly PFSS)
• Provide transportation needs, support groups,
  informational workshops, individual counseling,
  and referral services
• Community-in-Schools (CIS)
• MAC Program operates as CIS team member at
  Blackwell Primary Elementary Schools
• CIS is referral source and service broker to
  other agencies/programs

6

• Blackwell Community
• History of high poverty, high crime rate,
• high incarceration rate
• Hope IV Project
• Community school in transition
• MAC Childrens Program Sites
• Blackwell Primary Elementary Schools
• Swansboro Elementary School
• Sacred Heart Center

7
MAC Childrens Support Groups

• 8 Group Meetings w/monthly follow-ups
• 2 Overall Purposes
• To increase sense of social support and decrease
  sense of secrecy and stigma through safe and
  nurturing group environment.
• To increase self-esteem and self-respect through
  connection with positive cultural identity and
  positive interpersonal interactions.
• 3 Major Units
• Conflict Resolution
• Relationships/Families
• Self-esteem/Unity

8

• Rooted in Afrocentric perspective
• Name Group NANE (Swahili word for 8) NEVER
  ACCEPT NEGATIVE ENERGY
• Nguzo Saba (7 Principles) Respect
• Umoja (Unity)
• Ujima (Collective Work Responsibility)
• Nia (Purpose)
• Ujamaa (Cooperative Economics)
• Heshema (Respect)
• Kuumba (Creativity)
• Kujichagulia (Self-determination)
• Imani (Faith)

9
Overview of Evaluation

• First Year Monitoring
• Linkage Program Inputs ? Program Activities
• Documented Program Implementation
• Tracked Program Operations
• Assessed Program Efforts
• Second Year Process Evaluation
• Linkage Program Activities ? Program Outputs
• Assessed Program Efforts
• Evaluated Program Outcomes

10
Evaluation Products

• Program Logic Model Methods of Rationale
• Program Management Tools
• Family Demographic Sheet
• Intake Assessment Form
• Service Plan Service Review Form
• Program Service Activity Log
• Case Closure Form

11

• Program Evaluation Tools
• Pictorial Perceived Social Support Scale
  (Parent/Caregiver Friends Teacher)
• Secrecy Scale
• Stigma Scale
• Knowledge of Principles Index
• Teacher Rating of Student Behaviors Index
• School Related Measures
• Grades
• Absences/Tardiness
• Parent/Caregiver Interview
• Childrens Focus Group Interview

12
Description of Evaluation Participants

• 18 Parents/Caregivers 24 Children
• Children
• 50 boys 50 girls
• 91.7 African-American 8.3 Bi-racial
• Mean age 7.17 years
• 8.3 (n 2) 5 years old
• 37.5 (n 9) 6 years old
• 16.7 (n 4) 7 years old
• 20.8 ( n 5) 8 years old
• 8.3 (n 2) 9 years old
• 8.3 (n 2) 11 years old

13

• Parents/Caregivers
• 100 female
• 88.9 (n 16) African-American
• 5.6 (n 1) Bi-racial
• 5.6 (n 1) Latina-American
• Ages ranging from 35 to 67 years old
• 44.4 (n 8) Employed Full-time
• 16.7 (n 3) Employed Part-time
• 38.9 (n 7) Unemployed

14

• Types of Public Assistance
• 66.7 (n 12) Medicaid
• 66.7 (n 12) Food Stamps
• 33.3 (n 6) Public Housing
• 27.8 (n 5) TANF
• 22.2 (n 4) SSI/Social Security
• 5.6 (n 1) Child Care Assistance
• 5.6 (n 1) State General Assistance

15
Data re Incarcerated Parents

• Incarcerated Parent
• 59.1 (n 13) Father
• 18.2 (n 4) Mother
• 22.7 (n 5) Both Mother Father
• Contact with Incarcerated Father
• 88.3 (n 15) Yes
• 16.7 (n 3) No
• Contact with Incarcerated Mother
• 55.6 (n 5) Yes
• 44.4 (n 4) No

16

• Relationship with Father
• 22 (n 4) Poor or Very Poor
• 22 (n 4 ) OK
• 56 (n 10) Good or Very Good
• Relationship with Mother
• 33.3 (n 3) Very Poor
• 33.3 (n 3) OK
• 33.3 (n 3) Good or Very Good
• Know About Fathers Incarceration
• 89 (n 10) Yes
• 11 (n 2) No
• Know about Mothers Incarceration
• 89 (n 10) Yes
• 11 (n 2) No

17
Evaluation Data on Program Efforts

• 1st Year
• Served 21 Families (25 adults 35 children)
• 406 Services Provided
• 187 Group Counseling/Support Group Services
• 70 Individual Counseling
• 50 Family Assistance w/Concrete Needs
• 25 Information/Referral
• 22 Intake/Initial Assessments
• 22 Ongoing Assessments/Relationship Building
• 12 Mentoring Tutoring
• 11 Teacher Consultations
• 4 Case Closures
• 3 Parents/Caregiver Conferences

18

• 2nd Year
• Served 34 Families (34 adults 45 children)
• 14 Families continuing from 1st Year
• 937 Services Provided
• 320 Group Counseling/Support Group Services
• 190 Individual Counseling
• 102 Recreation/Enrichment Activities
• 72 Ongoing Assessment/Relationship Building
• 44 Mentoring Tutoring
• 44 Teacher Consultations
• 41 Information/Referral
• 34 Family Assistance w/Concrete Needs
• 30 Intake/Initial Assessments
• 24 Case Staffings
• 22 Parent/Caregiver Conferences
• 14 Case Closures

19
Process Evaluation Findings Childrens Support
Group

• Perceived Social Support
• Parent/Caregiver nonsignificant
• Friends Statistical trend (p .053)
• Pretest M 17.35
• Posttest M 18.50
• Teachers Statistical trend (p .074)
• Pretest M 12.55
• Posttest M 11.85
• Secrecy
• Statistically significant (p lt .05)
• Pretest M 12.57
• Posttest M 11.42

20

• Stigma
• Statistical trend (p .074)
• Pretest M 6.10
• Posttest M 5.33
• Knowledge of Principles
• Statistically significant (p lt .001)
• Pretest M 1.95
• Posttest M 5.32

21

• Teacher Ratings of Student Behaviors
• ( agree or strongly agree)
• Appears happier 84.6
• Grades improved 46.2
• Relating better to teacher 76.9
• Relating better to peers 84.6
• More compliant w/school rules 84.6
• Less frustrated 76.9
• Accepts suggestions/feedback better 76.9
• Behavior improved 76.9

22

• School Indicators
• Grades ( Improved)
• Language Arts 28.6
• Mathematics 14.3
• Social Science 50.0
• Science 28.6
• Absences 21.4 Reduced
• Tardiness 7.1 Reduced
• Absences (M 5.80, Range 0 - 19)
• Tardiness (M 1.86, Range 0 - 6)
• (after outlier of 39 times eliminated)

23

• Childrens Focus Groups Major Themes
• What did you learn in your group?
• Principles/words
• How to talk about parents being in jail
• How to share feelings/talk about families
• How to be friends
• How to behave/follow the rules
• What Im like
• What Im going to be when I grow up

24

• What did you like the most about your group?
• Learning new things
• Talking about feelings/talking about parents
• Eating/getting snacks
• Friends in group
• Reading books/drawing pictures
• Field trips
• Getting letters from volunteers
• Being in a special group

25

• What did you like the least about your group?
• Talking about feelings
• Thinking about parents
• People in group acting up
• When didnt get a snack or got in trouble
• When it was time to leave
• Nothing

26

• What do you think would make your group better?
• Everybody being good
• Go on more trips
• Have group last longer
• More people in group/all groups be 1 big group
• Have group meet somewhere else
• Make presents for others in group
• Make presents for brothers sisters
• More snacks
• Have a big party at end of the year
• Play on computer/more toys
• Have books about children like us

27

• Is there anything else you would like to say
  about your group?
• Love/miss Ms. Freeman (many, many comments)
• Glad to be in this group
• It was fun
• Liked the snacks

28
Evaluation Limitations

• Small, nonprobability sample
• One Group Pretest/Posttest design
• Self-report, except Teacher Ratings School
  Indicators
• Possible inappropriateness of social
• support measure

29
Lessons Learned Implications for Practitioners

• Identification and recruitment of families of
  children of incarcerated parents is a long,
  involved process due to issues of stigma or bias
  that produce shame and secrecy among family
  members and a protective stance among potential
  referral sources.
• Building trust and effective working
  relationships in disenfranchised and under-served
  communities is a slow, painstaking process with
  both community residents and professionals
  serving the community.

30

• Children of incarcerated parents and their
  families can be engaged in service delivery if
  program services are both relevant and
  culturally-sensitive.
• Effective strategies for both recruitment and
  engagement in service delivery include
• Assertive, yet respectful outreach efforts
• Commitment to ongoing relationship-building
• Collaborative partnership model vs. expert
  model
• Provision of helpful information and services

31

• Expectations regarding necessary start-up time
  need to be realistic and should take into account
  unforeseen complications and setbacks.
• Findings suggest that focused intervention with
  children of incarcerated parents can positively
  impact perceived levels of social support from
  friends, decrease feelings of secrecy and stigma,
  increase connection to positive cultural
  identity, and improve various school-related
  behaviors.
• More comprehensive and targeted resources are
  needed to positively impact the family systems
  and school systems of children of incarcerated
  parents.

32
Lessons Learned Implications for Evaluators

• Evaluators must take sufficient time to build
  respectful, collaborative relationships with
  program staff and others important to the
  program.
• Evaluators should develop a good, working
  knowledge of program structure and program
  activities in order to help staff develop an
  accurate model of the program.

33

• An evaluability assessment should be conducted
  prior to beginning any other evaluation
  activities. This includes determining
• Clarity concerning evaluations purpose
• Existence of accurate Program Logic Model
• Stage of program development
• Suitability of existing data and data collection
  mechanisms
• Clarity regarding roles of both evaluator and
  program staff
• Administrative and political climate surrounding
  evaluation

34

• Evaluators should recognize that they are
  conducting research within a fluid and
  ever-changing action setting. This requires
  flexibility and a willingness to change aspects
  of the evaluation based on realities and needs of
  the program.
• Evaluators must be able to detect the story
  beneath the numbers in order to produce the most
  accurate and fair picture of program efforts and
  outcomes. Qualitative data, including
  perspectives of both staff and program
  participation should be included.