Formative Evaluation as Community Development

1
Formative Evaluation as Community Development

• Daniela Stehlik
• Alcoa Research Centre for Stronger Communities
• Curtin University of Technology
• Lesley Chenoweth
• School of Social Work Applied Human Sciences
• The University of Queensland

2
Outline of presentation

• Key Learnings
• Program Context
• Formative Evaluation Philosophy
• Evaluation Dimensions
• Methods adopted
• Strengths Challenges

3
Key Learnings -Building Communities through
Evaluation

• Spatial communities
• Practice communities
• Human service communities

4
Program Context

• Rural/remote
• People with disabilities and their families
• Government staffed and funded
• Pilots
• Quest for better service delivery models

5
Formative Evaluation Philosophy Framework

• Social Justice
• Participatory
• Action research principles
• Ethical practice

6
Formative Evaluation Philosophy Framework - how?

• Evaluation as staff development
• Through Stakeholder Reference Group
• Involved in professional development activities
• Confidentiality/anonymity

7
Evaluation dimensions

• Complexities
• Multi-level interventions
• Cultural change agenda
• Searching for best practice
• Pressure to get evidence out
• Rural/remote
• Spectrum of disability service availability

8
What were the indicators used?

• From Program goals
• At the level of

9
What were the indicators used?

• From Program goals
• At the level of

• Government Policy
• Deptl Program management
• Field Practice
• People with a disability and their families
• Community capacity building

10
What were the indicators used?

• From Program goals
• At the level of

• Government Policy (4)
• Deptl Program management (3)
• Field Practice (6)
• People with a disability and their families (5)
• Community capacity building (7)

22 levels .
11
Methods adopted .

• Focus groups
• Questionnaires
• Indepth interviews
• Measurement of social relationships
• Participant observation
• Client records analysis
• Practioner as researcher

12
Evaluation dimensions

• Complexities
• Multi-level interventions
• Cultural change agenda
• Searching for best practice
• Pressure to get evidence out
• Rural/remote
• Spectrum of disability service availability

13
Multi-level Interventions
14
Multi-level Interventions
Program addressed each level and the interactions
between
15
Multi-level Interventions
Program addressed each level and the interactions
between
16
Multi-level Interventions
Program addressed each level and the interactions
between
17
Multi-level Interventions
Program addressed each level and the interactions
between
18
Multi-level Interventions
Program addressed each level and the interactions
between
19
Key Learnings -Building Communities through
Evaluation

• Spatial communities
• Worker/families/ngos/govt etc.
• Practice communities
• Workers/ngos/govt/academe
• Human service communities
• govt./ngos/academe

20
Building the Practice Community
21
(No Transcript)
22
CD in practice

• The role of the practitioner
• Capacity building doesnt just happen
• Facilitation Intervention are required
• Building a 21st century model of human service
  practice

23
Strengths of Evaluation Approach

• Accommodated complexity
• Provided both quantifiable and illustrative
  evidence
• Gave value beyond the evaluation per se
• Enabled participation including people
  with a disability

24
Strengths cont.

• Built practitioner confidence
• Enhanced program profile within dept.
• Worked in longitudinal and short term situations
• Enabled timely feedback of findings
• Aided writing of report

25
Challenges

• Assumed high level evaluator knowledge of
  program, practice issues
• Political sensitivity
• Distance across sites limited participant
  observation
• Demand for face to face contact
• Problematic access to client databases records

26
Thank - you