Supporting Narrative Evaluation Action Research (NEAR) in a Community Health Setting

1
Supporting Narrative Evaluation Action Research
(NEAR) in a Community Health Setting

• Melanie Block
• Jaime Timmerman
• Western Region Health Centre (WRHC)
• VIC

2
Introduction

• Describe WRHC NEAR peer education mentoring
  project
• Outcomes Capacity building
• Explore our learnings about the appropriateness
  of NEAR as a form of evaluation in Community
  health

3
Background

• Aim
• To increase the capacity of staff to be able
• to produce quality narrative evaluation stories
• using an action research method
• Method
• Peer education mentoring model by 2 staff
  trained
• Through DHS NEAR phase 1 program 2003-2004

4
Consumers

• GPs
• Medical Records
• Nurses

Medical
Board
Quality
Dental

• Puiblic
• Nurses

Clients
HARP
Community
Child Family

• VACP
• Counselling
• Volunteer Project

PM Counselling Support Services

• Health Smart
• IT

PM IM/.IT
Patients

• Drug Safety
• Primary Care
• Projects

PM Health Works
5
WRHC Management Perspectives

• Benefits to staff
• PD to improve writing and evaluation skills
• Give staff an opportunity to explore client views
  points and experiences as a valid form of
  evaluation
• Benefits to the client
• NEAR style is relevant to key target group and
  suits the way workers interact with clients
• Give clients a voice
• Clients enjoy being consulted and heard,
• Evaluation results more meaningful and easier for
  client to understand
• Benefits the program
• Peoples stories are remembered more than
  statistics
• Gives more depth into the nature of the program
  and its outcomes, positives results help to
  sustain programs

6
Training Program

• Criteria for staff doing internal training
• Health Promotion Project ready for evaluation
• Willing to complete a narrative for evaluation
• Structure Timelines
• 5 Staff 1 student participated
• Development of 5 narrative pieces
• Preparation Reflection on our learning through
  DHS program selection of practical concrete
  activities
• NEAR manual http//www.health.vic.gov.au/healthpr
  omotion/hp_practice/eval_dissem.htm

7
WRHC NEAR Workshop

• What is NEAR?
• What makes a good story?
• Forming your own stories
• Gaining material for your story
• Privacy Ethics
• Tips ideas about editing writing styles
• Resource kit with summary of additional resources
• Ongoing support and assistance

8
Examples of activities

• Evaluation Mug exercise

9
Examples of activities

• Action Research (AR) Map (Wadsworth 1991)

10
Examples of activities

• Who are you telling the story to why?
• Single or multiply voices? I, we, they

11
Outcomes of the Project

• Individuals who participated in training
• used their training skills to
• Create 2 narrative pieces, a third piece is in
  the data collection phase

12
Additional outcomes of the project

• Individuals who participated in training
• also used their training skills to
• Influence their reporting methods
• Provide support and encouragement to other staff
  in using NEAR principles
• Advocate for the use of NEAR methodology in other
  organisational evaluations

13
Use of narratives

• Reporting to funding body
• Lobbying for future funding e.g SHARE
• Tool for staff to understand the project and its
  meaning to participants
• Background information for future evaluation
• Increase life of the project

14
NEAR has given a voice to clients that wasnt
there before staff member

• Staff found NEAR valuable in capturing the voice
  of the target group
• You get bogged down in process and lose sight
  of (the clients perspective)
• I was surprised about how well the process
  worked with the consumers
• I received much more personal responses from the
  consumer than I thought
• Clients were responsive to the informal questions
  used

15
NEAR has given a voice to clients that wasnt
there before staff member

• Client voices
• Its been very cathartic for me. I think Ive
  learnt something about myself, that Im capable
  of doing something thats really hard. If I can
  do what I did with SHARE, I can do other hard
  things
• I was a teenager when I knew something was wrong
  with me. If I had something like SHARE when I
  was in school it would have been much easier for
  me. I looked in books but it was not very
  helpful. Its much better hearing someones
  personal experiences
• I was able to express things that I have kept
  inside me for a long time

16
NEAR has given a voice to clients that wasnt
there before staff member

• Client voices
• Im very happy to share my personal story. Im
  not ashamed. All of our stories are different
  and hopefully the students will be able to relate
  to someone
• I wanted to make high school students more aware
  of mental health
• My hope is that more people will understand that
  were just ordinary people with an extraordinary
  problem

17
Learnings

• Gave participants enthusiasm for evaluation
• Greater outcomes than just narrative pieces
• Low cost training but high staff time costs
• Continued management support despite time
  consuming nature of NEAR

18
Learnings

• Most effective if
• Staff trained are clear on the project to be
  evaluated and why
• Staff trained are confident to undertake
  interviews and write about sometimes complex
  themes
• Management support all evaluation outcomes
  whether positive or negative
• NEAR used as an evaluation method to complement
  quantitative data

19
Expansion beyond Health Promotion

• Future intentions for NEAR within the agency
• Mental Health Team - evaluation
• Diabetes Outreach Position - evaluation
• Illicit Drug Alcohol Research Project -
  evaluation

20
Conclusion

• The NEAR peer training provided at WRHC has met
  its aim To increase the capacity of staff to be
  able to produce quality narrative evaluation
  stories using an action research method
• Additionally it has lead to team and
  organisational benefits beyond the scope of those
  trained

21
Conclusion

• NEAR is a valuable tool in a community setting to
  evaluate programs. It is most valuable when
  supported by management, adequately resourced,
  based on a set project and used as an evaluation
  method to complement quantitative data
• It was valuable in gaining perspectives or voice
  of the clients and community members
• NEAR training can be delivered successfully by
  staff familiar with the concepts of actions
  research, evaluation and narratives and with the
  assistance
• of tools such as the DHS NEAR manual

22
Future of NEAR at WRHC

• Continue to provide support to individuals
• Staff trained will continue to mentor their peers
• Possibility of implementing another training
  cycle
• Ongoing organisational focus on narrative style
  evaluation e.g. annual report, program and
  position evaluations