Participatory Evaluation

1
Participatory Evaluation

• Mary Phillips, BME
• Former Circles of Care Program Coordinator,
  Oakland and an Evaluator, Los Angeles, CA

2
Its Not Just an Evaluation Methodology

• We all have a lot in common. We all care very
  deeply about the communities that we come from
  and about American Indian and Alaska Native
  communities across the country. We all share deep
  concern about the well-being and future of Indian
  children and families. We are all very
  self-sacrificing. We are in this business because
  of our personal beliefs and our desire to make a
  difference. Our personal backgrounds, our
  training, our education, and our professional
  experience all contribute to our desire to make a
  difference.
• Holly-Echo Hawk, M.S., Echo Hawk and Associates

3
Its Not Just an Evaluation Methodology

• Participation Evaluation is a key component of
  the systems of care evaluation process that
  informs the mission, vision and authority of the
  Tribe/community throughout the Circles of Care
  program.
• Participation evaluation allows the community
  to voice its cultural values and promote cultural
  appropriateness in systems changes development at
  all levels.

4
Its Not Just an Evaluation Methodology

• The Circles of Care initiative represented one of
  the first opportunities for American
  Indian/Alaska Native AI/AN families and their
  communities have substantial input into the
  design of services to address the needs of their
  children, from their own understanding of these
  needs.
• The Standard Infrastructure Program Announcement,
  INF 05 PA and the Circles of Care III NOFA have
  included community involvement and participation
  measures that you MUST address throughout the
  application and are considered in the review
  process.

5
How it Works

• blends evaluation process components to actively
  involve members of the community to collect
  qualitative data that better informs the system.
• Having members who are knowledgeable about
  cultural norms of the community bared an
  unobtrusive context to present systems change.

6
The Process of Participatory Evaluation

• Planning the assessment will require a working
  process between the project staff and evaluation
  staff and the key informants of each community.
  Taking into consideration the current status of
  delivery systems for Native youth and mental
  health, developing the assessment tool(s) was
  pivotal in getting necessary information on
  contextual issues of the system.
• Example Community Readiness Model was developed
  at the Tri-Ethnic Center for Prevention Research
  at Colorado State University to measure systems
  change in Native American and other communities.

7
Community Readiness Model

• An innovative methodology to evaluate systems
  change and focus community efforts toward
  realizing its objectives. It identifies
  characteristics related to problem awareness and
  readiness for change within the community,
  fostering sound strategic planning, ongoing
  feedback, and realistic assessments of
  accomplishments.

8
Community Readiness Model

• The tool focused on collective information from
  these Native American resources 3 community
  identified leaders, 3 agency directors, 2
  cultural advisors, 3 community members and 2
  youth group representatives.
• Input from key stakeholders and community members
  was collected through interview questions related
  to
• Community Efforts, Knowledge of Efforts,
  Leadership, Community Climate, Knowledge About
  the Issues, Resources for Prevention Efforts.

9
Community Readiness Model

• The information collected allowed our program to
  measure where the urban Native American community
  stands with respect to its knowledge and
  readiness to implement a specific program
  (Strategic Plan for Urban Native Americans and
  their families Oakland, CA

10
Ongoing Particpation

• The evaluation participation process also
  incorporated resource development within the
  community. Linkages were made throughout the
  project as community coordinators and cultural
  leaders were provided information on the core
  elements of the system of care. This process
  required continuous interaction between agencies
  and cultural groups that concentrated the efforts
  to achieve child and adolescent behavior goals
  among agencies and family empowerment through
  focus groups.

11

12
Items from Standard Infrastructure Program
Announcement and Circles of Care NOFA

• Respond to each of these required information
  items with statements to let the reviewer know
  what work has been done the work that the
  program intends to do and how you will address
  each issue.
• Though not all requirements listed here are given
  points, providing this information can help
  understand the ability of the program to connect
  and involve the community.
• These should be written in Sections B. Proposed
  Approach, C. Staff, Management and Relevant
  Experience, D. Evaluation and Data

13
Items from Standard Infrastructure Program
Announcement and Circles of Care NOFA

• Program Goals
• Engage their community members in assessing
  service system needs, gaps, potential resources,
  and plan infrastructure development strategies
  that meet those needs.
• Increase the participation of families, tribal
  leaders, and spiritual advisors in planning and
  developing service systems and treatment options
  based on the values and principles of the AI/AN
  community served by the project.
• (Circles of Care III NOFA, p.2)

14
Items from Standard Infrastructure Program
Announcement and Circles of Care NOFA

• Grantees must use funds to carry out the
  following required activities
• Facilitate culturally respectful strategic
  planning activities engaging community members,
  key stakeholders, youth, elders, spiritual
  advisors, and tribal leaders to identify outcome
  expectations and measures (Circles of Care III
  NOFA, p.3)

15

Items from Standard Infrastructure Program
Announcement and Circles of Care NOFA

• In Section D Evaluation and Data, add the
  following bullet
• Describe how project staff will work with
  evaluation staff to support the evaluation
  effort. The program planning activities and the
  evaluation activities will be expected to inform
  each other in a constant cycle, locally and
  collectively. (Circles of Care III NOFA, p.12)
• The Circles of Care process values
  participatory, community-based outcomes that are
  reflected heavily in the planning process.

16
Items from Standard Infrastructure Program
Announcement and Circles of Care NOFA

• Providing a venue for the Tribal/community to
  receive information and respond to with
  appropriate audience members can include meetings
  at all levels within the community.
• Example Two community-visioning meetings were
  conducted at 2 separate locations within the
  community, and were documented based on invited
  quests, attendance, activities planned to collect
  feedback, and information passed on to community
  participants. This type of community visioning
  planning resulted in setting priority areas to
  build a Youth Development Facility.

17
(No Transcript)
18
Participatory Feasibility Assessment

• Complete the feasibility assessment and process
  evaluation, leading to adoption of the model.
  (Circles of Care III NOFA, p.4)
• Strengthening Tribal capacity for evaluation of
  service systems through ongoing involvement with
  staff and evaluation partners.
• Careful deliberation on setting up effective
  strategies with an evaluation team, to engage
  all participants that have a stake in the
  evolution of the infrastructure.
• This integration of tasks contributes to and OPEN
  style of communication that in turn enhanced the
  collaboration of evaluation and program staff.

19
Participatory Evaluation related to Presidents
New Freedom Commission Goals

• The agenda of the Presidents New Freedom
  Commissions Report towards a transformation of
  mental health systems states
• 2. Mental Health Care is Consumer and Family
  Driven
• Involve consumers and families fully in orienting
  the mental health system toward recovery.
  (Circles of Care III NOFA, p.5)
• The consumer of the community has a voice in the
  process and is able to tell their story at the
  point in the process they choose.
• Parents are empowered through ongoing interaction
  in the process.

20
Participatory Evaluation related to Presidents
New Freedom Commission Goals

• Circles of Care approach emphasizes flexibility
  as stakeholders shift positions and encourage
  converge of a the cultural knowledge base within
  childrens mental health.
• Example A parent that attends parent focused
  support groups from the AI/AN clinic and has a
  child receiving services that is also attending
  the school in the same community will better
  inform on how services are impacting the service
  needs and accessibility.

21
Participatory Evaluation related to Presidents
New Freedom Commission Goals

• The outcome of involving parents can result in a
  cultural definition of Severely Emotionally
  Disturbed
• Emotional disturbance is a temporary disharmony
  often involving the family, school, and
  community, which may affect the mental, physical,
  spiritual, and/or emotional well being of its
  members. Choctaw Nation of Oklahoma, Circles of
  Care I grantee.