Using Focus Group Methodology to Assess CommunityBased Program Sustainability

1
Using Focus Group Methodology to Assess
Community-Based Program Sustainability

• Elisabeth Maring,
• Moireen Clark, Kendra Wells, Edith
• Hull, Tammy Humberson, Donielle Inskeep

CYFAR Conference 2009, Baltimore, MD. For
additional information, contact efmaring_at_umd.edu
2
CYFAR New Communities Project

• Three after-school programs in Maryland target
  low-income, school-aged youth in order to provide
  them with access to academic enrichment,
  computers and technology-based learning, as well
  as to increase computer literacy skills,
  strengthen academic performance and facilitate
  students use of technology.
• The programs are located across the state in
  Garrett, Somerset, and Frederick Counties.

3
CYFARCommunity Youth Connections

• Our project addresses community needs of
  children, youth and families in rural and urban
  communities through the development of community
  systems that support positive youth development.
  
• With youth, families, and community partners,
  this project builds community capacity to provide
  quality youth development experiences in
  community-based, afterschool settings.
• Young people participate in non-formal
  educational programs to support their academic
  goals and develop skills to become stronger
  citizens in their communities.
• The projects build on the assets of children,
  youth, families and communities and engage
  community members through partnerships and
  collaborative efforts.

4
Garrett County, MarylandAfter-School Extension
Project
The CYFAR New Communities Project in Garrett
County has met an identified need by bringing the
resources of Extension into after-school programs
across the county. Programs utilize an outdoor
education center, computer lab, life skills
curricula, and the expertise of the 4-H Youth
Development educator and program assistant.
5
Garrett County, MarylandAfter-School Extension
Project

• Impacts
• Access to enriching afterschool educational
  programs and field trips to supplement and
  enhance school education.
• Positive interaction with other youth and
  afterschool educators helped youth to build
  nurturing and beneficial relationships and
  increase interpersonal skills.
• Access to a variety of education activities
  including quilting, participation in the county
  fair, cooking, engineering projects, and
  robotics.

6
Somerset County, MarylandCyber-Town at the
Wilson Center
This computer lab was designed to be the
educational center of the community, providing a
safe and productive learning center for the
families living in Woodrow Wilson Housing. It has
also served as a resource to seniors, as well as
school age and pre-school youth. The center is
not limited to the residents of the housing
authority, as it has served many residents of the
town of Crisfield and the entire county.
7
Somerset County, MarylandCyber-Town at the
Wilson Center

• Impacts
• Regular attendance at the lab can positively
  impact youth.
• The computer lab is a safe and educational
  environment for after-school activities. This
  type of environment helps youth stay out of
  trouble in a community that struggles with a high
  crime rate.
• The computer lab provides activities for reading
  comprehension, vocabulary practice, and spoken
  communication.
• Youth who have access to these computers can gain
  an increased awareness of opportunities in higher
  education and careers in math, science, and
  technology.

8
Frederick County, MarylandSagner PAL Center
Technology Club
This lab has primarily served youth, ages 7-12
who live in the Sagner housing units where the
PAL Center is located. A 4-H educator provides
informal instruction to youth on software
applications and educational computer games.
Youth are encouraged to use the computers for
homework assignments and learn basic computer
skills and software.
After participating in instructional activities,
youth can earn time to play computer games and
learn about computer applications. They are
encouraged to join Reading Club to read aloud
together, then type re-told stories and keep book
journals on the computers.
9
Frederick County, MarylandSagner PAL Center
Technology Club

• Impacts
• Time and place provided for completion of
  homework assignments with adult supervision and
  help.
• Youth have access to education computer games and
  instructor led classes to supplement their
  education and improve school performance.
• Access to other enriching activities including
  quilting, gardening, reading club, and
  participation in the county fair.

10
New Communities ProjectEvaluation

• The project is in its fifth and final year of
  funding. The evaluation team noted several areas
  that had not been addressed by previous program
  evaluation methods. Notably, participant
  satisfaction and program effectiveness had
  consistently been evaluated based on self- and
  administrator-reports. These measures showed
  regular attendance rates, an overall increase in
  students computer literacy skills, and
  participant satisfaction with the afterschool
  programs.

The evaluation team recognized the need to assess
program strengths in an effort to gauge
sustainability once funding for the project
ceased.
11
Focus Groups An Evaluation Tool

• Focus groups methodology is a unique tool for
  obtaining information in research and evaluation
  (Morgan Kruger, 1993 Mohr, 2001).
• Focus groups help to ground discussions in
  reality because participants share personal
  experiences to support their opinions (Knodel,
  1993).
• The evaluation team used focus groups methodology
  because this project provided a unique
  opportunity to engage stakeholders and access
  their feedback in the three after-school sites.

12
Focus Group Methodology Achieving Intended
Outcomes

• The goal of the focus groups was to assess
  program needs and strengths from the perspective
  of stakeholders in the community including
  parents and guardians, police officers,
  Rotarians, local business owners who contribute
  time and/or materials to the after school
  programs, and other community members.
• A pilot focus group was co-led by the project
  P.I. and project evaluator during a project team
  meeting to provide coordinators with an
  opportunity to generate ideas for the stakeholder
  focus groups to be held in their counties.

13
Preparing to Conduct Focus Groups

• IRB Approval
• Stakeholder participation
• Scheduling
• Preparing materials
• Running the Focus Groups
• Transcribing Coding
• Analyzing Data
• Drawing Conclusions from Data

14
Obtaining IRB Approval

• An Institutional Review Board (IRB) is a
  committee designated by an institution to help
  assure the protection of the rights and welfare
  of human subjects.  The IRB approves the
  initiation of and conducts periodic reviews of
  research involving human subjects.  Investigators
  also share the responsibility for protecting
  human subjects.
• To submit an IRB proposal, you must draft a
  detailed description of the project, anticipated
  risk to participants, procedures, confidentiality
  clause, informed consent, permission to record,
  intended uses of data, etc.

15
Stakeholder participation

• Who will you invite to participate?
• How will you obtain contact information?
• Will you provide any incentives for
  participation?
• How will you explain the purpose of these focus
  groups and how they may benefit stakeholders?
• What kind of commitment will you ask of them?
  How long should they expect to be there? What
  should they anticipate discussing?

16
Scheduling

• Determine a month-long window in which you can
  be available to conduct the focus groups before
  you contact the stakeholders.
• In your initial contact, suggest 2-3 potential
  dates and times for the focus groups to be held.
• Request responses by a clearly stated deadline!
• Schedule the focus group on the date that the
  most stakeholders are able to attend.
• Ideally, a focus group should have between 8 and
  12 participants.

17
Preparing for Focus Groups

• The role of the focus group facilitator is to
  ensure that the discussion remains on topic in
  order to gather the richest data. He or she
  should play a minimal but significant role in
  maintaining the flow and direction of the
  conversation.
• Several broad, open-ended questions should be
  prepared ahead of time to be used as a guideline
  for discussion it is also important, however, to
  allow for spontaneity and encourage shared
  personal experiences of the participants.
• A consent to record must be signed by all
  participants before the focus group commences (in
  accordance with your approved IRB protocol).

18
Running the Focus Group

• The facilitator welcomes the group and initiates
  the discussion with the lead-off question.
• A co-facilitator records process notes
  important aspects of the interaction/discussion
  that will not be visible or noticeable on the
  audio-recording.
• CYFAR focus groups were recorded using a handheld
  digital voice recorder/USB port device (Olympus
  WS-331M). Once the recording is complete, the
  device plugs directly into the computer and opens
  the sound file in MP3 format.
• Members of all three focus groups provided
  consent to be audio-recorded

19
Transcribing Coding

• The Olympus AS-2300 PC Transcription Kit was used
  to transcribe the CYFAR focus groups a foot
  pedal allows you to stop and start the clip
  without having to move your hands from the mouse
  to the keyboard. The software also allows you to
  slow the recording down in order to decipher
  hard-to-hear words.
• In the transcribing stage, data were transformed
  from oral speech to written text. This procedure
  enabled the researchers to prepare the focus
  group material from the digital recording for
  analysis. The transformation from oral to
  written mode involved judgment and was done
  carefully to reflect the participants words
  verbatim.

20
Analyzing Data

• Following transcription, documents are
  transported into NVivo, the most recent version
  of the QSR qualitative data analysis software
  (www.qsrinternational.com).
• A secure filing system for paper files must also
  be created.
• There are three main phases of data analysis in
  qualitative research

21
Drawing ConclusionsThree phases of analysis

• Open Coding
• The process of breaking down, examining,
  comparing, conceptualizing, and categorizing
  data.
• Axial Coding
• New connections are made between categories and
  subcategories defined in open coding. According
  to LaRossa (2005), the difference between open
  coding and axial coding is that in open coding,
  the researcher is essentially developing the
  categories while in axial coding, the
  relationship between or among categories is
  explicitly examined.
• Selective Coding
• Data are interpreted to build a story that
  connects the categories defined through open and
  axial coding (Creswell, 1998).

22
Interview Protocol for community Focus Groups

• What difference does this program make for
  participating youth?
• How does this program provide a safe place for
  youth?
• What are your ideas to make this program
  sustainable, so that community members will act
  to keep it from going away?
• What resources are currently available to keep
  this program going after the funding is completed
  at the end of this year?
• Who are the individuals and groups that need to
  be at the table to make this program sustainable?
• What ideas do you have for replicating or
  expanding this program in other communities?

23
Themes
24
Results Funding Strategies

• Garrett County notes the challenges even when
  funds available
• I think that were poised where if theres an
  opportunity, we can probably try to take
  advantage. Its just never the same as having
  one grant that funds what you do. How many
  quarterly reports can you do?
• Somerset County participants question program
  sustainability
• the University is having its funding issues
  too. So, I dont know, its a strong possibility
  that come June 30th of 2009, unfortunately, the
  lab may close its doors. And thats always been
  the worry for the MCE administrator, the
  driving force behind getting this startedwe
  didnt want to start a program and then
  disappear.

25
ResultsCommunity Stakeholder SupportWho else
needs to be at the table?

• Frederick County Non-profit civic organizations
• What were looking at is specific non-profit
  organizations that are civic-minded
  organizations, that look for particular cause
  that they believe inOur karate program is like
  that because we found a group, it was the group
  that built the building, that really wanted to
  see the Sagner PAL continue. So they invested in
  the karate program, they support the entire
  program. Wed like to find that exact scenario
  for this computer lab, so that its sustainable.
  Thats our goal.
• Somerset County Parents
• Now Im not gonna say we dont have great
  parents around here because we do. Theyre in
  this room now, you know. But there are kids here
  that need help. Sometimes, its not easy to help
  them because we as adults see their parents and
  we know what their parents are doing, but its
  hard to reach out to them, because you dont want
  them to feel as though you pushing at it. But,
  the people that we gotta reach is not the kids.
  Its the parents.

26
Results Leveraging Partnerships

• Garrett
• Ill have to throw in there, I think this was one
  of the real successes as a county of cooperation
  because robotics coordinator and her friends at
  the Board of Education were doing the robotics
  and first Lego League and that kind of things,
  and they have the support of the Board of Ed, and
  then the Commissioners on this facility. But
  then we were able to help with some of the new
  actual pieces of equipment that they needed,
  through CYFAR
• Somerset
• So thats more like a collaborative effort. So
  those things, we all work together to help kind
  of build a foundation for the program actually
  three different agencies working together. Our
  agency, It Takes a Village, was started mainly by
  parents, which was a good thing, it was all
  parents. We got together and said ok we need to
  find some things for these kids to do during the
  summer.

27
Results For the children

• Academic Enrichment
• Our after-school program, basically is focused
  on reading, because one of the things that we
  found was that there is a major hole between
  their reading skills and their grade level. So,
  we had actually the department together a while
  ago to start trying to do some stuff in the
  summertime, you know with the computer and you
  know, finding programs that will help them with
  that as well. . Somerset County Participant
• A safe place
• Lately theres been an increase in drug
  activity, just even right outside the center.
  And in here, there is no, there is no fear. You
  know, they know they officers are here to help
  them, which creates a bond, you know, its just
  priceless, in the hopes that a lot of these kids
  will come tell us stuff later on. Frederick
  County
• Resource needs
• I know in speaking with some of the children in
  some of the programs, when they get there they
  would have a healthy snack, they would have this
  healthy lunch, and then they had another healthy
  snack before they left. I would say across the
  board there were a small percentage, but possibly
  1 or 2 children in each program, where thats why
  they were there.

28
Limitations/Conclusions

• Strengths
• Helps to build sustainability focus
• Networking
• Represent community members in their own voices
• Limitations
• Small sample size
• Question of replicability