Qualitative Methods and How to Dovetail With Quantitative Methods

1
Qualitative Methods and How to Dovetail With
Quantitative Methods

• Melanie Percy, RN,PhD,CPNP
• The University of Texas at Austin
• School of Nursing
• Dawn Nixon, MA
• Young Wings
• Red Cliff Birth to Six Mental Health Program
• Red Cliff Ojibwe Reservation
• Wisconsin

2
(No Transcript)
3
(No Transcript)
4
(No Transcript)
5
(No Transcript)
6
What is Young Wings?

• Initial Definition
• A pediatric mental health program, based upon
  Dr. T. Berry Brazelton Touchpoints model, located
  within a tribal early childhood center. It will
  provide early identification of, and intervention
  for, signs of developmental derailment in
  children aged birth to 6.
• (Goals 4.1, 5.3, 5.4)

7
(No Transcript)
8
Initial Plan for Services

• Developmental screening.
• Identification of children with early signs of
  difficulty.
• Interventions
• Psychological assessment.
• Parent education.
• Individual play therapy.
• Referral to community partners (e.g. Evenstart
  Literacy program).

9
Barriers to Receipt of Mental Health Services
Addressed

• Lack of pediatric mental health availability.
• 60 mile round-trip.
• 3-6 month wait.
• Lack of reliable transportation.
• Lack of cultural competency
• Limited understanding of hx., Native American
  values, range of acculturalization and
  significance of spirituality and ritual.
• Stigma for receipt of mental health services.
• (Goals 1.1, 3.1, 3.2)

10
(No Transcript)
11
Initial Evaluation Plan

• Pre- and post- administration of the Parenting
  Stress Index.
• Client satisfaction survey.
• Child/family identification rate.
• Longitudinal statistical analysis of county child
  abuse and neglect statistics.
• Analysis of community utilization rates.

12
What is Young Wings?

• All that was in the initial plan, plus
• A valued member of the ECC management team who
  effects and writes policy from a mental health
  perspective.
• An advocate for parents with the local school
  district. Liaison between ECC, school district,
  and community.
• A resource for the ECC and community during times
  of stress.
• And more

13
(No Transcript)
14
Questions..

• How would the families/staff/community define
  Young Wings? How is it perceived?
• How has Young Wings been accepted by
  families/staff/community?
• What factors make Young Wings successful and
  accessible?

15
(No Transcript)
16
Core Principles of Qualitative Evaluation

• Each person/community is unique.
• Each person/community deserves respect.
• People and communities should be understood in
  context and holistically.
• The process (how things are done) is as important
  as the outcomes (what is achieved).
• Information should be openly shared and honestly
  communicated. Patton, 1990

17
Why Qualitative?

• The personal nature of qualitative inquiry
  derives from its openness, the evaluators close
  contact with the program, and the procedures of
  observation and in-depth interviewing,
  particularly the latter, which communicate
  respect to respondents by making their ideas and
  opinions (stated in their own terms) the
  important data source for the evaluation.
• Patton, 1990 pg.124

18
Cornerstones of Evaluation

• Why?
• To validate your program goals.
• To guide future program development.
• To maintain or increase funding.

19
Basic questions to consider

• What are your goals?
• Who is your population?
• Who are the stakeholders?
• Where does your funding come from and what will
  you have to do to continue it?

20
Quantitative vs Qualitative

• Quantitative - statistical analysis of an
  objective number. That number is usually created
  by counting the number of clients served, or
  statistical analysis of data received.
• Qualitative - using written or spoken interviews
  to determine impact of program on families. Then
  analyzed for themes and reported as a narrative.

21
Why quantitative?

• Use standardized measures that fit diverse
  opinions and experiences into predetermined
  categories.
• Statistically significant findings.
• Broad, generalizable set of findings.

22
Why Qualitative ?

• Can study selected issues in depth.
• Provides detailed data
• Direct quotation, careful descriptions are used
  in an open- ended narrative.
• No quantitative tools may exist for variables of
  interest.

23
Comparing qualitative and quantitative data

• Please rate your satisfaction with the
  healthcare you child has received in this clinic.
• a. very satisfied
• b. somewhat satisfied
• c. satisfied
• d. not satisfied
• e. very dissatisfied

24
Please tell me about your experience of
receiving healthcare for your child in this
clinic.

• Kay Gibbons is wonderful! Very kind, thoughtful.
  I hope that this clinic is not taken away because
  we need it badly, and its helped us a lot
  already. I am happy because they speak Spanish.

25
Philosophy behind qualitative evaluation

• What is reality? What is knowable?
• What is the nature of human experience?
• Value seeing the world as understood by the
  client.
• Desire to capture accurately and thoroughly the
  experiences, and perspectives of the staff,
  administration, and participants of the program
  being evaluated.

26
Key Elements

• Observing
• Participant observation of program
• Observing during interviews (nonverbal behavior,
  relationship with interviewee)
• Interviewing
• Formal interviews
• Informal, naturally occurring conversations.

27
Doing Interviews

• Qualitative evaluation means the evaluator must
  quickly attempt to develop an insiders view of
  what is happening
• While maintaining an outsiders ability to
  describe the experience.

28
Gaining entry

• One of most difficult parts of qualitative
  evaluation is knowing whom to interview or
  observe.
• Need key informants. People who have insight
  and respect in the community.
• Key informants can explain and interpret what the
  evaluator is experiencing.

29
Doing Qualitative Evaluation

• Consider
• what was said?
• what was happening?
• what was the context?
• what is the history of the program?
• the larger community?

30
Inductive Analysis

• Audiotapes will be transcribed and transcripts of
  tapes will be analyzed.
• Attempt to make sense of program without imposing
  pre-existing expectations on the program setting.
• Begin with specific observations and builds
  towards generalized patterns

31
Analyzing Data

• Multiple computer programs have been developed to
  analyze qualitative data
• Ethnograph, Hyperesearch, Nudist, Hermann, etc
• Transcripts are entered into program and
• themes are developed

32

• Statements are placed on cards and sorted
  according to their theme.
• Themes are collapsed and reorganized as major
  patterns are seen and categorized.
• Then rival or competing themes and explanations
  should be examined and rejected or explained.

33
Presenting Data

• Detailed descriptions and in-depth quotations are
  the data that are reported to validate the
  categories decided upon.
• Presenting qualitative data can be very lengthy
  and time-consuming, but also interesting and
  insightful.

34
Objectivity and Truth vs. Reliability and Validity

• Important that the evaluator is neutral - has no
  theory to prove, and no predetermined results to
  support.
• Finally results need to be brought back to the
  program and discussed to determine the goodness
  of the data.
• Does it fit the program? Does it seem factual and
  confirmable?

35
Summary

• Qualitative evaluation can be a useful tool for
  understanding the impact you are having on a
  community.
• Conducting a qualitative evaluation can be
  difficult and time-consuming but ultimately
  rewarding as you gain new understandings about
  your program and the people you serve.

36
Now its your turn

• In groups of 3-4 write an evaluation question
  from your clinic as a qualitative question and as
  a quantitative question.
• Discuss in your groups what kind of information
  would you expect from each question. Be prepared
  to have a spokesperson to present this to the
  larger group.

37
Group Discussion

• A spokesperson from each group will present the
  two questions from the group.
• For group discussion
• What makes one qualitative, and one quantitaive?
• How will the information be different based on
  these questions?
• How is each question valuable to the evaluation?

38
Mock Interviews

• Who would your key informant be? Why did you
  choose that person?
• Let the group become a focus group, pick one
  person to be the interviewer. Ask one of your
  questions, and see what kind of information you
  receive.
• Report to group.

39
Analysis

• Next, take tapes to the data transcriber, or
  transcribe them yourself.
• Once tapes have returned, go ahead and start
  analyzing - look for similar subject lines.

40
What does it mean?

• Collapse categories
• What does it all mean?
• May want to go back to your key informants, to
  discuss what you think the interviews meant.

41

• 4. Have you ever brought your child to the school
  based health center?
• 5. What was that experience like?
• 6. Is there anything else that you would like to
  tell us about your or your childs experience
  with healthcare?

42
How to present this to stakeholders

• Numbers (statistics) and then talk about themes
  discovered in qualitative method.
• Then present theme, quotes that support that
  theme, and then the final analysis of what it all
  means.

43
Example
44
Qualitative Evaluation of School Based Health
Centers

• City of Austin asked to have an evaluation of
  their school-based health centers.
• After meeting with city officials, health
  department administrators, school officials, and
  school-based health center personnel it was
  determined to conduct several focus groups with
  parents of children seen at the center.

45
Interviews

• PTA and community leaders were asked to identify
  people who would like to participate in the focus
  group.
• Also a notice went out in the school newsletter
  asking people to participate.
• Participants were given a 20 grocery store
  coupon to thank them for their time food and
  child care were provided.

46
Focus groups

• Focus groups were formed with 6-8 people each.
• English language and Spanish language focus
  groups were conducted separately.
• Focus groups were audiotaped and note takers
  (undergraduate students) took notes as people
  talked.

47
Questions

• Questions began as very generalized and became
  more specific.
• 1. Tell me about a time when your child was
  sick and you took your child to a health care
  provider.
• 2. Tell me about a good experience with a
  healthcare provider.
• 3. Tell me about a bad experience with a
  healthcare provider.

48
Themes

• Mother's descriptions of their experiences with
  the health care system were organized into the
  following thematic categories
• "doing what I have to",
• "caring for my child",
• "no respect".

49
Doing what I have to do

• They wont see her if she doesnt have a fever,
  so I always say she has a fever when I go to take
  her in.
• Its a long way to the health dept clinic. Before
  the SBHC was here, I would take three buses to
  get to the clinic. Sometimes it would be pouring
  rain, and Im standing out at the bus stop
  waiting with my three small children, and one of
  them is already sick. But you have to take them
  in when they get sick.

50
Caring for my child

• When my child is sick, I or my husband stay home
  from work. We wont get no pay that day but
  thats what you have to do. Ive gotten fired
  before.
• The medicine is so expensive, sometimes it takes
  all the money we have for the week.

51
No Respect

• I go to the clinic and it takes all morning. They
  dont care that you are just sitting and sitting
  there. You can see the nurses and doctors just
  drinking coffee, and here we are just waiting.
• Even the doctors and nurses talk down to you.
  They act like you dont know nothing. Im a good
  mother n I love my kids. Im taking care of them
  the best way I know how.

52

• Mothers described feeling angry and frustrated
  when they were forced to wait for long periods of
  time with sick children.
• The attitudes of professional and
  non-professional staff strongly influenced their
  satisfaction with the health care experience.
• Mothers were most satisfied when their child was
  seen quickly by someone who was caring and
  attentive to their needs.

53
Conclusions

• This study emphasizes the importance of valuing
  and respecting the knowledge that mother's bring
  to the health care environment and clearly
  demonstrating that respect throughout the health
  care setting.

54

• Questions? Further discussions?
• Resources
• Patton, M. (1996). Utilization Focused
  Evaluation The New Century Text. Sage
• Patton, M. (2001). Qualitative Research and
  Evaluation Methods. Sage.
• Patton, M. (1995). How to Use Qualitative Methods
  in Evaluation, Vol. 4. Sage.

55

• Shaw, I. (1999). Qualitative evaluation. Sage
• Denzin, N., Lincoln, Y., (1998). Collecting
  Interpreting Qualitative Materials. Sage
• Babiss, F. (2004). An Ethnographic Study of
  Mental Health Treatment Outcomes Doing What
  Works. Haworth Press.

56
(No Transcript)