Qualitative Research

1
Qualitative Research

• Insuring High Quality

2
Criteria for Quality

• Where Quantitative Researchers seek certain
  elements to insure quality, so do Qualitative
  Researchers look for similar elements.
• The basic issue is one of trustworthiness. In
  other words, How can an inquirer persuade his or
  her audiences (including self) that the findings
  of an inquiry are worth paying attention to,
  worth taking account of? 

3
High Quality inquiry in any sphere must

• Demonstrate its truth value
• Provide the basis for applying it
• Allow for external judgments to be made about the
  consistence of its procedures and the neutrality
  of its findings or decisions.

4
Previously (Module 3) we discussed the criteria
for good research. Those criteria are repeated
below.
CONSTRUCT QUANTITATIVE QUALITATIVE Seriousness Sc
ientific Rigor Trustworthiness Truth
Value Internal Validity Credibility Applicability
External Validity Transferability Consistency Reli
ability Dependability Neutrality Objectivity Confi
rmability
5
Just as a quantitative study cannot be valid
unless it is reliable, a qualitative study cannot
be transferable unless it is credible, and it
cannot be credible unless it is dependable.

• In the following slides, we will examine each of
  the criteria for good qualitative research.
• The first criterion of trustworthiness is
  embodied in the other four criteria credibility,
  transferability, dependability, and
  confirmability.

6
Credibility -- Does it "ring true"?

• Credibility asks the question
• Is there compatibility between the constructed
  realities that exist in the minds of the
  inquiry's respondents and those that are
  attributed to them?

7
How can we improve the credibility of a study?

• Prolonged Engagement (Stay in the field until
  data saturation occurs.)
• counters distortions from researcher's impact on
  the context
• limits researcher biases
• compensates for effects of unusual or seasonal
  events
• Persistent Observations (Consistently pursue
  interpretations in different ways in conjunction
  with a process of constant and tentative
  analysis. Look for multiple influences. Search
  for what counts and what doesn't count.)
• Triangulation (The best way to elicit the various
  and divergent constructions of reality that exist
  within the context of a study is to collect
  information about different events and
  relationships from different points of view.)
• ask different questions
• seek different sources
• utilize different methods
• Referential adequacy (What materials are
  available to document your findings? Video tape
  provides a good record but it can be obtrusive.)

8
How can we improve the credibility of a study?
(continued)

• Peer Debriefing (This is done with a similar
  status colleague (not with a junior or senior
  peer) who is outside the context of the study and
  who has a general understanding of the nature of
  the study and with whom you can review
  perceptions, insights, and analyses.)
• provides a "devils advocate"
• tests working hypotheses
• helps develop next step
• serves as a catharsis
• Member Checks (Go to the source of the
  information and check both the data and the
  interpretation.)
• assesses intentionality of respondents
• corrects errors
• provides additional volunteer information
• puts respondent on record
• creates an opportunity to summarize which is the
  first step to data analysis
• assesses the overall adequacy of the data in
  addition to individual data points

9
Transferability (extent to which the findings
can be applied in other contexts or with other
respondents)

• Most contemporary researchers view applicability
  in terms of generalizability and address the
  issue by focusing on those aspects of the inquiry
  that do not shift. The naturalistic researcher
  maintains that no true generalization is really
  possible all observations are defined by the
  specific contexts in which they occur.
• The naturalistic researcher does not maintain
  that knowledge gained from one context will have
  relevance for other contexts or for the same
  context in another time frame. In a traditional
  study it is the obligation of the researcher to
  ensure that findings can be generalized to the
  population in a naturalistic study the
  obligation for demonstrating transferability
  belongs to those who would apply it to the
  receiving context (the reader of the study).

10
Strategies for Transferability

• Thick Description
• Because transferability is a naturalistic study
  depends on similarities between sending and
  receiving contexts, the researcher collects
  sufficiently detailed descriptions of data in
  context and reports them with sufficient detail
  and precision to allow judgments about
  transferability to be made by the reader.
• Purposive Sampling
• In contrast to random sampling that is usually
  done in a traditional study to gain a
  representative picture through aggregated
  qualities, naturalistic research seeks to
  maximize the range of specific information that
  can be obtained from and about that context by
  purposely selecting locations and informants that
  differ.

11
Dependability

• An inquiry must also provide its audience with
  evidence that if it were replicated with the same
  or similar respondents (subjects) in the same (or
  a similar) context, its finding would be
  repeated.

12
Increasing Dependability

• Don't Need to Do It Since there can be no
  validity without reliability (and thus no
  credibility without dependability), a
  demonstration of the former is sufficient to
  establish the latter. If it is possible using the
  techniques outlines in relation to credibility to
  show that a study has that quality, it ought not
  to be necessary to demonstrate dependability
  separately. (Arguable)
• Overlap In effect, overlap methods represent
  triangulation which is typically undertaken to
  establish validity, not reliability, although
  demonstration of the former is equivalent to
  demonstration of the latter. (Still Arguable)
• Stepwise Replication Teams deal with data
  sources separately and, in effect, conduct their
  inquiries independently. (Not recommended)
• Inquiry Audit An auditor examines documentation
  (through critical incidents, documents, and
  interview notes) and a running account of the
  process (such as the investigator's daily
  journal) of the inquiry. The auditor examines the
  process of the inquiry, and in determining its
  acceptability the auditor attests to the
  dependability of the inquiry. The inquiry auditor
  also examines the product--the data, findings,
  interpretations, and recommendations--and attests
  that it is supported by data and is internally
  coherent so that the "bottom line" may be
  accepted. This latter process establishes the
  confirmability of the inquiry. Thus a single
  audit, properly managed, can be used to determine
  dependability and confirmability simultaneously.

13
Confirmability (This is the degree to which
the findings are the product of the focus of the
inquiry and not of the biases of the researcher.)

• Confirmability Audit Trail
• An adequate trail should be left to enable the
  auditor to determine if the conclusions,
  interpretations, and recommendations can be
  traced to their sources and if they are supported
  by the inquiry.

14
Lincoln and Guba (1985) noted that Halpern (1983)
suggested six classes of raw record data to be
reviewed.

• Raw Data recorded videotapes, written field
  notes, documents, survey results
• Data Reduction and Analysis Products write-ups
  of field notes, summaries and condensed notes,
  theoretical notes such as working hypotheses,
  concepts, and hunches
• Data Reconstruction and Synthesis Products
  themes that were developed, findings and
  conclusions, final report
• Process Notes methodological notes,
  trustworthiness notes, audit trail notes
• Material Relating to Intentions and Dispositions
  inquiry proposal, personal notes, expectations
• Instrument Development Information pilots, forms
  and preliminary schedules, observation formats,
  surveys

15
A more detailed account of conducting an audit
can be found in
Lincoln, Y. S., Guba, E. G. (1985).
Naturalistic inquiry. Newbury Park, CA Sage.