The Quality of Qualitative Research

1
The Quality of Qualitative Research

• Uwe Flick
• Alice-Salomon-University
• of Applied Sciences Berlin
• Flick_at_asfh-berlin.de

2
What is qualitative research and are we referring
to?

• Theoretical and methodological schools
• Local approaches and schools
• Discipline-specific developments
• Different fields of application
• Qualitative health research
• Qualitative evaluation
• Same answers to the quality question across the
  various areas and contexts?

3
Quality of qualitative research

• Still an unanswered question
• Answers expected from outside
• Answers needed for further establishing
  qualitative research
• Answers needed for reassuring qualitative
  research in times of evidence and mixed methods
• One size fits all?

4
Four ways to answer the question of quality in
qualitative research

• To apply the traditional criteria
• To reformulate traditional criteria like validity
  and reliability
• To design new appropriate criteria
• To use strategies of promoting quality

5
Answer I

• Traditional
• Criteria

6
Reliability, validity and objectivity

• are strongly based on standardisation of methods
  and situations
• are based on concepts like repeatability
• are based on a detachment of the person of the
  researcher
• are incompatible with the degrees of freedom, of
  flexibility, and of non-standardisation of
  qualitative research

7
Answer II

• Reformulation of Traditional
• Criteria

8
Validation in Qualitative Research

• Communicative validation
• Pragmatic validation
• To validate is to question
• To validate is to check
• To validate is to theorise (Kvale 2007)

9
Answer III

• New
• Criteria

10
Credibility of Qualitative Research

• Prolonged engagement and persistent observation
  in the field and triangulation
• Peer debriefing
• The analysis of negative cases
• Appropriateness of the terms of reference of
  interpretations and their assessment
• Member checks (Lincoln and Guba 1985)

11
Criteria for Grounded Theory

• Credibility
• Originality
• Resonance
• Usefulness (Charmaz 2006)

12
Credibility as Criterion?

• Benchmark problem
• How much credibility needed?
• Member checks all members or some
  membersconsent?
• Criterion or bona fides?
• Criteria or strategies?

13
Quality in Qualitative Evaluation

• Research should be
• contributory in advancing wider knowledge
• defensible in design by providing a research
  strategy which can address the evaluation
  questions posed
• rigorous in conduct through the systematic and
  transparent collection, analysis and
  interpretation of qualitative data
• credible in claim through offering well-founded
  and plausible arguments about the significance of
  the data generated (Spencer et al. 2003).

14
Answer IV

• Strategies

15
Triangulation

• Different methods also for collecting data
• Equal weighting of methods or approaches
• Systematic use of different methods
• Integration/reflection of the theoretical
  backgrounds of the different methods
• Addressing different levels, for example
• subjective meaning and social structure
• process and state,
• knowledge and activity
• knowledge and discourse
• Purposeful choice and use of methods

16
Quality Management in Qualitative Research

• a definition of the goals to be reached and the
  standards of the project to be kept, which should
  be as clear as possible all researchers and
  co-workers have to be integrated in this
  definition
• a definition, how to reach these goals and
  standards and more generally the quality to be
  obtained a consensus about the way how to apply
  certain methods maybe through joint interview
  trainings and their analysis, are preconditions
  for quality in the research process
• a clear definition of the responsibilities for
  obtaining quality in the research process and
• the transparency of the judgement and the
  assessment and quality in the process.

17
Indication in Psychotherapy and Medicine

• which
• treatment
• or
• therapy?

• Which disease
• which symptoms indicate
• which diagnosis
• which population

18
Indication in Qualitative Research

• Which issue
• which population
• which research question indicate
• which knowledge
• of issue and
• population

• which
• method(s)
• or
• which combination of methods?

19
Indication in Qualitative Research

• When is/are which method(s) appropriate and
  indicated?
• Are there criteria for a rational decision for
  or against specific methods?

20
Sound qualitative research ..

• ... is based on previous research and refers to
  state-of-the-art theoretical, methodological and
  topic-related literature. In this sense it stands
  on the shoulder of giants.
• What about explanatory research, new fields and
  grounded theory research in the strict sense of
  the word?
• ... is clearly linked to a social scientific
  theory like Symbolic Interactionism,
  Ethnomethodology, Cultural Representation Theory,
  Hermeneutics, Phenomenology, Discourse Theory or
  the like.
• These are not theories in the strict sense of the
  word (as far as concerning our context) but
  research perspectives, which are taken on an
  issue (or field ) under study.

21
Sound qualitative research ..

• ... uses only data appropriate within the
  framework of the particular theory and applies
  corresponding methods for analysis and
  interpretation.
• This statement is too much theory-driven
  Shouldnt the research perspective be appropriate
  to the issue under study and then only data be
  used that are appropriate to the issue under
  study?

22
Sound qualitative research ..

• Depending on the particular theory and methods
  used qualitative research presents an emic as
  well as an etic perspective. Including the
  subjective or native point of view does not
  discharge the researcher from the task of
  developing and defending his/her own
  interpretation which may or may not correspond to
  that of the research subjects.
• Nevertheless, it may be appropriate to seek
  members consensus or disagreement for results or
  evaluations coming from qualitative research,
  depending on purpose and issue of the research

23
Sound qualitative research ..

• ... is neither partisan nor indifferent toward
  social problems, but it aims at differentiating
  scientific analysis from personal and political
  stance on a given topic. Qualitative research
  reveals the complexity of the social world
  instead of lending itself to simplifications.
• In the context of practical/political use (and
  relevance) of qualitative research it may be
  necessary to elaborate qualitative results in a
  way that allows a political argumentation or
  decision and to condense them in a way that they
  are comprehensible and relevant for
  non-researchers if we want to become/remain
  relevant with our research.

24
Sound qualitative research ..

• .... is contextualised and the relevant
  information considering the concrete context is
  provided (without jeopardising the anonymity of
  informants, organisations etc.). Generalisations
  of results beyond this context must be discussed.
  
• Internal generalizability refers to the
  generalizability of a conclusion within the
  setting or group studied, while external
  generalizability refers to its generalizability
  beyond that setting or group (Maxwell 2005, p.
  115).

25
(No Transcript)
26
Types, Goals and Interest groups ...
27
Who may be interested ..

• Three answers given
• Anyone, who ....
• A circular answer .... qualitative research
  methods are relevant for those, whose research
  question can be answered with the relevant
  qualitative approach and for which appropriate
  data are available.
• It depends ... As there are not enough common
  characteristics in qualitative research

28
Who may be interested ..

• Three possible answers
• Those who are interested in fields or topics,
  which are not well accessible for other forms of
  data
• Those who are interested in having a
  comprehensive range of approaches and tools for
  social research
• Those, who are interested in exploring new
  fields, finding new insights

29
Types of qualitative methods

• As the examples of interviews and textual data
  show, we should distinguish between
• Data collection
• In this step the interview is a qualitative
  method depending on how it is applied, but not on
  how it is analysed in the end
• Data analysis
• Other forms of data are mostly transferred into
  texts when it comes to analysis (oral data are
  transcribed, sooner or later, images are
  described and analysed by using words and texts
• Writing about data and findings
• Any presentations has to be reductionistic in
  some way, if the reader shall grasp the message
  of the report. It depends rather on how adequate
  the selected form of reductionism is to what is
  under study - in collecting, analysing and
  presenting the data.

30
Types of qualitative methods

• Inductive and deductive approaches
• Good qualitative research is always a combination
  of induction (in the beginning) and deduction
  (along the way)
• Research always starts from some assumptions
  about the field and the issue, it depends rather
  on how far these assumptions are made explicit
  and subject of revision against the data

31
Goals of qualitative methods

• The major goal should be to use methods that are
  appropriate to issues under study, but also to
  the fields and participants where these issues
  are studied
• Indication of methods starts from selecting the
  appropriate ones from the existing range of
  methods and approaches or from developing new
  ones if necessary
• Do methods have goals?
• What are the goals of researchers which are
  normally linked with using specific qualitative
  methods?
• Which methods allow (support, facilitate etc.) to
  pursue which goals more adequately?

32
Teaching Qualitative Methodology
33
Goals of Teaching

• MA/PhD level
• Critical selection and reflection of a variety of
  research methods for ones own research project
• Independent use of (a variety of) methods for
  ones own research
• In general
• Combination of an overview and of practical
  application of selected examples

34
Goals of Teaching

• Bachelor level in applied areas
• Ability to understand research examples and to
  distinguish good from bad research
• Ability to apply a certain number of methods for
  answering a limited research questions in a
  thesis
• MA/PhD level
• Critical selection and reflection of a variety of
  research methods for ones own research project