Standards for Qualitative Research in Education

1
Standards for Qualitative Research in Education

• Margaret Eisenhart

2

• A Prolegomenon on Standards
• Howe, K. Eisenhart, M. (1990). Standards for
  qualitative (and quantitative) research A
  prolegomenon. Educational Researcher, 19 (4),
  2-9.

3

• The Trajectory of Debate 1980-90
• - Entrenched quantitative methods and standards
  vs. a highly suspect newcomer
• - Two strands
• 1. Research methods vs. epistemologies
• 2. Research methods for different purposes

4

• Our position
• Side with the second strand
• Distinguish standards for specific research
  designs from standards for the general value of
  an educational research study
• Focus on standards for general value

5

• The Nature of General Standards
• Abstract
• Refer to the educational value (or value added)
• Apply to both quantitative and qualitative methods

6

• 5 General Standards (guiding principles)
• Specific data collection and analysis methods
  must be competently applied.
• Contributions from previous theories, previous
  research, other relevant literature and
  researcher subjectivity (bias) should be made
  explicit
• The fit between research questions, data
  collection and data analysis should be clear.

7

• Overall warrant (validity) must be established by
  rejecting rival or alternative inferences or
  explanations.
• Value constraints must be recognized and met.
• --Educational research should be valuable to
  education (external value).
• --Educational research should be ethical
  (internal value).

8

• Whats Happened to the Debate about Qualitative
  Standards since 1990?
• Epistemology takes center stage
• Internal value constraints receive lots of
  attention
• Other general standards receive little attention

9

• National Research Council Committee on Scientific
  Principles in Education Research (est. 2000).
• NRC (2000). Scientific research in education.
  Washington, DC The National Academies Press.

10

• SREs 6 General Principles for High Quality,
  Scientific Research in Education
• Pose significant questions that can be
  investigated empirically
• 2. Link research to relevant theory
• Use methods that permit direct investigation of
  the question

11

1. Provide a coherent and explicit chain of
   reasoning
2. Replicate and generalize across studies
3. Disclose research to encourage professional
   scrutiny and debate

12
Side-by Side Comparison

• Howe Eisenhart
• 1. Competent application of methods
• External contributions made explicit
• Fit of research questions, data collection and
  analysis
• Overall warrant established
• Internal and external value constraints met

• NRC
• 1. Link research to relevant
• theory
• 2. Use appropriate methods
• 3. Pose significant questions
• 4. Provide a chain of reasoning
• 5. Replicate and generalize
• 6. Disclose research for scrutiny and debate

13

• Can We Talk about Shared Standards for the
  Trustworthiness (Validity and Reliability) of
  Educational Research?
• I think so, but both qualitative and quantitative
  researchers have to take this effort seriously.

14

• Necessary First Steps (qualitative and
  quantitative)
• Share a definition of trustworthiness
• One possibility When the evidence for the
  results reported is sound and when the argument
  made based on the results is strong
• Make strategies for achieving trustworthiness a
  formal component of all research designs

15

• Qualitative Strategies for Trustworthiness
• Identify threats to understanding (of social
  context, meanings, a group, etc.) in situ.
• Key Q How could the researchers understandings
  be wrong?

16

• Key References
• Maxwell, J. (1996). Qualitative research design
  An interactive approach. Thousand Oaks, CA
  Sage.
• Maxwell, J. (2002). Understanding and validity
  in qualitative research. In A.M. Huberman M.B.
  Miles (Eds.) The qualitative researchers
  companion (pp. 37-64). Thousand Oaks, CA Sage.

17

• 4 Kinds of Understandings that Qualitative
  Research Provides
• Descriptive understanding
• Interpretive understanding
• Theoretical understanding
• Generalizability

18

• Descriptive validity Could the description be
  wrong?
• Descriptive accounts must be factually
  accuratewhere the physical, concrete, or
  behavioral details are agreed to by both
  researcher(s) and participants.

19

• Main threat
• Inaccurate or incomplete evidence
• Main strategies to reduce threat
• Detailed evidence
• Triangulation of data sources

20

• Interpretive validity Could the interpretation
  be wrong?
• Interpretations must be meaningful to study
  participants, or meaningful from the
  participants perspective.

21

• Main threat
• Imposing the researchers own perspective
• Main strategies to reduce threat
• Systematic testing of emergent interpretations
  against participants words and actions
• Member checks

22

• Theoretical validity Could the theory be wrong?
• A theory must provide a good explanation for the
  phenomena studied. Both the concepts and the
  relationships among them must be valid.

23

• Main threat
• Discrepant data, negative cases, or rival
  explanations not taken seriously
• Main strategies to reduce threat
• Systematic attempts to find discrepant data and
  disprove alternative explanations
• Consensus of other researchers

24

• Qualitative generalization
• 2 kinds
• Internal (within group or site, but beyond those
  studied directly)
• External (to other groups or sites)

25

• Main threat to internal generalizability
• Those observed and interviewed not typical of
  whole group
• Main strategy to reduce threat
• Purposeful sampling
• Surveying

26

• Main threats to external generalizability
• Phenomena studied is a special or idiosyncratic
  case
• Main strategies to reduce threat
• Site selection
• Assess generalizability of theory or big idea,
  not findings

27

• A Note on Reliability
• Qualitative research does not attempt to
  eliminate variance between researchers or the
  researchers influence on the setting.
• It does attempt to illuminate how a researchers
  values, expectations, and background influence
  the study.

28

• Threat Researchers influence unclear
• Strategy Audit trails

29
Conclusion

• General standards for qualitative (or
  quantitative) research in education are within
  our reach, but to get there, we must get passed
  epistemological battles and turn our attention
  squarely to standards for trustworthiness
  (validity) that are meaningful (relevant) and
  useful in our field.