QUALITATIVE

1

• QUALITATIVE
• EVALUATION METHODS AND DESIGNS

2
QUALITATIVE EVALUATION IN PERSPECTIVE

• Qualitative evaluation differs from quantitative
  evaluation
• Greater emphasis on holistic descriptions of
  phenomena
• Minute details of activities or environment are
  examined
• Looks at what makes a program "tick"

3
QUALITATIVE EVALUATION IN PERSPECTIVE

• Five characteristics that qualitative methods
  have in common
• Involve a natural setting
• Do not involve reduction of data into numbers
• Concerned with process as well as product
• Involve inductive rather than deductive
  reasoning
• Interest in understanding the people who are
  under study

4
Circumstances under which qualitative evaluation
should be considered

• Individualized outcomes emphasized
• Decision makers want to understand dynamics of
  program
• In-depth information desired about particular
  clients, cases, settings
• Unique aspects of program being assessed
• How clients experience program is of interest
• Formative evaluation is being stressed
• Unobtrusive or less obtrusive data collection
  needed

5
Circumstances under which qualitative evaluation
should be considered

• Case-specific quality assurance is an issue
• Evaluation responsive to stakeholders interests
  required
• Program goals are vague or undetermined
• Quantitative measures fail to measure
• Outcomes deemed important
• Evaluation still exploratory
• Evaluability of program unknown, type of
  summative procedures unclear
• Need to add depth, detail and context to
  statistical data
• New measurement perspectives are desired

6
QUALITATIVE METHODS

• NATURALISTIC OBSERVATIONS
• Non-laboratory settings where people congregate
• Observer effect
• Observer bias

7
QUALITATIVE METHODS

• PARTICIPANT-OBSERVER STUDIES
• Involves becoming a participant as well as
  observer
• Provides investigator with first-hand experience

8
QUALITATIVE METHODS

• FOCUS GROUP INTERVIEWING
• Qualitative approach to learning about population
  subgroups
• Primarily used in market research in the past
• Provides feedback and feedforward
• Moderator recruits participants based upon trait
  or characteristic pertinent to investigation
• Role of moderator critical must be trained

9
QUALITATIVE METHODS

• HISTORICAL ANALYSIS
• An account of events that occurred in the past
• May come from a variety of sources
• Investigators must be cautious about introducing
  bias
• Accuracy of information regarding historical
  accounts may be in doubt
• Ability of investigator to interpret historical
  data is constant limitation

10
QUALITATIVE METHODS

• CONTENT ANALYSIS
• A strategy for studying content of messages
• Done in many diverse areas (art, literature,
  music, popular culture, health area)
• Requires careful planning and sequence of steps

11
QUALITATIVE METHODS

• CASE STUDIES
• Approach for viewing "social reality
• Examines a social unit (person, family, worksite,
  community)
• Investigator may employ in-depth interviewing,
  observation, records, reports, other sources
• Not generalizable

12
QUALITATIVE METHODS

• FILMS, PHOTOGRAPHS, AND VIDEOTAPE EQUIPMENT
• Visual anthropology or film ethnography
• Used to capture daily events, and provide rich
  data when accompanied by descriptive narrative
• Weaknesses include ethics of filming, bias in
  filming (can "manipulate reality"), and
  prohibitive costs

13
QUALITATIVE METHODS

• NOMINAL GROUP PROCESS
• Orderly procedure for obtaining qualitative
  information from target groups
• Group of five to eight well-chosen individuals
• Each panel member makes list of problems to be
  addressed
• Structured discussion
• Panel's collective views are assessed to arrive
  at a consensus decision

14
QUALITATIVE METHODS

• DELPHI TECHNIQUE
• Expert participants are polled by means of mailed
  (or e-mailed) self-administered questionnaires
• Technique proceeds through three or four rounds
• Process declared complete with one of two events
  occur
• Consensus reached
• Point of diminishing returns is reached