Methodology

1
Methodology

• Tips for Constructing Instruments

2
Matching methods to research paradigm
3
Quantitative/Descriptive Studies involve specific
Research Designs

• Designs structure how the interviews are
  conducted.
• Allow different groups and time periods to be
  compared.
• Minimize threats to internal and external
  validity.

4
Components of Designs

• Sampling (Random Other probability sampling or
  nonprobability such as convenience, purposive,
  or snowball)
• Groups one group two, or multiple group
  comparisons.
• Time intervals pre and post tests or multiple
  observations.

5
Common design types

• Experimental Design (pre-post random sample, two
  or more groups compared.
• Quasi Experimental with nonequivalent comparisons
  groups- pre and post.
• Quasi Experiments with post-test only.
• One group - pre and post test only.
• One group multiple time periods measured (time
  series or longitudinal studies)
• One group one measure usually called
  cross-sectional survey.

6
Random designs help ensure internal/external
validity

• Internal validity attribute of people included
  or way the research was conducted may limit our
  ability to establish a real connection between
  the independent and dependent variables.
• External validity can we generalize results to
  other people, settings, and times.
• (Note for some types of studies qualitative
  research and needs assessments and some types of
  evaluations we do not need generalizability. We
  are only interested in finding things out about a
  particular situation).

7
Exceptions to Experimental/Quasi-experimental
Designs One group studies

• Case studies (combination of methods looking at
  one individual, group, family, organization, or
  community)
• Survey research
• Consumer satisfaction surveys
• Needs Assessments

8
Use these alternative methods

• For descriptive studies
• When you do not have an intervention
• When you want to look at a specific situation or
  group and you do not intend to generalize.
• When it would be difficult or unethical to create
  an experimental or control group.
• When you do not have time or money to conduct an
  experiment.

9
Threats to Internal Validity Include

• History
• Maturation
• Testing
• Instrumentation
• Regression to the mean
• Selection
• Interaction between Selection and the other
  threats to Internal Validity.

10
Qualitative Research

• Usually one group
• Comparisons may be made after data analysis
  decision to compare made based on obvious
  patterns in the data.
• Sampling nonprobability purposive,
  convenience, snowball
• Uses observation, interviews, and content
  analysis.

11
Focus groups may also be used to conduct
qualitative research. Focus groups

• Are used to assess peoples perceptions about
  programs or services.
• Are used to generate recommendations for changing
  programs and services.
• Are often used in conjunction with needs
  assessments or to generate ideas or explore
  themes that can be used in descriptive studies.
• Are used to answer questions about how or why
  programs work.
• May be used in conjunction with other research
  methods.
• Include six to eight participants who are
  selected using purposive sampling methods.
• Utilize a semi-structured interview guide with
  6-8 questions.
• Require that a facilitator ask each of the
  questions and solicit responses from members.
• Are used to generate a common response or a
  consensus from group members.

12
In any study

• A mixture of methods may be used.
• Researchers may seek to increase reliability and
  validity by using both quantitative and
  qualitative approaches. This is called
  triangulation!
• Methods used should match the research question
• What happens/what is the outcome? quantitative)
• Why does something happen? How does it happen?
  (qualitative)
• A description of what exists (Descriptive)

13
Descriptive studies

• Generally involve surveys, standardized
  interviews, or the use of pre-existing (secondary
  data).
• Include needs assessments.
• Can include both qualitative and quantitative
  data collection. In some cases, one instrument
  can include a mix of qualitative and quantitative
  data collection methods.

14
Survey and Interview Construction Tips

• Only measure one concept per question.
• Dont write the question to solicit a specific
  response
• Response categories (if used) should be mutually
  exclusive and exhaustive
• Questions should be tactful and should not invoke
  a social stigma.
• May be filter questions respondents can be
  directed to successive questions based on one
  response.
• Demographic questions are generally put at the
  beginning or the end of the instrument.
• Be aware that respondents do not like to answer
  some types of questions (age, income,
  organization budgets, etc.)

15
Example of closed-ended (response category)
questions

• 1. What is your gender
• 1. Male 2. Female
• 2. What is your ethnic identification?
• 1. African American
• 2. American Indian
• 3. Asian/Pacific Islander
• 4. European American
• 5. Mexican American/Latino
• 6. Multicultural or multi-racial
• 7. Other ____________________

16
More examples

• 3. What is your age?
• 1. 18 or under
• 2. 19-29
• 3. 30-49
• 4. 50 64
• 5. Over 65
• 4. How satisfied are you with Social Work 292?
• 1. Very satisfied
• 2. Satisfied
• 3. Neutral
• 4. Unsatisfied
• 5. Very Unsatisfied

17
Sample question How would you assess your own
practice ability in the following areas
18
SAMPLE QUALITATIVE INTERVIEW QUESTIONS Interview
Protocol Perspectives on Deafness
19
Each type of question corresponds to a level of
measurement. These levels of measurement will
determine what type of data analysis we use

• All qualitative research is nominal.
• All categorical data in which there is no
  difference in value among the categories is
  nominal.
• All categorical data in which there is an implied
  ranking (for example, high-medium-low) is
  ordinal.
• Any question that asks respondents to rank order
  responses is also ordinal.
• Any question in which the response is a number or
  can be interpreted as a number with equal values
  among the data points is ratio.

20
For example, ratio data can include

• What is your age? _______
• Five point likert scales (1Very likely, 2
  likely, etc. Usually likert scales are
  illustrated on the survey instrument as 1 2 3
  4 5. Respondents are asked to circle the best
  response.
• Scores on standardized instruments are also
  interpreted as ratio data.