Chapter Nineteen

1
Chapter Nineteen

• Nonexperimental Methods

PowerPoint Presentation created by Dr. Susan R.
BurnsMorningside College
2
Descriptive Methods

• Do not involve manipulation of an independent
  variable
• When we use descriptive methods, we can only
  speculate about causation that may be involved.

3
Descriptive Methods

• Archival and Previously Recorded Sources of Data
• Refers to use of data recorded by other
  individuals for other purposes
• (e.g. public health and census data)
• The General Social Survey (GSS) is an archival
  source that can be accessed online
• http//www.icpsr.umich.edu/gss/home.htm

4
Descriptive Methods

• Archival and Previously Recorded Sources of Data
• Potential Problems
• You will not know exactly who left the data you
  are investigating.
• The participants may have been selective in what
  they chose to write.
• This problem is also known as selective deposit.
• Archival and previously recorded sources of data
  may not survive long enough for you to make use
  of them.

5
Descriptive Methods

• Comparisons with the Experimental Method
• We are not able to exercise any control with
  regard to gathering these data and cannot make
  any statements regarding cause-and-effect.

6
Descriptive Methods

• Observational Techniques
• Case studies
• Involves intense observation and recording of
  behavior of a single (perhaps two) participant(s)
  over an extended period of time.
• There are no guidelines for conducting a case
  study and the procedures employed, behaviors
  observed, and reports produced may vary
  substantially.
• Frequently used in clinical settings to help
  formulate ideas and hypotheses for further
  research.
• Although case studies often provide interesting
  data, their results may be applicable only to the
  individual participants who was observed. That
  is, the researcher should not generalize beyond
  the individual participant who was observed.

7
Descriptive Methods

• Observational Techniques
• Naturalistic Observation
• Involves seeking answers to research questions by
  observing behavior in the real world.
• The first goal of naturalistic observation is to
  describe behavior as it occurs in the natural
  setting without the artificiality of the
  laboratory.
• The second goal of naturalistic observation is to
  describe the variables that are present and the
  relations among them.

8
Psychological Detective

• Why should the researcher be concealed or
  unobtrusive in a study using naturalistic
  observation?
• The reactance or reactivity effect
• Refers to biasing of the participants responses
  because they know they are being observed.
• The reactivity effect is also known as the
  Hawthorne effect because of the location of the
  original study.

9
Descriptive Methods

• Participant Observation
• The researcher becomes part of the group being
  studied.
• Often used when the goal of the research project
  is to learn something about a specific culture or
  socioeconomic group.
• Ethnography is a form of participant observation
  based on the anthropological tradition of
  research.
• Observer as participant refers to a researcher
  who primarily observes a situation but who
  interacts with the others (Glesne, 1999).
• Participant as observer refers to the
  researcher who becomes a part of the culture by
  working and interacting extensively with the
  others (Glesne, 1999).

10
Psychological Detective

• What are the drawbacks and weaknesses of the
  participant observer technique?
• An extended period of time may be necessary
  before the participant observer is accepted as a
  member of the group that is under study.
• Cannot make cause-and-effect statements.

11
Descriptive Methods

• Clinical perspective
• Schein (1987) argued convincingly that the
  clinical perspective or model is not a
  subcategory of participant observation because
• A client typically chooses the clinician, whereas
  the participant observer chooses the others to be
  studied.
• Unlike participant observers, clinicians cannot
  be unobtrusive because they have been asked to
  participate in the situation.
• Although the participant observer can remain
  passive, clinicians must intervene in the
  situation.
• The participant observers goal is understanding,
  whereas the clinicians goal is helping.
• Participant observers validate their findings by
  replication while clinicians validate their
  findings by being able to predict the results of
  a given intervention.

12
Descriptive Methods

• Choosing Behaviors and Recording Techniques.
• Time sampling
• Involves making observations at different time
  periods in order to obtain a more representative
  sampling of the behavior of interest.
• Selection of time periods may be determined
  randomly or in a more systematic manner.
• The use of time sampling may apply to the same or
  different participants.

13
Descriptive Methods

• Choosing Behaviors and Recording Techniques.
• Situation sampling
• Involves observing the same behavior in several
  different situations. This techniques offers the
  researcher two advantages
• By sampling behavior in several different
  situations, you are able to determine whether the
  behavior in question changes as a function of the
  context in which you observed it.
• You are likely to observe different participants
  in the different situations and because different
  individuals are observed, your ability to
  generalize any behavioral consistencies across
  the various situations is increased.

14
Descriptive Methods

• Deciding how to present the results of your
  research project.
• Qualitative presentation of results
• Report consists of a description of the behavior
  in question (a narrative record) and the
  conclusions prompted by this description.
• Quantitative or numerical presentation of
  results
• Need to know how behavior under investigation is
  going to be measured and how these measurements
  will be analyzed.

15
Descriptive Methods

• Using More than One Observer
• There are two main reasons for using more than
  one observer
• One observer may miss or overlook a bit of
  behavior.
• There may be some disagreement concerning exactly
  what was seen and how it should be rated or
  categorized.

16
Descriptive Methods

• Using More than One Observer
• When two individuals observe the same behavior,
  it is possible to see how well their observations
  agree.
• The extent to which the observers agree is called
  interobserver reliability.
• Low interobserver reliability indicates that the
  observers disagree about the behavior(s) they
  observed.
• High interobserver reliability indicates
  agreement.

17
Descriptive Methods

• The formula for calculating interobserver
  reliability is as follows
• 85 agreement is generally considered to be an
  acceptable minimum level for interobserver
  reliability.

18
Qualitative Research

• Qualitative research
• is defined as an inquiry process of
  understanding a social or human problem, based on
  building a complex, holistic picture, formed with
  words, reporting detailed views of informants,
  and conducted in a natural setting (Creswell,
  1994, p. 2)
• The qualitative research style is much less
  formal and impersonal, and the reader of a
  qualitative research report can expect to find
  such additions as definitions that evolved
  during a study (Creswell, 1994, p. 7)

19
Qualitative Research

• The qualitative researcher believes that a full
  description of human behavior includes peoples
  feelings in addition to what they are doing and
  how they are doing it.
• The qualitative researcher is committed to
  studying particular people in specific settings.
• Qualitative researchers prefer to use inductive
  logic.
• Qualitative research begins with guiding
  hypotheses reflecting a global issue of interest
  (Marshall Rossman, 1989).
• Qualitative researchers typically analyze their
  data simultaneously with data collection, data
  interpretation, and narrative reporting writing
  (Creswell, 1994, p. 153).

20
Correlational Studies

• A correlational study involves the measurement
  and determination of the relation between two
  variables.
• In terms of control, empirical measurement, and
  statistical an analysis, a correlational study is
  likely to be more rigorous than one of the
  descriptive methods.
• Researchers use correlational studies when data
  on two variables are available, but we can only
  measure, rather than manipulate, either variable.
  
• Although we can determine the degree of relation
  that exists between these two variables, we are
  not able to offer a cause-and-effect statement
  concerning these two variables.

21
Ex Post Facto Studies

• Ex post facto is a Latin phrase meaning after
  the fact.
• When we conduct an ex post facto study, we are
  using an IV after the fact it has already
  varied before we arrived on the scene.

22
Surveys, Questionnaires, Tests, and Inventories

• Surveys typically request our opinion on some
  topic or issue that is of interest to the
  researcher.
• Types of Surveys
• Descriptive seeks to determine what percentage
  of the population has a certain characteristic,
  holds a certain opinion, or engages in a
  particular behavior.
• Examples
• The Gallup Poll
• Nielsen television ratings
• Analytic seeks to determine what the relevant
  variables are and how they might be related.
• Questions need to be chosen carefully.
• Pilot testing is generally necessary to determine
  the type of questions that will be used in the
  final survey instrument.

23
Surveys, Questionnaires, Tests, and Inventories
24
Surveys, Questionnaires, Tests, and Inventories

• Step 1 decide what type of instrument to use.
  How will the information be gathered?
• Will you use a mail survey, conduct a phone
  interview, or conduct the survey in person? Will
  you use trained interviewers?
• Step 2 identify the types of questions to use.
• Yes-No Questions
• The respondent answers yes or no to the items.
• Forced Alternative Questions
• The respondent must select between two
  alternative responses.
• Multiple-Choice Questions
• The respondent must select the most suitable
  response from among several alternatives.
• Likert-Type Scales
• The individual answers a question by selecting a
  response alternative from a designated scale. A
  typical scale might be the following (5)
  strongly agree, (4) agree, (3) undecided, (2)
  disagree, or (1) strongly disagree.
• Open-Ended Questions
• A question is asked to which the respondent must
  construct his or her own answer.

25
Surveys, Questionnaires, Tests, and Inventories

• Step 3 Write the items They should be clear,
  short, and specific.
• Use familiar vocabulary and be at the reading
  level of the individuals you intend to test.
• Step 4 Pilot-test and seek opinions from
  knowledgeable others.
• It is important to ask others, especially
  professionals who have expertise in your area of
  research interest, to review your items. They
  may be able to detect biases and unintended
  wordings that you had not considered.
• Step 5 determine the relevant demographic data
  to be collected.
• Includes items such as age, sex, annual income,
  size of community, academic major, and academic
  classification.
• Step 6 determine administration procedures and
  develop instructions.
• Instructions must be clear, concise, easy to
  follow, and thoroughly rehearsed (for in-person
  or phone interviews).
• How will informed consent be obtained?

26
Surveys, Questionnaires, Tests, and Inventories

• Survey Administration Procedures
• Mail Surveys
• Advantages
• The researcher does not have to be present while
  the survey is being completed.
• Surveys can be sent to a much larger number of
  participants than a single researcher could ever
  hope to contact in person.
• Disadvantages
• The researcher cannot be sure who actually
  completes the survey.
• There is no guarantee the respondent answered the
  questions in the same order in which they
  appeared in the survey.
• Low return rate.

27
Psychological Detective

• Assume that you are planning to conduct a mail
  survey project. You are concerned about the
  possibility of having a low response rate and
  want to do everything to ensure the return of
  your surveys. What can you do to increase your
  response rate?
• The initial mailing should include a letter that
  clearly summarizes the nature and importance of
  the research project, how the respondents were
  selected, and the fact that all responses are
  confidential. You should include a prepaid
  envelope for the return of the completed survey.
• It may be necessary to send an additional
  mailing(s) to respondents. Because the original
  survey may have been misplaced or lost, it is
  important to include a replacement. One extra
  mailing may not be sufficient you may find it
  necessary to send two or three requests before
  you achieve an acceptable response rate. These
  extra mailings are typically sent at two- to
  three-week intervals.

28
Surveys, Questionnaires, Tests, and Inventories

• Personal Interviews
• Advantages
• When a trained interviewer administers a survey
  in a respondents home, the response rate climbs
  dramatically.
• The trained interviewer is able to clarify
  ambiguous questions, making sure that all
  questions are answered in the proper sequence,
  and generally assisting with any problems that
  the respondents may experience.
• Disadvantages
• The potential for considerable expenditure of
  time and money exists.
• People not always at home.

29
Surveys, Questionnaires, Tests, and Inventories

• Telephone Interviews
• Advantages
• It is easy to create random samples with
  random-digit dialing.
• 95 of households in the U.S. have telephones.
• Computer technology allows responses to be
  entered directly as they are made.
• Disadvantages
• Incoming calls may be screened and potential
  respondents lost .
• It is easy to say no to a phone interviewer.
• Cannot use visual aids to clarify questions.
• Cannot evaluate nonverbal cues such as facial
  expressions, gestures, and posture.
• It is more difficult to establish rapport.

30
Surveys, Questionnaires, Tests, and Inventories

• Tests and Inventories
• Tests and inventories are designed to assess a
  specific attribute, ability, or characteristic
  possessed by the individual being tested.
• Characteristics of Good Tests and Inventories
• Validity a test or inventory has validity when
  it actually measures what it is supposed to
  measure.

31
Surveys, Questionnaires, Tests, and Inventories

• Validity (ways to establish)
• Content validity indicates that the test items
  actually represent the type of material they are
  supposed to test. A panel of expert judges is
  often used to assess the content validity of test
  items. Their degree of agreement is known as
  interrater reliability.
• Concurrent validity is established when a score
  on a test or inventory agrees with another
  measure (e.g. aggression score on a test agrees
  with a clinicians assessment).
• Criterion validity is established when a test
  score compares favorably with a predicted future
  outcome (e.g. SAT score and college success).

32
Surveys, Questionnaires, Tests, and Inventories

• Reliability - refers to the extent that the test
  or inventory is consistent in its evaluation of
  the same individuals over repeated
  administrations.
• The greater the similarity between scores
  produced by the same individuals on repeated
  administrations, the greater the reliability of
  the test or inventory.

33
Surveys, Questionnaires, Tests, and Inventories

• Reliability (assessment)
• Test-retest procedure
• The test is simply given a second time and scores
  from the two tests are compared the greater the
  similarity, the higher the reliability
• Split-Half technique
• Involves dividing a test or inventory into two
  halves or subtests and then administering them to
  the same individuals on different occasions or by
  administering the entire test and then splitting
  it into two halves.

34
Psychological Detective

• On the surface, the test-retest procedure appears
  to be quite straightforward and reasonable.
  However, there may be a problem with establishing
  reliability in this manner. What is it?
• The main problem with test-rest concerns the fact
  that the participants are repeatedly administered
  the same test or inventory. Having already taken
  the test or inventory may result in the
  individuals remembering the questions and answers
  the next time the instrument is administered
  (thus allowing potential bias).

35
Surveys, Questionnaires, Tests, and Inventories

• Types of Tests and Inventories
• Achievement tests
• Are given when an evaluation of an individuals
  level of mastery or competence is desired.
• The score that distinguishes passing from failing
  determines the minimum level of achievement that
  must be attained.
• Aptitude tests
• Are used to assess an individuals ability or
  skill in a particular situation or job.
• Personality tests or inventories
• Measure specific aspects of an individuals
  motivational state, interpersonal capability, or
  personality.