Sources of Demand Characteristics

1
Sources of Demand Characteristics

• Natalie P. Goodwin
• UAB Medical Psychology
• Research Methods

2
Artifact and Bias

• Issues that can threaten validity of an
  experiment.
• An artifact is an extraneous influence which can
  include anything that the experimenter is not
  interested in examining.
• Stages of artifacts include ignorance, coping and
  exploiting.
• Bias
• Can stem from those conducting research, demand
  characteristics, subject roles and subject
  selection.

3
What is a Demand Characteristic?

• Can be an artifact AND the results of bias.
• Demand characteristics refer to cues in the
  experimental situation that may influence how
  subjects respond.
• The cues may influence results.
• Cues can come from the experimenter behavior, the
  setting, experimental materials and context.

4
Clever Hans 1900s

• Clever Hans was a horse who belonged to Mr. Von
  Osten, a German mathematician.
• Hans was able to solve math problems submitting
  his answer by tapping his foot.
• Oskar Pfungst, a psychologist, was called upon to
  investigate the story.
• He found that the horse was responding to
  involuntary cues given by his owner.

5
Hawthorne Effect 1924-1932

• Experiments done with Hawthorne Works near
  Chicago to evaluate productivity increase.
• They changed the conditions of the environment to
  see what would increase productivity.
• All changes initially brought on more
  productivity, but eventually there was regression
  to the mean.
• One explanation may have been that they worked
  harder because they were being individually
  monitored and were yielding to demand
  characteristics, doing what they thought the
  experimenter wanted.

6
Bringing in the Placebo

• Demand characteristics and the Placebo Effect can
  go hand in hand.
• The behavior of the doctor administering the
  placebo can impact results.
• The context the placebo is administered in can
  impact results
• This can help lead to the placebo effects that we
  spoke of last week.

7
On the Social Psychology of the Psychological
Experiment With Particular Reference to Demand
Characteristics and their Implications

• Martin T. Orne

8
The issue

• What factors are apt to affect the subjects
  reaction to the well-defined stimuli in the
  situation? What is done to the subject?
• Differences between physical science and
  behavioral science. The subject is an active
  responder to the environment with thought and
  consciousness.
• What is the subjects motivation, perception, how
  will they react to cues?
• Need to have techniques of control to isolate
  environment effects so as to separate them from
  experimental effects.

9
The Subject

• The agreement to be a part of the experiment
• How well will the subject play their role and
  allow themselves to be under the control of the
  investigator.
• They agree often with limited information about
  what they will do.
• The experimenters degree of control is an issue.

10
The extent our subjects may go to

• Orne wanted to develop a study that would show
  the degree of control in an experiment
• Subjects were asked to complete tasks that were
  incredibly noxious, meaningless or boring.
• One task included doing an obscene amount of
  additions, (Im not kidding 224 per page, 2000
  pages).
• To make things even worse, the next task included
  having subjects tear their sheets into 32 pieces
  when they were done.

11
The extent our subjects may go to

• In the experimental setting there is a high
  degree of control in the experimental situation
  itself.
• Subjects continued to perform boring, unrewarding
  tasks, with little errors and decrement in speed.
  
• Remarkable compliance.
• Efforts are justified by the ultimate purpose.

12
On being a good subject

• Useful contribution means that the experimenter
  must be competent and that he acts as a good
  subject
• Did I perform well in my role as experimental
  subject? Did my behavior demonstrate that
  which the experiment is designed to show?
• Play the good subject who validates the
  experimental hypothesis? not a passive responder!

13
Orne on demand characteristics

• Subject participates in problem solving behavior
• Demand characteristics are thus, the cues a
  subject pays attention to that convey an
  experimental hypothesis that helps inform them of
  how to be a good subject
• This can include campus rumors, the experimenter,
  information conveyed, setting and implicit
  communication.

14
Orne on demand characteristics

• Another important factor is that cues can lie
  within a subject meaning that their past
  knowledge and experience may influence them.
• An exampleif you are given a pre-test for
  anxiety before and intervention, then you do a
  post-test, how many of you would be confused
  about how you were expected to respond.

15
The proposed heuristic assumption

• A subjects behavior is determined by
• Experimental variables
• Perceived demand characteristics
• If behavior is due to demand characteristics,
  this will influence replicability and
  generalizability of the influence of experimental
  variables
• Demand characteristics will always exist, but we
  need to be aware of them and when they become
  significant.
• View demand characteristics as a contextual
  variable

16
Ornes Experiments

• Showed that subjects were only influenced by
  demand characteristics if they could verbalize
  the experimenters hypothesis.
• Perceiving the experimenters hypothesis was a
  more accurate predictor of the subjects
  performance.
• It was not just conscious compliance, but
  nonconscious as well.

17
Potency

• Demand characteristics are most potent when they
  convey the purpose of the experiment, but not
  obviously.
• If the purpose is too obvious, there is a
  tendency to lean over backwards to be honest.
• If the purpose is ambiguous, different hypotheses
  are formed and results are not consistent.

18
So then

• We need to take demand characteristics into
  account, study them and manipulate them.
• One of the basic characteristics of the human
  being is that he will ascribe purpose and meaning
  even in the absence of purpose and meaning.

19
How to Deal

• Post experimental inquiry
• Subjects realize they are supposed to act naïve
  about the purpose of the experiment
• A simple procedure that at the end of an
  experiment, simply asks What do you think the
  experiment is about?, What was expected?, How
  were you supposed to perform?
• This will inform you if demand characteristics
  have been involved in performance.
• Issues
• The subjects perception of the hypothesis is
  based on his own experimental behavior and
  correlation between these two variables may have
  little to do with the determinants of behavior.
• The inquiry itself may be subject to demand
  characteristics.

20
How to Deal

• Preinquiry (Riecken, 1958 and Orne, 1959)
• An attempt to control first issue of Post-inquiry
• Subject is exposed to all the procedures, but
  does not actually go through the experiment and
  then does the measures.
• If I had asked you to do all these things, what
  do you think that the experiment would be about,
  what do you think I would be trying to prove,
  what would my hypothesis be?
• Issues Again they are subject to demand
  characteristics.

21
How to Deal

• Simulators
• An attempt to hold the demand characteristics
  constant and eliminate the experimental variable.
• Simulators are a group of subjects that is not
  exposed to the experimental variable, but are
  expected to act as if they were.
• They complete assessment and experimenter is
  blind.
• If simulators can deceive an experimenter, or act
  identically as a real subject, then results can
  be due to demand characteristics.

22
Experimenter and Clinician Effects in Scientific
Inquiry and Clinical Practice

• Robert Rosenthal

23
Rosenthal

• Interested in biasing effects of the experimenter
• Did a symposium with Orne since their topics seem
  to go hand in hand.
• Both focused on social psychology issues.
• Orne focuses on the subjects role
• Rosenthal focuses on the experimenters role.

24
Non interactional Effects

• Observer Effects
• Maskelyne and his assistant, Kinnebrook, who just
  couldnt get his observations of star movements
  right.
• As Bessel would later explain, Kinnebrook was
  probably not making errors, but there are
  individual differences in observation.
• Usually the differences are in relation to
  observers hypothesis.

25
Non Interactional Effects

• Interpreter Effects
• Interpretation of data can be up for debate!
• Accurate interpretation can differ across
  interpreters. May be due to tightly held
  hypotheses.
• Seen as less severe than observer error because
  data is public and can be reviewed by others
  (meta-analyses).

26
Non Interactional Effects

• Intentional Effects
• Fabrication of data in order to get a result.

27
Interactional Effects

• More related to demand characteristics because
  they affect the subjects response.

28
Interactional Effects

• Biosocial Effects
• Sex, age, race can predict results
• Is this due to how different experimenters behave
  toward subjects or how the subjects respond to
  different experimenters.
• Past research shows that male and female
  experimenters conduct the same experiment
  differently.

29
Interactional Effects

• Psychosocial Effects
• Personality can predict results of research
• Anxiety
• Need for approval
• Hostility
• Authoritarianism
• Status
• Warmth
• This may have influence by demand characteristics

What we hope for in an experimenter
30
Interactional Effects

• Situational Effects
• More experience conducting experience
• Previous experience with subjects can lead to
  different results.
• Modeling
• If an experimenter has previous experience with
  the materials, their own performance may be the
  basis on which they predict subject performance.

31
Interactional Effects

• Expectancy Effects
• Most researchers hold an expectation for the
  results.
• These hypotheses can unintentionally alter
  behavior toward their subjects which may lead
  their subjects to respond in accordance to those
  hypotheses.
• Self-fulfilling prophecy AND demand
  characteristics

32
Expectancy Experiments

• Rats mazes
• Experimenters were told rats were either maze
  savvy, or maze dull
• Pygmalion Experiment
• Children at an elementary school were
  administered a nonverbal test of intelligence
  that assessed intellectual blooming potential
• Teachers were told that certain children were
  more likely to show remarkable gains in
  intellectual competence even though this was not
  true.
• Children who were expected to shows gains, DID!

33
Last thoughts

• So we have looked at demand characteristics and
  experimenter characteristics to see how they can
  affect our research
• If all this is true, can we ever expect that our
  experiments really mean anything?
• It may be important to standardize experimenter
  behavior when doing research.
• It would be interesting to look at brain
  activation like they did with placebo effects.
• How might this be different with the physical
  sciences? Obviously we dont have to worry as
  much about demand characteristics, but what about
  experimenter bias and expectations?
• To ameliorate-larger population, larger effect
  size ( study an effect that can overcome demand
  char)