Ethics and Validity

1
Chapter 2

• Ethics and Validity

2
Acknowledgements

• http//www.cafeastrology.com
• www.duke.edu
• http//www.markwebtest.netfirms.com/teachRDE/C02/c
  2Lec2.html
• www.twu.edu/inspire/Images
• tutor2u.net
• graphics.stanford.edu
• www.rainbowray.com
• www.valerielorimer.com
• www.keithbond.co.uk
• zeo.sgul.ac.uk
• spunangel.com
• stuff.co.nz
• weirdspy.com
• nytimes.com
• rogerhollander.wordpress.com
• history.ucsb.edu
• thesituationist.wordpress.com

3
Overview

• Questions about Applying Techniques from Physical
  Sciences to Psychology
• Validity Questions
• Internal Validity Questions
• Construct Validity Questions
• External Validity Questions
• Ethical Questions

4
Applying techniques from older sciences to
psychology

• A chemist fills two test tubes with hydrogen and
• oxygen molecules. She leaves the first test tube
  alone.
• She heats the second over the flame. She observes
  
• that water forms only in the second test tube.
• She concludes the following
• flame caused the group of molecules in the
• second test tube to behave differently from the
• molecules in the first test tube
• 2) heat causes hydrogen and oxygen to combine

5
Applying techniques from older sciences to
psychology
A novice investigator fills two rooms with
people. He leaves the group in the first room
alone. He heats up the second room. He
observes that the second group behaves more
aggressively than the first. He then concludes
that feeling hot always makes people more
aggressive.
6
Applying techniques from older sciences to
psychology
Whats wrong with the novice investigators
study?
7
Applying techniques from older sciences to
psychology
Whats wrong with the novice investigators
study?

• did the treatment manipulation really cause one
  group to behave differently from the other?
• did the investigator really manipulate and
  measure the two psychological variables (feeling
  hot and aggressive) that he claimed he did?
• would the results generalize to other settings
  and participants?
• was it ethical to perform the study?

8
Internal Validity Questions

• Did the Treatment Cause a Change in Behavior?

TREATMENT
CHANGE IN BEHAVIOR
CAUSES
9
Internal Validity Questions

• If the study establishes that putting the
  participants into different rooms caused one
  group to behave differently from the other group,
  the study has internal validity
• if something else could be causing the groups to
  differ, the study does not have internal validity

10
Internal Validity Questions

• people assigned to the hot room might have been
  naturally more aggressive than those in the other
• to disentangle treatment effects from natural
  differences between groups, an experiment is
  needed- where participants are randomly assigned
  to treatment groups

11
Internal Validity Questions

• maybe the groups behaved differently because they
  were tested at different times of the day? (cf.
  people being tired after a long day at work vs.
  fresh early in the morning, etc. also crime
  stats- aggression at night)
• time of testing should be controlled- everybody
  tested at the same time, OR

12
Internal Validity Questions

• making sure there are no systematic differences
  between groups in time of testing by
• randomly determining for each group what time
  each group would be tested
• balancing out the time of testing by
  systematically trading off when the groups are
  tested

warm room participants
normal room participants
Monday 900 1100 Tuesday 1000 1200
Monday 1000 1200 Tuesday 900 1100
13
Internal Validity Questions

• maybe the groups behaved differently because of
  external factors (some loud voices outside one
  room, but not the other, during testing, etc.)

14
Internal Validity Questions

• in summary to establish internal validity, the
  investigator would need to show that
• the treatment group behaved differently than the
  non-treatment group
• the difference in behavior was caused by the
  treatment manipulation and NOT anything else
• it is very hard to establish internal validity
  (i.e., to rule out the possibility that the
  groups differ due to some factor other than
  treatment)

15
Construct Validity Questions Are the variable
names accurate?

• psychological constructs characteristics of
  individuals that cant be directly observed, such
  as
• mental states (e.g., love, hunger, feeling hot)
• traits (e.g., agreeableness)
• abilities (e.g., intelligence)
• intentions (e.g., aggression the intention to
  harm another)

16
Construct Validity Questions Are the variable
names accurate?

• manipulating the temperature of molecules in a
  chemists experiment is easier than the mind
  control involved in manipulating how people feel

17
Construct Validity Questions Are the variable
names accurate?
Constructs unobservable, psychological, abstract
variables (love)

• going from objective, observable, physical events
  to inferring invisible, subjective, psychological
  constructs may involve jumping to conclusions

• Operational definitions
• observable, physical, concrete
• measures and manipulations such as
• How much do you love your spouse?
• 2 3 4 5
• not at all very much

18
Construct Validity

• construct validity the degree to which the study
  measures and manipulates the underlying
  psychological elements that the researcher claims
  to be measuring and manipulating

19
Construct Validity

• flaws in the temperatures study
• The manipulation was poor so the construct
  feeling hot was not manipulated adequately.
• The measure was poor, so the construct
  aggression was not measured accurately.

20
Construct Validity

• flaws in the temperatures study
• Participants figured out what the hypothesis was
  about and played along, so the high scores on the
  aggression measure were due to lying/acting
  rather than to feeling aggressive

21
3 Key Construct Validity Questions

• What does the treatment really manipulate?
• What does the measure really measure?
• Is the participants behavior genuine or an act?

22
Construct Validity What does the treatment
really manipulate?

• questioning the temperature manipulations
  construct validity is it right to call the
  raising-the thermostat manipulation a
  feeling-hot manipulation?
• did the manipulation make the warm group really
  feel hot?
• did the manipulation have any effect besides
  making the warm group feel hot?

23
Construct Validity What does the treatment
really manipulate?

• the only possible way to manipulate participants
  mental states is indirectly- by changing the
  physical environment

24
Construct Validity What does the treatment
really manipulate?

• for any indirect manipulation to work,
  participants must mentally react to the
  manipulation in the way the researcher expects
• usually, manipulations are not so pure that their
  only effect is to change the one thing you
  intended to change

25
Construct Validity What does the treatment
really manipulate?

• manipulations often contain extra ingredients
  that produce unwanted side effects
• e.g., turning up the thermostat might make the
  room noisier or decrease the rooms air quality

26
Construct Validity What does the treatment
really manipulate?

• even if pure at the physical level, manipulations
  may not be pure at the psychological level
• e.g., participants frustrated about not being
  able to open the window, etc.
• so is it a warmth manipulation, or a
  frustration/anger manipulation?

27
Construct Validity What does the treatment
really manipulate?

• it is difficult to manipulate variables (compare
  an aspirin manipulation)
• conclusion always question the name that a
  researcher decides to attach to a manipulation

28
Construct Validity What does the measure really
measure?

• psychological constructs (aggression) are
  abstract, invisible, and therefore impossible to
  measure directly- we cant see directly into
  participants minds

29
Construct Validity What does the measure really
measure?

• the best we can do- set up situations in which
  what they are thinking will be reflected in their
  behavior
• BUT participants behaviors may be mislabeled

30
Construct Validity Is participants behavior
genuine or an act?

• participants may mask their feelings, esp. if
  they know they are in a research project

31
Comparing Internal Validity and Construct Validity

• if the study has internal validity, but no
  construct validity, the only legitimate
  conclusion is this
• Turning up the thermostat caused a difference in
  how participants answered the multiple-choice
  questions.

32
Comparing Internal Validity and Construct Validity

• if the study had construct validity, but no
  internal validity
• The group that felt hot was more aggressive

33
External Validity Can the results be generalized?

• Can the results be generalized to other
  participants?
• Can the results be generalized to other settings?

Results from the research study (A Mini-World)
Larger World
GENERALIZE
34
External Validity Can the results be generalized
to other participants?

• to maximize external validity, you should test a
  large, random sample of participants
• in the warm room study-people used to working in
  very warm conditions, or less vs/ more aggressive
  individuals, etc.

35
External Validity Can the results be generalized
to other settings?

• peoples behavior may change depending on the
  situation
• ecological validity- the degree to which a
  studys setting resembles a real-life setting

36
External Validity Can the results be generalized
to other settings?

• even if temperature did increase aggression in
  the lab
• with this particular group of participants
• at this particular time
• we cant assume that the same effect would obtain
  in future studies
• with different people
• in different settings

37
Internal vs. External Validity TheTwinkie Project

• http//www.twinkiesproject.com/solubility.html
• How do researchers establish internal and
  external validity in the project?

38
Internal vs. External Validity TheTwinkie Project

•   1. internal validity is "established" by
  assuming that, before the test,  the control and
  experimental Twinkies are identical (otherwise,
  there would be no need for the control Twinkie),
•     2. external validity is "established" by
  asserting that both Twinkies "appeared within the
  range of what would be described as a 'normal'
  Twinkie."

39
Ethical Questions Should the study be conducted?

• ethics- (Greek ethos- character, disposition)
• refers to the values by which people morally
  (from the Latin moralis, meaning custom or
  manner) evaluate character or behavior

40
Ethical Questions Should the study be conducted?

• in modern science- ethics refers to the values by
  which the conduct of researchers and morality of
  the various stages they use are evaluated
• in deciding if the study is ethical, consult the
  American Psychological Associations Ethical
  Principles of Psychologists and Code of Conduct
  (Principles) (APA, 2002)

41
Ethical Questions Should the study be conducted?

• how would you answer the following three
  questions
• Is it right to withhold information from
  participants if I think that a full disclosure
  will bias their responses?
• Am I justified in misleading participants by
  using a deception if it is necessary to study an
  important issue?
• Is it permissible for me to invade the privacy of
  participants if there is no other way to gather
  essential facts?

42
Ethical Questions Should the study be conducted?

• Five broad principles to organize researchers
  thinking about ethical issues (Sales and Folkman,
  2000)
• Respect for persons and their autonomy (a
  prospective participants right and ability to
  choose whether to take part or to continue in the
  study)
• Researchers must agree to do no harm
  (nonmaleficence - not doing harm) and to try to
  maximize the benefits of the research
  (beneficence, doing of good)

43
Ethical Questions Should the study be conducted?

• Five broad principles to organize researchers
  thinking about ethical issues (Sales and Folkman,
  2000)
• The pursuit and promotion of justice- the burdens
  and benefits of research are distributed equally
  and without favoritism
• Establishment of the relationship of trust with
  the research participants.
• Fostering of fidelity and scientific integrity.

44
Ethical Questions Has the potential harm been
minimized?

• participants have the right to informed consent
  to understand what will happen in the study and
  then agree to participate
• the consent form should
• explain the potential benefits of the research
• explain any risks to the participant
• describe what the researcher will do to prtect
  the participants privacy

45
Ethical Questions Has the potential harm been
minimized?

• the consent form should
• describe any compensation the participant will
  receive
• explain that participation is voluntary
• explain that the participant will receive that
  compensation even if the participant withdraws
  from the study
• make it clear to participnts that they can quit
  the study at any point

46
Example of the wrriten-consent portion of the
informed-consent agreement (Rosnow Rosenthal,
2008, p. 53)

• Instructions to the participant Before you
  participate in this study, please
• print and then sign your name in the space
  provided in Section A. Once the
• study is over and you have been debriefed, you
  will be asked to initial the
• three statements in Section B to indicate your
  agreement.
• Section A
• I, ............................, voluntarily give
  my consent to participate in this
• project. I have been informed about, and feel
  that I understand, the basic
• nature of the project. I understand that I may
  leave at any time and that
• my anonymity will be protected.
• _________________________ _____________
• Signature of Research Participant Date
• SectionB
• Please initial each of the following statements
  once the study has been completed and you have
  been debriefed
• _________ I have been debriefed.
• _________ I was not forced to stay to complete
  the study.
• _________ All my questions have been answered
  satisfactorily.

47
Ethical Questions Has the potential harm been
minimized?

• participants have the right to confidentiality
• use code numbers when recoding participants
  responses
• store data in a locked file cabinet
• password-protect data files stored on a computer
• sign a pledge to keep all information about
  participants confidential

48
Ethical Questions Has the potential harm been
minimized?

• anticipate all possible risks to participants and
  protect them from these risks (planning phase)
• behave in an ethical manner
• ensure that others behave ethically
• debrief participants after the session (explain
  the purpose of the study, answer questions,
  address concerns, undo any harm that the
  participant may have experienced)

49
Ethical Questions Has the potential harm been
minimized?

• the decision to undertake research rests upon a
  considered judgment by the individual
  psychologist about how to best contribute to
  psychological science and human welfare
  (Principles, APA, 1982)

50
Ethical Questions Has the potential harm been
minimized?

• two implications of this statement
• Even if the investigator fulfilled all his
  responsibilites to the participants, the study
  might still be unethical if it was unlikely to
  contribute to psychological science and human
  welfare

51
Ethical Questions Has the potential harm been
minimized?

• two implications of this statement
• Even if the investigator violated certain
  participant rights (e.g., not telling
  participants what the study is trying to find
  out), the study might still be ethical if the
  expected benefits would compensate for those
  violations

52
Ethical Questions Is deception ever justified?

• deception in research
• some argue that deception in any form is morally
  wrong
• others argue that there are special circumstances
  in which it is needed to ensure the integrity of
  important scientific data (Principle 5)

53
Ethical Questions Is deception ever justified?

• two forms of deception
• active deception (deception by commission)
  participants are actively misled (e.g., are given
  false info about the purpose of the research or
  unwittingly interact with confederates, or are
  given a placebo)

54
Ethical Questions Is deception ever justified?

• two forms of deception
• passive deception (deception by omission)
  information is withheld from the participants
  (e.g., they are not given the full details of the
  research or not informed of the meaning of their
  responses on a projective test, etc.)

55
Informed consent

• Can you think of cases when obtaining informed
  consent might be unnecessary?
• consider researching public archives
• risk-free experiments where informed consent
  could be counterproductive

56
Informed consent

• Rind Bordia (1996) tipping behavior study

57
Informed consent

• clinical studies consent about being assinged
  blindly to control (placebo) or treatment (new
  drug) group
• participants with a limited or diminished
  capacity to understand the consent form

58
Ethical Questions Has the potential benefit been
maximized?

• determining the importance of the research
  question- highly subjective
• no one knows what the researcher will discover
  (cf. Pavlovs discovery of classical
  conditioning)

59
Ethical Questions Has the potential benefit been
maximized?

• to estimate the potential value of the study
• the research psychologist should put
• less emphasis on his/her subjective impression
  and
• more on the more objective judgment of how well
  the study would answer the research question

60
Ethical Questions Has the potential benefit been
maximized?

• Is the study likely to provide valid data?
• valid does not necessarily mean three types of
  validity construct, internal, external
• different research goals require different
  validities

61
Different research goals require different
validities

• Internal validity The ability to isolate the
  cause of an effect.
• Internal validity is necessary if your research
  question concerns explaining why a behavior
  occurs or trying to find out how to
  manipuluate/control a certain behavior.
• Internal validity is unnecessary if your research
  question concerns describing what people
  typically do.

62
Different research goals require different
validities

• Internal validity The ability to isolate the
  cause of an effect.
• The early experimental psychologists who tried to
  make psychology as much like experimental physics
  as possible really valued internal validity.
• Untrained people usually think that a study has
  more internal validity than it really has.

63
Different research goals require different
validities

• External validity The ability to generalize the
  results to others.
• External validity is necessary if you are to
  generalize your results to others

64
Different research goals require different
validities

• External validity The ability to generalize the
  results to others.
• External validity is often not necessary,
  especially when looking for causes of behavior
• a chemist may not care if lab results would hold
  in a real life setting
• a psychologist may not care if the results would
  hold in a real-life setting

65
Different research goals require different
validities

• External validity The ability to generalize the
  results to others.
• External validity is most important in
  descriptive research, where you are trying to
  describe what most people typically do.

66
Different research goals require different
validities

• It is easy to question the external validity of a
  study ask the following
• "Who were the participants?" Often times, you
  will find that women, minorities, and non-college
  students are NOT in the study.
• "Were there enough participants?
• "Was the study conducted in a real-life
  environment or an artificial one?"

67
Different research goals require different
validities

• Construct validity Are we dealing with the
  variables we think we are dealing with?
• The primary concern of people who devise
  psychological tests and scales.
• Not important to radical behaviorists because
  they don't measure abstract, invisible constructs
  such as liking, loving, etc.

68
Ethical Questions Has the potential benefit been
maximized?

• after determining that the research question is
  important and the study would provide a valid
  answer to this question, the researcher can
  suggest changes to
• maximize the studys potential for benefiting
  humankind
• minimize its potential for harming participants

69
Ethical Questions Has permission to conduct the
research been obtained?

• consult your study with others first- why?
• its hard to be impartial when weighting the
  benefits of your own research against the costs
  to participants
• consulting with others may give you insights
  about how to protect participants from harm

70
Ethical Questions Has permission to conduct the
research been obtained?

• obtain permission from the schools IRB (Internal
  Review Board)
• if your school has an IRB, it is a violation of
  federal law to do research without first
  submitting that research to the IRB

71
Ethical Questions Has permission to conduct the
research been obtained?

• the IRB might require additional steps to protect
  the participants, such as
• make the informed consent form more specific and
  easier to understand
• exclude individuals whose ability to give
  informed consent could be questioned
• exclude individuals who may be more at risk for
  negative reactions to the treatment

72
Ethical Questions Has permission to conduct the
research been obtained?

• the IRB might require additional steps to protect
  the participants, such as
• eliminate rewards for participants that might
  make them feel obliged to be in the study
• use alternative procedures that would involve
  less distress or deception
• produce a detailed plan for dealing with
  participants who are upset or harmed by the study
• take additional steps to protect participants
  privacy

73
Ethical Guidelines Violated

• The Tuskegee study
• the US Public Health Service (1932-1973)
• monitoring the course of syphillis in 400
  low-income African-American men
• subjects told they had bad blood and received
  no treatment
• local doctors prohibited from prescribing them
  antibiotics
• subejcts threatened to be dropped from the study
  if they sought treatment elswhere

74
The Tuskegee study

• Principle 1 informed consent
• Principle 2 harm to participants
• Principle 3 justice
• Principle 4 trust
• Principle 5 integrity

75
Ethical Questions Has permission to conduct the
research been obtained?

• if the study uses animals instead of human
  participants, consult APA 1996 Ethical Principles
  for the Care and Use of Animals
• get approval of the schools internal animal care
  and use committe (IACUC)
• do the interests of human beings supersede the
  interests of animals?

76
Ethical Questions Has permission to conduct the
research been obtained?

• animal rights activists
• enterprising researchers would be forced to think
  of alternative methods if they were banned from
  using animals
• e.g., computer models of human beings,
  anthropomorphic dummies
• using lower order species (fruit flies in
  genetics research)
• studying animals in their natural habitat, etc.

77
Ethical Questions Has permission to conduct the
research been obtained?

• The three Rs principle defining moderns research
  on animal subjects (Russel Burch 1959)
• Given scientists own interest in the humane
  treatment
• of the animals used in research, it would be
  prudent to
• search for ways to
• reduce the number of animals used in research
• refine the experiments so that there was less
  suffering
• replace animals with other procedures whenever
  possible

78
How scientific principles protect us from common
persuasive techniques

• Science vs. the Testimonial--or talk show guest
• External validity Too small a sample
• Construct validity
• Lying
• Biased perceptions (People often think they've
  grown wiser or more mature when they haven't)
• Internal validity People might have changed on
  their own even without the treatment.

79
How scientific principles protect us from common
persuasive techniques

• Science vs. many informal polls
• External validity
• Small sample
• Biased sample (mail polls)
• Construct validity
• Lying
• Misinterpreting questions
• Internal validity People don't know why they act
  the way they do.

80
How scientific principles can improve on
"ordinary logic"

• External validity questions
• False consensus effect "Is everybody really
  doing it
• Beware of stereotyping based on small,
  nonrepresentative samples

81
How scientific principles can improve on
"ordinary logic"

• Internal validity "False cause" errors
• Are the variables even related? We sometimes see
  illusory correlations.
• Are we confusing causes with effects?
• Are both factors side effects of some other
  cause?
• Beware of "before-after "stories People can
  change on their own

82
How scientific principles can improve on
"ordinary logic"

• Internal validity "False cause" errors
• Beware of stories that claim to assess the
  effects of a treatment even though they are
  comparing "apples and oranges
• e.g., assess illegal drug's effect by comparing
  drug users with non-users

83
How scientific principles can improve on
"ordinary logic"

• Construct validity Beware of labels
• Tests in magazines may not measure what they
  claim to measure.
• Everyday judgments of people based on their
  behavior good, bad, lazy, stupid
• Drugs and therapies may work because of placebo
  effects rather than because of their specific
  ingredients