CLINICAL RESEARCH SKILLS

1
RESEARCH DESIGN

• STUDY DESIGN will depend on the specific goal of
  the study

2
RESEARCH DESIGN

• SINGLE-CASE DESIGNS
• 1 or more individuals or a group over a time
  period
• Pre / post-tests
• GROUP DESIGNS
• Several participants studied and assigned to
  conditionsbetween-group design
• Random assignment
• Can be true, quasi or observational

3
RESEARCH DESIGN

• A well-designed experiment is one in which
  competing hypotheses that might explain results
  are made relatively implausible or ruled out
  (VALIDITY).
• TYPES OF DESIGNS
• TRUE EXPERIMENTS maximum control over
  independent variable random assignment
• QUASI EXPERIMENTS all features cannot be
  controlled non random assignment
• OBSERVATIONAL STUDIES e.g. case- control study

4
EXPERIMENTAL METHOD

• 4 GOALS OF RESEARCH
• DESCRIPTION OBSERVATIONAL SURVEY RESEARCH
• PREDICTION CORRELATIONAL RESEARCH
• EXPLANATION EXPERIMENTAL METHOD

5
EXPERIMENTAL METHOD

• WHY PSYCHOLOGISTS CONDUCT EXPERIMENTS?
• RESEARCHERS CONDUCT EXPERIMENTS TO TEST
  HYPOTHESES ABOUT THE CAUSES OF BEHAVIOUR
• EXPERIMENTS ALLOW RESEARCHERS TO DECIDE WHETHER A
  TREATMENT OR PROGRAM EFFECTIVELY CHANGES BEHAVIOUR

6
EXPERIMENTAL METHOD

• LOGIC OF EXPERIMENTAL RESEARCH
• RESEARCHERS MANIPULATE AN IV IN AN EXPERIMENT TO
  OBSERVE THE EFFECT ON BEHAVIOUR, AS ASSESSED BY
  THE DV
• CONTROL IS THE ESSENTIAL INGREDIENT OF
  EXPERIMENTS WHICH IS GAINED THROUGH MANIPULATION,
  HOLDING CONDITIONS CONSTANT, AND BALANCING

7
EXPERIMENTAL METHOD

• EXPERIMENTAL CONTROL ALLOWS RESEARCHERS TO MAKE
  THE CAUSAL INFERENCE THAT THE IV CAUSED THE
  OBSERVED CHANGES IN THE DV
• AN EXPERIMENT HAS INTERNAL VALIDITY WHEN IT
  FULFILLS 3 CONDITIONS
• Covariation (when one changes,the other also
  changes)
• Time-order relationship (one comes before the
  other)
• Elimination of plausible alternative causes (no
  confounds)

8
EXPERIMENTAL METHOD

• WHEN CONFOUNDING OCCURS, A PLAUSIBLE ALTERNATIVE
  EXPLANATION FOR THE OBSERVED COVARIATION EXISTS,
  AND THEREFORE, THE EXPERIMENT LACKS INTERNAL
  VALIDITY
• PLAUSIBLE ALTERNATIVE EXPLANATIONS ARE RULED OUT
  BY HOLDING CONDITIONS CONSTANT AND BALANCING

9
EXPERIMENTAL METHOD

• DESCRIBE A RESEARCH SITUATION IN WHICH THESE
  CONDITIONS ARE NOT MET
• THREATS TO INTERNAL VALIDITY
• HISTORY (due to some other event other than the
  treatment)
• MATURATION (due to the fact that the participants
  changed over time)

10
EXPERIMENTAL METHOD

• HOW ABOUT A RESEARCH SITUATION WHICH MEETS THESE
  CRITERIA?
• LOFTUS BURNS (1982) EXPERIMENT INVESTIGATING
  EFFECT OF WITNESSING A MENTALLY SHOCKING EVENT ON
  THE PARTICIPANTS MEMORY FOR INFORMATION THAT WAS
  PRESENTED IN THE FILM SECONDS BEFORE THE VIOLENT
  SCENE (PG. 226-7, SHAUGNESSY)

11
EXPERIMENTAL METHOD

• CONTROL TECHNIQUES
• Students viewing exact same film except for
  critical incident, given same instructions,
  received same questionnaire at the end (holding
  conditions constant)
• However researchers only hold conditions constant
  that we CHOOSE toSO ONE MUST ALWAYS BE ALERT TO
  POSSIBILITY THAT THERE MAY BE OTHER CONFOUNDS WE
  DID NOT ANTICIPATE
• Make sure that 1 grp is not smarter, more
  attentive, more motivated, have more women
  (balancing) which occurs through RANDOM
  ASSIGNMENT (random groups design)
• STRETCHING EXERCISE PG. 229, SHAUGNESSY

12
INDEPENDENT GROUPS DESIGN

• EACH GRP OF PARTICIPANTS REPRESENTS A DIFFERENT
  CONDITION AS DEFINED BY THE IV (E.G. EYEWITNESS
  BEHAVIOUR EXPERIMENT)
• Random groups design is needed to BALANCE
  individual differences

13
INDEPENDENT GROUPS DESIGN

• RANDOM GROUPS DESIGN
• Comparable groups of individuals are formed and
  the groups are treated the same in all respects
  except that each grp receives only ONE level of
  the INDEPENDENT VARIABLE
• This allows for causal inferences
• Random assignment is used to form comparable
  groups by balancing or averaging subject
  characteristics (individual differences) across
  the conditions of the IV manipulation

14
INDEPENDENT GROUPS DESIGN

• RANDOM GROUP DESIGN contd
• Block randomization balances subject
  characteristics and potential confoundings that
  occur during the time in which the experiment is
  conducted, and creates groups of equal sizes
• ONE PARTICIPANT IS ASSIGNED TO EACH CONDITION IN
  THE BLOCK BEFORE A 2ND ONE IS ASSIGNED IN ANY ONE
  CONDITION

15
INDEPENDENT GROUP DESIGN

• BLOCK RANDOMIZATION
• If we want to have 20 participants in each of 3
  conditions, then there would be 20 blocks in the
  block-randomized schedule
• Essentially, we take the first 3 people and
  randomly assign them to each condition, then the
  next 3 until youve done this 20 times
• Groups are tested in blocks, not based on the IV
  thereby creating a balance

16
INDEPENDENT GROUPS DESIGN

• BLOCK RANDOMIZATION
• ADVANTAGES
• PRODUCES GROUPS OF EQUAL SIZE
• BECAUSE EXPERIMENTS CAN TAKE A LONG TIME TO
  COMPLETE, PARTICIPANTS CAN BE AFFECTED BY EVENTS
  THAT OCCUR DURING THIS TIME, BLOCK RANDOMIZATION
  HELPS TO CONTROL FOR THESE TIME-RELATED EVENTS
  BECAUSE PARTICIPANTS ARE ASSIGNED TO CONDITIONS
  ONE BLOCK AT A TIME

17
INDEPENDENT GROUPS DESIGN

• BALANCES CHANGES IN EXPERIMENTERS
• BALANCES CHANGES IN THE POPULATIONS FROM WHICH
  PARTICIPANTS ARE DRAWN
• DISADVANTAGE
• CAN GET COMPLICATED WHEN DEALING WITH MULTIPLE
  EXTRANEOUS VARIABLES
• STRETCHING EXERCISE PG. 229, SHAUGNESSY

18
INDEPENDENT GROUPS DESIGN

• THREATS TO INTERNAL VALIDITY
• TESTING INTACT GROUPS (formed prior to start of
  experiment)
• SOLUTION DONT DO IT IN A RANDOM GROUPS DESIGN
• EXTRANEOUS VARIABLES (potential variables are not
  directly of interest to the researcher but that
  could still be sources of confounding in the
  experiment)
• SOLUTION BLOCK RANDOMIZATION

19
INDEPENDENT GROUPS DESIGN

• SUBJECT LOSS
• MECHANICAL when a participant fails to complete
  the experiment because of equipment failure (low
  threat to internal validity)
• SELECTIVE destroys comparability of groups
  (e.g. pg. 235, Shaugnessy)
• SOLUTION NONE EXPERIMENT IS UNINTERPRETABLE
• PREVENTIVE STEPS 1. pretest to screen for
  possible losses 2. pretest then drop a comparable
  person from control grp

20
INDEPENDENT GROUPS DESIGN

• PARTICIPANT EXPERIMENTER EXPECTATIONS
• Demand characteristics
• SOLUTION PLACEBO CONTROL
• Experimenter effects
• SOLUTION DOUBLE BLIND PROCEDURES

21
DATA ANALYSIS

• DATA ANALYSIS involves 3 stages
• 1. getting to know the data
• What is going on in the data set
• Look for errors
• Make sure the data make sense
• 2. summarizing the data
• Use descriptive statistics and graphs
• 3. confirming what the data reveal
• Seek evidence for what the data tell us what can
  we conclude using various stat techniques

22
DATA ANALYSIS

• DESCRIBING THE RESULTS
• USING MEAN, STANDARD DEVIATIONS
• EFFECT SIZE (strength of relationship between IV
  and DV)
• META-ANALYSIS (uses measures of effect size to
  summarize the results of many experiments
  investigating the same IV or DV)

23
DATA ANALYSIS

• CONFIRMING WHAT THE RESULTS REVEAL
• Researchers use inferential stats to determine
  whether an IV has a reliable effect on a DV
• 2 methods used to make inferences NHST and CI

24
DATA ANALYSIS

• NULL HYPOTHESIS SIGNIFICANCE TESTING (NHST)
• Used to determine whether mean differences among
  groups in an experiment are greater than the
  differences that are expected simply because of
  error variation
• Assumes first that the groups do not differ
• Because decisions about outcomes are based on
  probabilities, Type I (rejecting true H0) or Type
  II (failing to reject a false H0) errors may
  occur

25
DATA ANALYSIS

• NHST contd
• When results are statistically significant, we
  know that the smaller the exact probability fo
  the observed outcome, the greater is the
  probability that an exact replication will
  produce a statistically significant finding
• Doesnt mean that results will be replicated 95
  of the time (with alpha.05)
• Always report exact probability of results when
  carrying out NHST

26
DATA ANALYSIS

• NHST contd
• Choose level of significance BEFORE beginning
  your experiment
• Type I alpha level
• Can be reduced by lowering alpha level but this
  increases probability of a Type II error
• Two possible results
• 1. reject the null
• 2. fail to reject the null (WE RARELY ACCEPT THE
  HO)

27
DATA ANALYSIS

• NHST contd
• Because of Type I II errors, researchers rarely
  use the word prove and instead use consistent
  with hypothesis or confirms/supports the
  hypothesis
• STATISTICAL INFERENCE CAN NEVER REPLACE
  REPLICATION AS THE BEST TEST OF THE RELIABILITY
  OF AN EXPERIMENTAL OUTCOME

28
DATA ANALYSIS

• CONFIDENCE INTERVALS
• Researchers determine whether an IV has had an
  effect on behaviour by examining whether the CIs
  for different samples in an experiment overlap
• When CIs do not overlap, we can be confident that
  the population means for the 2 groups differ

29
DATA ANALYSIS

• ESTABLISHING EXTERNAL VALIDITY
• Generalizability (e.g. use of college students)
• In theory-testing other types of experiments,
  researchers may choose to emphasize internal
  validity over external validity
• Other researchers may choose to increase external
  validity using sampling or replication

30
DATA ANALYSIS

• ESTABLISHING EXTERNAL VALIDITY
• Conducting field experiments is one way that
  researchers can increase the external validity of
  their research in real-world settings
• Conceptual replication increases external
  validity
• REPLICATING SAME VARIABLES OF INTEREST IN
  DIFFERENT SAMPLES e.g. insults lead to aggression
  in 5 yr olds and then, in 35 years olds (change
  nature of insult used in experiment)

31
DATA ANALYSIS

• ESTABLISHING EXTERNAL VALIDITY
• Partial replication can help establish external
  validity
• E.g. doing a study in a public school and then in
  a private school

32
MATCHED GROUPS DESIGN

• Used to create comparable groups when there are
  too few participants available for random
  assignment or to work effectively
• Or when different levels of IV are needed and
  repeated measures design will not be effective
• Random groups design individual differences
  average out across groups

33
MATCHED GROUPS DESIGN

• Researcher makes groups equivalent by matching
  participants
• Pretest task is usually used to match
  participants
• Best if task is one that uses the same task that
  will used in the experiment itself
• Less preferred is a similar task when the primary
  DV cannot be used (e.g. if it would familiarize
  participants to task)

34
NATURAL GROUPS DESIGN

• INDIVIDUAL difference variables (participant
  variables) are selected rather than manipulated
  to form natural groups designs
• E.g. religion, gender, age, personality traits,
  divorce, victims of abuse
• Have to be careful of making causal statements
  (has the time-order condition or elimination of
  plausible alternatives been met?)

35
NATURAL GROUPS DESIGNS

• To draw causal inferences in a natural groups
  design
• Individual differences must be studied in
  combination with IVs that can be manipulated
  (complex design)

36
REPEATED MEASURES DESIGN

• Also called WITHIN-SUBJECTS
• Independent grp has CONTROL, cannot totally
  eliminate indiv diff, only balance
• Repeated measures participants serve as their
  own control, no problems with individual diff but
  with PRACTICE EFFECTS, but can also be balanced

37
REPEATED MEASURES DESIGN

• Why use this design?
• When few participants are available
• Conduct experiment more efficiently /
  conveniently
• Increase sensitivity of experiment
• Ability to detect effect of IV on DV even if its
  a small one (which is affected by individual
  diff)
• Study changes in participants behaviour over
  time
• E.g. learning experiment or longitudinal study

38
REPEATED MEASURES DESIGN

• CREATING A WITHIN-SUBJECTS DESIGN, HELPS LIMIT
  THE CONFOUND OF THE PARTICIPANT VARIABLE BY USING
  EITHER
• 1. MATCHED-GROUPS DESIGN (PAIRING PARTICIPANTS
  WHO MATCH ON THE PARTICIPANT VARIABLE
• 2. REPEATED MEASURES DESIGN (TESTING
• SAME PARTICIPANTS UNDER ALL LEVELS OF A
• FACTOR)

39
REPEATED MEASURES DESIGN

• MATCHED-GROUPS
• WORK BEST WHEN THERE IS ONLY 1 MAJOR PARTICIPANT
  VARIABLE TO CONTROL
• REPEATED MEASURES
• WHEN PARTICIPANT VARIABLES ARE A CONCERN, USUALLY
  MANY VARIABLES ARE AN ISSUE SO REPEATED MEASURES
  IS BEST

40
REPEATED MEASURES DESIGN

• E.G. we are testing peoples dress code and their
  comfort level in social settings.
• Great differences exist in reasons why ppl would
  feel uncomfortable (shyness, past social
  experiences etc.), so we decided to use a
  repeated measures design
• How could we set this up?

41
REPEATED MEASURES DESIGN

• Problem of PRACTICE EFFECTS
• Change in participants performance across
  conditions because of repeated testing (and not
  because of IV)
• Threatens internal validity when different
  conditions of IV are presented in the same order
  to all participants

42
REPEATED MEASURES DESIGN

• Order effects
• Influence on scores that arises from completing a
  particular order
• Participants may change over the course of the
  experiment due to practice, fatigue, carry-over
  effects or response sets (e.g. they may feel more
  comfortable on the 2nd test condition because of
  practice)
• Individuals change over time due to subject
  history, maturation (which affects internal
  validity)
• SO TO COUNTERACT, WE BALANCE THEIR INFLUENCE

43
REPEATED MEASURES DESIGN

• 2 types of repeated measures designs
• COMPLETE
• PRACTICE EFFECTS FOR EACH PARTICIPANT ARE
  BALANCED FOR EACH PARTICIPANT by administering
  the conditions to each participant several times
  using different orders each time
• INCOMPLETE
• ORDER OF ADMINISTERING THE CONDITIONS IS VARIED
  ACROSS PARTICIPANTS RATHER THAN FOR EACH
  PARTICIPANT

44
REPEATED MEASURES DESIGN

• Methods for balancing PRACTICE effects in THE
  COMPLETE DESIGN
• 1. Block Randomization
• Each condition must be repeated several times
  before we can expect practice effects to be
  balanced / average out when this is not
  possible
• 2. ABBA Counterbalancing
• Presenting conditions in one sequence, followed
  by opposite of that sequence can use ABCCBA

45
REPEATED MEASURES DESIGN

• Methods for balancing PRACTICE effects in THE
  INCOMPLETE DESIGN
• Each participant is given treatment only once and
  practice effects are balanced ACROSS
  participants, rather than for each participant
• Each condition must be presented in each ordinal
  position (1st, 2nd) equally often
• Best method to balance with 4 or less conditions
  is to use ALL POSSIBLE ORDERS of the conditions