How to Conduct a Good Research Study

1
How to Conduct a Good Research Study

• STA 309H
• January 31, 2005

2
Lecture Outline

• Group project information
• Research methods
• Introduction to SPSS
• Stop me if you have questions!

3
Group Project Overview

• The group project involves conducting an
  investigation of an interesting and original
  question related to one or more themes of this
  class.
• This may involve designing an experiment, or
  collecting data from various sources.
• You will conduct statistical analyses on you
  data.
• Present your findings in your group presentation.

4
Group Projects

• See handout for your group assignments
• Presentation dates
• Groups 1 to 3 April 27th
• Groups 4 to 6 May 2th
• Groups 7 to 9 May 4th
• The week of April 18th is reserved for group
  meetings with Dr. Koehler
• See Group Project FAQ section on course website
  for other administrative questions

5
Grading

• PRESENTATION RATING FORM Which presentation of
  the day        1st                   
  2nd                  3rd                  
  (please circle) All ratings are to be made on
  a scale from 1 - 5 (1 very low, 5 very
  high) 1.  ORIGINALITY                   a. 
  TOPIC, ANGLE                 _________     
               b.  STYLE                          
       _________        2.  EDUCATIONAL
  VALUE              _________       3.  OVERALL
  QUALITY                   _________     

6
My Role

• Consult with groups on your projects
• Where to meet
• PCL, in the lobby (right inside the front door)
• When to meet
• Email me to make an appointment (the earlier the
  better)
• Best times are Monday 12-4, Tuesday 12-7
• EVERY group member should be present when meeting
  with me

7
Steps for conducting research

• Identify a research question

8
Choosing a good research question

• Interesting and original
• Wow, I cant wait to tell some of my smartest
  friends about this one!
• Not obvious (Will your findings be surprising and
  exciting?)
• A question where you look at conditions under
  which
• Doable (in your time frame)

9
Choosing a good research question

• Past projects include
• Can experts predict the stock market?
• Is racial profiling used by airport security?
• Should fast-food consumers drive-thru or walk-in?
• What is the density and variability of raisins in
  Raisin Bran cereals?
• Does confidence affect test performance?
• Why do some like Leno while others like
  Letterman?
• What is the optimal strategy in rock, papers,
  scissors?
• What is the probability of transmitting HIV in
  the boxing ring?
• Do blondes have more fun?
• Do winning sports teams improve the citys
  economy?
• Do good liars make good leaders?

10
Research questions to AVOID!!

• In general, try to avoid questions about
• Grades and salaries
• Gender differences
• Comparing students across majors and colleges

11
Choosing a good research question

• The best research questions will be tied to one
  or more underlying theories that will direct the
  hypotheses.
• Hypothesis
• A tentative, educated guess of the expected
  outcome of your study
• What do you think is the answer to your research
  question? (You should not KNOW the answer to your
  question, but you may have a hunch or a good
  guess)
• Theory
• a well tested explanation for observed events
• Why do you think your hypothesis is true?

12
Steps for conducting research

• Identify a research question
• Design and conduct a study/investigation to test
  the research question
• Analyze data that is already available
• Collect your own data

13
Analyzing data that is already available

• Analytical projects (integration of
  statistical/probability information from
  different sources)
• Transmitting HIV in the boxing ring
• Analyzing data from an existing data source (a
  database)
• Stock market data
• Sports statistics
• Crime data

14
Collecting your own data

• Surveys
• Collect data from the population without
  manipulating or controlling anything,
• E.g. Political attitudes survey
• Experiments
• Collect data from the population by manipulating
  something (an independent variable)
• E.g. drug studies

15
Surveys

• A lot in common with archival/database studies
• Difference you collect the data with a survey,
  while the data is already there with a database
• Similarity both yield observational data (you
  observe the world without intervening). You
  cannot determine any causal relationships.

16
Surveys

• Common things to look out for
• Random sampling?
• Non-response error?
• Social desirability/self-reports?
• Wording of questions?
• The importance of pilot testing

17
Surveys

• Does association causation?
• If there is a relationship between A and B, does
  that mean that A causes B?
• Coffee may make older lovers perky In a
  survey of people 60 and over
• 62 of coffee drinkers were sexually active,
  while only 37 of non-coffee drinkers were
  sexually active
• Is there a causal relationship between coffee and
  sexual activity?

18
Surveys

• Nevertheless there is a lot of survey and
  database (archival) research out there
• For some research questions, researchers are not
  concerned about causality
• Ethical issues may prevent the collection of
  experimental data (e.g. physical or sexual abuse)
• People have tried controlling for outside factors
  (matched samples design)

19

Experiments

• In an experiment, the researcher attempts to
  examine causality by systematically varying or
  altering one variable or set of variables and
  examining the resultant changes in or
  consequences for another set of variables.

20
Experiments

• Researcher manipulates a feature of a task called
  the INDEPENDENT VARIABLE and determines its
  effect on the DEPENDENT VARIABLE
• Independent variable manipulated by the
  experimenter (this is the factor that we think is
  causing something else)
• Dependent variable measured from the subjects
  (we hypothesize that the independent variable
  affects this variable- we measure to see if there
  is an effect)

21
Experiments

• At least 2 groups of subjects are treated exactly
  alike in all ways except one (the independent
  variable). Any differences in the observed
  behavior of the two groups are attributed to
  (caused by) the differences in the independent
  variable.
• Experimental group undergoes the manipulation
  hypothesized to have an influence on the
  dependent variable (e.g. the drug)
• Control group receives a different or no
  manipulation this group is used to make certain
  that the result is due to the hypothesized cause
  (e.g. no drug)

22
Experiments

• Random assignment to the groups
• VERY IMPORTANT!!!!
• Why?
• What do you mean by treated alike in all ways
  except one?

23
Experiments

• Drug experiment experimental group gets the
  drug, control group gets nothing is this good
  enough?
• As this study is currently designed, the
  experimental group gets a tablet AND the medicine
  in the tablet (two things are being varied at the
  same time)
• If there is an effect of the drug on health or
  pain (or whatever the DV is), we have a confound
  was the effect due to the tablet or the
  medicine in the tablet?
• Solution give the control group a tablet with no
  medicine (a placebo). Now subjects in the
  study are blind to their condition
• What is a double blind study?

24
Experiments

• Be careful of potential confounds when designing
  your experiment (esp. in your control group)
• Ask yourself Is everything the same EXCEPT for
  the variable Im manipulating?

25
Experiments

• Strength
• You can attribute differences in the dependent
  variable to the independent variable (causal
  relationship)
• Weakness
• Sometimes too far removed from the real world

26
Who should be in our study?

• Best case scenario random sample from the
  population of interest
• Next best scenario sample that does not differ
  from the population of interest on the dimension
  being studied
• Dont look to us to obtain subjects for you, and
  dont distribute any questionnaires in class

27
How many subjects do we need?

• As many as it takes to detect a difference if
  there really is one
• Depends on the variability in your data and the
  effect size that you think is meaningful
• See FAQ page for some more guidance

28
Steps for conducting research

• Identify a research question
• Conduct a study/investigation to test the
  research question
• Draw conclusions
• You can never PROVE a theory you can only find
  evidence to support a theory or evidence
  consistent with a theory

29

• In class exercise