Teaching Ethics in Statistics Class

1
Teaching Ethics in Statistics Class

• John H. Walker
• Department of Statistics
• Cal Poly, San Luis Obispo
• jwalker_at_calpoly.edu

2
Outline

• Ethical Guidelines for Statistical Practice (ASA,
  1999)
• Some thoughts on the JSM session
• Teaching ethics at Cal Poly
• A statistical pet peeve

3
ASA Ethical Guidelines Overview

• Prepared by the Committee on Professional Ethics
• Approved by the Board of Directors
• How many have read them?
• Eight sections
• Professionalism
• Responsibilities to Funders, Clients, and
  Employers
• Responsibilities in Publications and Testimony
• Responsibilities to Research Subjects
• Responsibilities to Research Team Colleagues
• Responsibilities to Other Statistician or
  Practitioners
• Responsibilities Regarding Allegations of
  Misconduct
• Responsibilities of Employers

4
Ethical Guidelines Professionalism

1. Strive for practical relevance in statistical
   analyses
2. Guard against predisposition about results
3. Remain current in statistical methodology
4. Assure adequate statistical and subject-matter
   expertise
5. Use only methodologies suitable to the data
6. Do not join a research project unless you can
   expect valid results and your name is not used
   without consent
7. Understand the theory, data, and methods behind
   automated procedures
8. Recognize the implications of multiple
   frequentist tests
9. Respect and acknowledge the contributions of
   others
10. Disclose conflicts of interest and resolve them

5
JSM Session Teaching Ethics in Statistics Class

• George McCabe, Purdue Univ.
• Ethics and the Introductory Statistics Course
• Patricia Humphrey, Georgia Southern Univ.
• Ethics, Its for Everyone!
• Paul Velleman, Cornell Univ.
• Truth, Damn Truth, and Statistics
• Journal of Statistics Education
• www.amstat.org/publications/jse/v16n2/velleman.htm
  l

6
McCabe

• New Course vs. Old Course
• Old course ethics examples
• Decide the significance level before looking at
  the p-value
• Make sure assumptions are satisfied. (Then
  what?)
• Dont say too much!
• Students are afraid to conclude anything
• New course ethics emphasizes
• Question formulation
• Correct choice of method
• Focusing on the data
• Are we teaching the new course with old
  ethics?

7
Humphrey

• Data ethics
• Much more than have we avoided bias
• Institutional Review Boards
• Confidentiality
• Informed Consent
• Case studies are great ways to teach ethics!
• Continuous reinforcement throughout the class
• Project data collection
• Labs
• Exams

8
Velleman

• Statistics is the honest search for truth about
  the world
• Good statistics requires judgment
• The best analysis often arises from the
  Darwinian competition among alternative models.
• Survival of the best fit
• In the end, there may not be a single best
  model
• Public mistrust of statistics
• The problem isnt that another sample may give
  a different answer, but that another statistician
  working with the same sample may give a different
  answer.

9
Teaching Ethics in the Cal Poly Statistics Major

• First quarter (Concepts and Controversies level
  course)
• Debates on ethical issues (animal testing,
  informed consent)
• Complete NIH online ethics training
• http//researchethics.od.nih.gov
• Late 1st/Early 2nd year (2 course Applied Stat
  sequence)
• Choice of statistical method
• Data collection
• Type I Type II errors, power
• Multiple comparison methods
• Assumptions and alternative methods
• Limits of a statistician
• Importance of subject matter knowledge
• Practical significance

10
Teaching Ethics in the Cal Poly Statistics Major

• Third fourth year (Electives, e.g. Survey
  Sampling)
• Nonresponse in survyes
• Class project
• Institutional Review Board / Human Subjects
  Committee
• Last quarter (Capstone Communication and
  Consulting)
• Team projects
• Mock consulting sessions
• Unaided choice of statistical method
• Dealing with pushy clients
• ASA Ethical Guidelines

11
Teaching Ethics Conclusions

• Emphasize judgment points in data analysis
• Discuss alternative choices and consequences
• Study design
• Observational study vs. designed experiment
• Was the data collected ethically?
• Assumption checking and reexpression
• What is the possible effect of a violation? Of
  reexpression?
• All assumptions are not created equal. Some are
  more important.
• Multiple comparisons
• How many tests did you run? Each p-value you
  look at is a test.
• Outliers and influential observations
• Identify and gather information. Are they real
  or errors?
• How do they affect the results?
• Disclose any changes to your data.

12
Teaching Ethics Conclusions

• Focus on the data an analytical outline
• Look at the data. (Make a graph.)
• Analyze the data.
• Draw conclusions.
• Look at the data again. Reevaluate conclusions.
• Reporting results
• State conclusions with authority (within reason).
• Dont confuse causation and association.
• Report statistical significance (p-value).
• Report practical significance (effect size and
  direction).
• When possible report intervals, not just point
  estimates.
• Report any unresolved problems and possible
  consequences.

13
My (Current) Statistical Pet Peeve

• Remember Ethical Guideline 8
• Recognize the implications of multiple
  frequentist tests
• Problem Uneven application!
• Why do we usually talk about multiple tests only
  when we teach ANOVA?
• Classic example
• One-way ANOVA with 4 factor levels
• 6 pairwise comparisons
• Standard multiple comparison methods control
  overall error rate

14
What about

• Multiple regression
• Multifactor ANOVA
• Multiple response variables
• Multiple regression or multifactor ANOVA with
  several response variables in the study
• Canned multiple comparison methods do not control
  the overall Type I error rate in these
  situations.
• Do we tell our students enough about this problem?

15
Examples

• A multiple regression with 10 predictor variables
• Each predictor tested at a .05
• 10 tests. No control on overall Type I error
  rate
• A three-factor experiment, each factor with 4
  levels
• Each term tested at a .05 with multiple
  comparisons at a .05
• 3 main effects, 3 two-way interactions, 1
  three-way interaction
• Canned MC methods will adjust for comparisons
  within each factor, but not across the different
  terms in the model.
• 7 tests. No control on overall Type I error rate
• The above design with 5 univariate responses
• 35 tests. Yikes!

16
Solutions

• Bonferroni adjustment
• Controls overall Type I error rate, but very
  conservative
• Simple enough to use everydayeven in intro
  classes
• Higher level classes could use more powerful step
  down versions
• What about power?
• Who says you have to have a 5 overall Type I
  error rate?
• Before analysis, just choose a higher overall
  Type I error rate.
• A pseudo-Bonferroni adjustment (working
  backwards)
• Dont like weird fractional individual
  significance levels?
• Use a small, rounded comparison-wise rate.
• Back compute the upper-bound on the overall Type
  I error rate.

17
What To Tell Students

• Be aware of the problem Ignorance is not bliss!
• Discuss the consequences of different approaches
• Exploratory vs. confirmatory analyses
• Balancing power and overall Type I error rate
• What to do may be a judgment call
• If you adjust, understand the power implications
• If not, count the tests, then compute and report
  the Type I error bound
• Stand up to clients who dont want to adjust
• Guiding principle The honest search for truth
  about the world
• Then, make sure we practice what we teach!

18

• Thank you!