Counting Quantitative Reasoning as a Teaching of Psychology Priority

1
Counting Quantitative Reasoning as a Teaching of
Psychology Priority

• Neil Lutsky
• Carleton College, Northfield, Minnesota USA,
  nlutsky_at_carleton.edu

Third International Conference on the Teaching of
Psychology 13 July 2008
2
One of the things that makes psychology unique
is that it makes the whole scientific enterprise
so clear.
Nicky Hayes, University of Bradford ICOPE 2002
3
The fundamental goal of education in psychology,
from which all the others follow, is to teach
students to think as scientists about behavior.
Charles Brewer et al. (1993), Curriculum,
Handbook for Enhancing Undergraduate Education
in Psychology
4
The longer I have been teaching psychology, the
less important I have come to believe the
teaching of psychology to be!
5
Presentation/Argument Overview

• The concepts and findings of psychology are
  incredibly useful in making sense of behavior in
  the real world.
• Nonetheless, there is something else we could be
  emphasizing in the teaching of psychology to help
  students confront the world knowledgeably.
• That something else reflects a central element of
  life and citizenship in the contemporary world
  numbers.
• One of the great gifts we can give students is
  the ability to use quantitative reasoning.

6
Presentation Overview (continued)

• What QR habits of mind do students need to gain
  or strengthen?
• How can we reinforce QR when teaching psychology?

7
The Shelf Life Problem
Manning, Levine, Collins (2007). The Kitty
Genovese murder and the social psychology of
helping The parable of the 38 witnesses.
American Psychologist, 62, 555-562.
8
It is a proud thing to say I taught him and a
wise one not to specify what. -Jacques Barzun
9
Derek Bok (2005), Our Underachieving Colleges
...certain basic quantitative methods seem
applicable to a wide enough range of situations
to be valuable for almost all students.
10
Learn statistics. Go abroad.-K. Anthony
Appiah, Princeton University
11

• US National Numeracy Network, http//serc.carleto
  n.edu/nnn/
• Great Britain More or Less, http//news.bbc.co.uk
  /2/hi/programmes/more_or_less/default.stm
• International Statistical Literacy Project,
  http//www.stat.auckland.ac.nz/iase/islp/home

12
As psychologists we believe in the power of
numbers, the power of numbers to influence
the power of numbers to inform.
13
The Power to Inform
Statistics can tell us things about the world
that we could not imagine on the basis of our
senses alone...The social world is unimaginably
more complex than we can see directly from any
subjective vantage point within it. -Paul
Seabright, Times Literary Supplement
14

• Numbers in Newspapers of Record

15

• Numbers in Newspapers of Record.

16
It sort of makes you stop and think, doesnt
it?
17
What quantitative concepts would a reader need to
know in order to make sense of this important
article?

• Know to Read to the end of the article!
• Recognize the strengths of a Random clinical
  trial vs. Case Method.
• Understand Statistical Significance.
• Appreciate the difference between a Single study
  vs. a Literature.

18
What do I mean by QR ?
Quantitative reasoning is the power and habit of
mind to search out quantitative information,
critique it, reflect upon it, and apply it in
ones public, personal, and professional
lives. -National Numeracy Network
19
10 QR Questions at the Ready

• What do the numbers show?
• How representative is that?
• Compared to what?
• Is the outcome statistically significant?
• Whats the effect size?
• Are the results those of a single study or of a
  literature?
• Whats the research design (correlational or
  experimental)?
• How was the variable operationalized?
• Whos in the measurement sample?
• Controlling for what other variables?

20
How representative is that?
21

• Examples and stories.
• For instance is no proof. Old Yiddish Saying
• Extremes
• Up to 50 off. Up to 36 hours...

22

• How might Styrons experience not be
  representative of those of others suffering from
  depression?

23
The deep, fundamental question in statistical
analysis is Compared with what? -Edward Tufte
24
Is the outcome statistically significant?
25
Chance is lumpy. -Robert Abelson
26
An article in a leading medical journal
said...that Pargluva seemed to significantly
increase heart attack and stroke risks.
27
Whats the effect size?
28
New Diabetes Drug Poses Major RisksWashington
Post, October 21, 2005
New Diabetes Drug Poses Major Risks, Panel
Says Review Finds FDA Overlooked Data on
Life-Threatening Cardiovascular Effects of
Pargluva By Rob Stein and Marc
Kaufman Washington Post Staff Writers, Friday,
October 21, 2005 A02 A diabetes medicine poised
to win Food and Drug Administration approval
sharply increases the risk of heart problems,
strokes and death, researchers reported yesterday
in an analysis that raises new questions about
how the agency handles drug safety concerns.
A new diabetes medicine...sharply increases the
risk of heart problems...researchers reported
yesterday.
...the analysis found those taking the drug had
more than twice the risk of death, heart
attacks, and strokes, and nearly triple the risk
when all types of heart problems were included.
29
Are the results those of a single study or a
literature?
30
(No Transcript)
31
Ioannidis Review of Medical Research (Journal of
the American Medical Association, 2005)
Even carefully peer-reviewed studies in the
medical literature may not be fully supported
subsequently. Ioannidiss research suggested
that will happen about 32 of the time.
32
Whats the research design?
33
Students need to recognize
Not all research studies are experiments.
Not all studies called experiments are true
experiments.
34
RCTs
It was more of a triple-blind test. The
patients didnt know which ones were getting the
real drug, the doctors didnt know, and, Im
afraid, nobody knew.
35
How was the variable operationalized?
36
Whos in the sample?
37
Johns Hopkins Iraqi Casualty StudyThe Lancet

• Estimated 601,027 deaths by violence.
• For the single most important category--the
  total number of deaths by violence during the
  war--the confidence interval ranges from 426,369
  to 793,663. That means that we are 95 certain
  that the correct number is between those two...

If statesmen were better at arithmetic, wars
would be far fewer. -Benjamin Franklin
38
Medical Efficaciousness vs. Effectiveness
39
Controlling for what other variables?
40
Sir Richard Doll

• British epidemiologist, who with Bradford Hill,
  was asked to investigate the alarming increase in
  lung cancer cases in Britain after WWII.
• Interviewed lung cancer patients in 1949 and
  found that cigarette smoking was the strongest
  characteristic they shared.
• Went on to study 40,000 British doctors over 50
  years. Found that cigarette smoking reduced the
  life span by an average of 10 years.

41
What do the numbers show?
42
It is easy to lie with statistics, but easier
to lie without them.-Frederick Mosteller
43
What can we as teachers of psychology do to
promote QR habits of mind?

• Teach QR skills values.
• Teach for generalization!

44
Statistics and psychology have long enjoyed an
unusually close relationship--for they are
inextricably bound together.-Stephen Stigler
(1999)
45
Science is not a collection of facts, any more
than opera is a collection of notes. Its a
process, a way of thinking, a method, based on a
single insight--that the degree to which an idea
seems true has nothing to do with whether it is
true, and that the way to distinguish factual
ideas from false ones is to test them by
experiment. -Timothy Ferris (1998)
46

• How we present information.
• What we ask students to do.
• What we can do as teachers.

47
How we present information

• Be explicit about
• Whether a study represents itself or a
  literature.
• The contexts or typicality of examples.
• How terms and concepts in psychology are related
  to equivalents in public discourse.

48
How we present information

• Preach
• Why we use numbers.
• Why precision is important.
• How we evaluate numbers.
• How we can construct meaningful, principled, and
  effective arguments with numbers.

49
the purpose of statistics is to organize a
useful argument from quantitative evidence using
a form of principled rhetoric.-Robert Abelson
50

• Use findings from psychology to underscore the
  virtues of Q inquiry.
• Cognitive heuristics Lawson et al. (2003),
  Teaching of Psychology.
• Misperceptions of chance
  http//thehothand.blogspot.com/
• The psychology of belief.

51

• Show how psychology has contributed to QR.
• Measurement and its Q evaluation.
• Effect size and meta-analysis.

52
What we ask students to do

• Two-minute papers/Exam Qs
• Interpret a graphic, table, or numerical finding.
• Identify key questions that could be raised about
  a quantitative finding.

53

• Ask students to write with numbers.
• Beins (1993), Teaching of Psychology.
• Miller (2004), The Chicago Guide to Writing about
  Numbers.

54
Even for works that are not inherently
quantitative, one or two numeric facts can help
convey the importance or context of your
topic. -Jane Miller, The Chicago Guide to
Writing about Numbers
55

• Only the uppermost part of the oceans--the top
  two hundred meters--bears any resemblance to the
  sunlit waters we are familiar with, yet below
  that zone lies the largest habitat on Earth.
• Ninety percent of all the oceans water lies
  below two hundred meters, and its volume is
  eleven times greater than that of all of the land
  above the sea...
• Below six thousand meters lies a region known as
  the hadal zone... in the Marianas Trench off the
  Philippines it is 11,000 meters deep. Ships
  plying the waters over the trench glide as far
  above the Earths surface as do jet aircraft
  crossing the face of America.

-from a review by Tim Flannery of Claire
Nouvians The Deep, The New York Review of Books,
12/20/07.
56
QR Writing Assessment The Quant Squad

• Few (12) of the papers for which QR peripherally
  relevant in fact used numbers.
• Many students rely on weasel words (e.g., few,
  many).
• Students assume staples convey meaning.
• QR terminology varies by discipline (e.g.,
  experiment).

57
For any case study assignment

• Ask students to use numbers to set an instance
  in its wider context.
• How representative is this?

58
A radical notion illustrated Research on
Personality and Facebook
Opening sentence Facebook is a wildly popular
online social networking tool...
Revision Facebook is the Internet site most
frequently viewed by males and females, aged
17-25, in the United States today (eMarketer,
2007). Sixty-nine percent of females and 56 of
males in that age group have Facebook accounts
nearly 65 of users log on to Facebook once a
day.
59
How can students find relevant numbers?
60

• Ask students to engage in quantitative
  inquiry.
• http//www.acad.carleton.edu/curricular/PSYC/class
  es/psych110_Lutsky/RMI/index.html

Measures FF Traits, happiness, background
variables. Readings DeNeve (1999), Happy as an
Extraverted Clam, Current Directions in
Psychological Science, Text statistics
appendix. Learning goals Ss think like
psychological scientists, write with numbers,
learn about meta-analysis.
61
learners need to generate responses, with
minimal cues, repeatedly over time with varied
applications so that recall becomes fluent and is
more likely to occur across difference contexts
and content domains. -Diane Halpern Milt
Hakel (2003)
62
...numbers are the principal language of
public argument.More or Less, BBC News
Programme
63
we teachers do not automatically deserve a
future. We must earn it by the skill with which
we disorient our students, energize them, and
inculcate in them a taste for the hard
disciplines of seeing and thinking. -James
ODonnell, Avatars of the Word From Papyrus to
Cyberspace.