The Social Cost of Scientific Misconduct and Questionable Research Practices

1
The Social Cost of Scientific Misconduct and
Questionable Research Practices

• William Gardner, PhD

Departments of Pediatrics, Psychology,
Psychiatry The Ohio State University
Center for Innovation in Pediatric Practice,
Columbus Childrens Research Institute
Supported by NIH/ORI 1R01 NS49591.
2
Social Costs of Research Misconduct (RM)

• How much should we care about RM?
• A lot, because integrity is essential to
  science.
• But why should non-scientists care about this?
• Consequentialist answer A lot, because research
  misconduct may have a large social cost.
• My talk is conceptual, not empirical. Ill define
  this social cost and suggest how to study it.

3
Direct, Visible Economic Costs of Research
Misconduct

• Steneck What does the NIH lose from RM?
• Direct Cost ? Loss Grant Award s.
• Visible Cost ? Use only known RM cases.
• Loss / Year (N RM Cases / Year) x Cost / Case.
• Cost / Case 427K (Avg 2005 NIH award).
• N RM Cases 10 cases / Year.
• Therefore, Loss / Year 4.3M.

4
Direct, Invisible, Economic Costs of Research
Misconduct

• Invisible Cost ? Impute rate of RM cases based
  on hypothesized incidence of RM.
• N of NIH Grants (2005) 47,345.

5
Indirect Economic Costs of Research Misconduct

• Indirect Cost ? Loss monetary value of
  benefits that society would have received from
  research, but did not.
• Challenge 1 Measure monetary value of science.
• Challenge 2 Identify the benefits that society
  did not receive because of RM.

6
Two Thought Experiments

• Did research misconduct delay the start of
  smoking prevention efforts?
• Does research misconduct reduce the efficiency of
  pharmaceutical research?

7
Thought Experiment Scientific Misconduct
Smoking
Massive behavior change, catalyzed by
dissemination of scientific research.
8
Reduction of CVD Mortality Attributable to
Smoking Reduction
Based on Cutler (2001).
4.5 years improvement in life expectancy over 50
years due to reduction in CVD mortality.
Of that, 1.5 years reflects behavioral change.
Of that, assume .75 years is smoking reduction.
9
Additional Life-Years From CVD Mortality
Reduction Attributable to Smoking Reduction
Accumulate the per capita gain in chance of
surviving a year across population.
10
Annual Value (in B) of CVD Mortality Reduction
Attributable to Smoking Reduction
Valuing a year of life at 100,000 (Murphy
Topel, 2001).
11
What if the Surgeon General Spoke Out in 1959?
Scientific debate about smoking was rife with
misconduct.

• Counterfactual curve
• Suppose that RM QRP had not clouded the debate.
  
• Suppose that prevention efforts had started 5
  years earlier?

12
Additional Benefit From Reduced CVD Mortality
Attributable to Earlier Smoking Reduction
Shaded area is the value of the reduction of CVD
mortality, had we started 5 years earlier ( gt
100B).
Moving prevention efforts forward 1 year saves
27B.
Under these counterfactuals, these are the social
losses attributable to RM.
13
Thought Experiment Misconduct, QRP, New Drugs
RM QRP results in the direct loss of investment.
However, RM in the drug development may also
reduce N of new drugs / invested.
Fewer new effective drugs means less reduction in
mortality. Indirect Cost Value of life-years
not saved.
14
The Drug Development Process

• Drug development is a multiple stage process,
  from pre-clinical investigations through multiple
  phases of CTs.
• Later stages of development involving animals and
  humans are far more expensive than earlier stages.

15
Misconduct The Drug Development Process

• Misconduct increases the False Positive Rate in
  the drug development process.
• Small increases in the False Positive Rate in
  drug testing will substantially increase the
  proportion of useless compounds tested in
  expensive later stages.
• This wastes research dollars, a direct cost of
  RM.
• However, it also increases the investment cost of
  developing a useful drug ? fewer drugs are
  developed.

16
The Value of the Missing Drugs

• The indirect cost of RM in drug development are
  those missing drugs.
• Econometric models suggest that better drugs for
  CVD have a larger impact than smoking reduction.
• Hence the annual value of life-years saved due to
  improved CVD medication alone is worth tens of
  B.
• Small inefficiencies in the drug development
  process will result in losses that are likely to
  be scaled in B.

17
Summary Social Cost of Fraud in Pharmaceutical
Research

• RM and QRP make it harder to eliminate bad ideas
  early, and may induce other researchers to pursue
  false leads, reducing research efficiency.
• Therefore, RM and QRP increase the cost of
  discovery, reduce the rate of discovery, and
  reduce the rate of growth of life expectancy.
• RM QRP are like securities fraud Society loses
  from both the loss of the investment and reduced
  economic growth from the misallocation of capital.

18
So we have proved what, exactly?

• Buck Mulligan, in the Martello Tower, speaking of
  Stephan Dedalus
• Its quite simple. He proves by algebra that
  Hamlets grandson is Shakespeares grandfather
  and that he himself is the ghost of his own
  father.
• James Joyce, Ulysses

I am waving my hands. The point of waving my
hands is to point out a new question, and suggest
how we might try to answer it.
19
Limitations of Thought Experiments

• Our examples do not include data on misconduct.
• Our arguments rely on economic models that are
  controversial.
• We consider only medical examples.

20
Conclusion to Thought Experiments

• We have clarified the meaning of an indirect
  social cost of research misconduct.
• We have also clarified the stakes in RRI.
• Stenecks estimate of the direct costs of RM were
  expressed in M.
• I think the value of indirect social costs of RM
  should be expressed in B (3 orders of
  magnitude).

21
Conclusion The Social Costs of Scientific
Misconduct

• Because RM is relatively rare, the direct costs
  of RM are likely small.
• The social value of science, however, is titanic.
• Inefficiencies in technology transfer caused by
  RM should be projected across this large value.
• Social costs of RM are greater than are
  imaginged, and vastly greater than the cost of
  research on RI.