How to Assess a Miracle Cure

1
How to Assess a Miracle Cure

• Chapter 8

2
How to Assess a Miracle Cure

• People can have numerous emotional motivations
  for believing a claim about the efficacy of a
  treatment. They may be compelled by fear of an
  illness or of the side effects of a particular
  treatment, by the emotional appeal of sales
  pitches or promises of relief, by the pleasing or
  reassuring demeanor of a certain practitioner, or
  by mistrust of physicians.
• Such feelings may deserve our understanding, but
  it should be clear that they provide no grounds
  for belief that a given treatment is effective or
  ineffective.

3
How to Assess a Miracle Cure

• Theyre not adequate reasons for supposing the
  truth of any claim that a treatment is effective.
  
• There are, however, other reasons that people
  offer in support of claims about the
  effectiveness of treatments. Here are some of the
  most common and persuasive

4
How to Assess a Miracle Cure

• I tried it, and it worked
• Someone else tried it and it worked
• Dr.X says it works
• Dr. X observations of several patients show that
  it works
• An ancient practice or folklore shows that it
  works
• A scientist study shows that it works

5
How to Assess a Miracle Cure

• In statements 1 and 2, personal experience is
  supposed to be a good enough reason. In 3 and 4,
  a doctors authority or observations are offered
  as proof. In 5, its the experience of past
  generations or social groups. In 6, its the
  objective investigations of science. Each one of
  these reasons is probably assumed by millions of
  people to be perfectly adequate as proof or
  strong evidence of a treatments power to help or
  cure.

6
Personal Experience

• You have a headache. You drink a cup of herbal
  tea. In an hour your headache is gone. What
  could be more natural than to credit the tea for
  your pain relief? Isnt such a personal
  experience ( what is often called anecdotal
  evidence) the best and most direct way to learn
  whether a treatment works?
• Many people say yes. In fact, a large proportion
  of the claims made for unconventional therapies
  are biased solely on personal experience.
• Despite its strong appeal and despite the number
  of people who swear by it, there are good reasons
  why personal experience generally cannot tell you
  if a treatment really works.

7
Personal Experience

• Personal experience alone generally cannot
  establish the effectiveness of a treatment beyond
  a reasonable doubt.
• There are three reasons why this principle is
  true Many illnesses simply improve on their own
  people sometimes improve even when given a
  treatment known to be ineffective and other
  factors may cause the improvement in persons
  condition.

8
The Variable Nature of Illness

• One of the complexities that frequently confounds
  efforts to discover whether a treatment works is
  the self-limiting nature of illness. The fact
  is, most human ailments improve on their own
  whether a treatment is administered or not.
• Some chronic diseases like rheumatoid arthritis
  and multiple sclerosis (MS) can have spontaneous
  remissions, with symptoms vanishing for long
  periods of time MS symptoms can disappear for
  years.

9
The Variable Nature of Illness

• Even the course of cancer is variable.
  Spontaneous remissions of cancer, even
  particularly lethal types, have also been
  documented. Theyre rare, and their frequency
  varies according to tumor type. But because they
  do happen, they undermine attempts to
  legitimately claim that a single instance of a
  cure was due to any particular treatment.

10
The Placebo Effect

• A peculiar fact about people is that sometimes
  even if theyre given a treatment thats inactive
  or bogus, theyll respond with an improvement in
  the way that they feel. This response, called
  the placebo effect, is not all in the mind it
  can involve both psychological and physiological
  changes. What exactly is behind this effect isnt
  clear, but many experts say it depends on
  suggestibility, classical conditioning (previous
  experience with healing acts), expectation, and
  other factors.
• In many illnesses about one-third or more of
  patients will get better when given a placebo.
  (Placebos can also cause negative side effects,
  just as drugs can.)
• Placebo effects can be induced by sugar pills,
  worthless injections and devices, a
  practitioners reassuring manner, and
  incantations even by the act of walking into
  the doctors office.

11
The Placebo Effect

• The placebo effect can be especially impressive
  in the relief of pain.
• The risk of being misled by the placebo effect is
  why scientists include a placebo group in medical
  studies. The changes shown in the treatment group
  are compared to any changes in the placebo group.
• To be considered effective, the treatment under
  study must do better than sugar pills or sham
  therapies. Placebo may have a place in the modern
  practice of medicine. But they can also make
  worthless remedies look potent.

12
Do MDs Actually Prescribe Placebos?

• More often than you might expect
• http//www.physorg.com/news118591087.html
• http//www.bri.ucla.edu/bri_weekly/news_051212.asp
  
• But go here and read this critique
  http//skepdic.com/placebo.html
• And finally http//www.ncbi.nlm.nih.gov/sites/ent
  rez?dbpubmeduid11406341cmdshowdetailviewinde
  xedgoogle

13
Explaining Placebo

• Placebo pain relief is understood enough such
  that it can be chemically blocked.
• Classical Conditioning for placebo effects in
  general?
• Nocebo effect, the opposite, causing pain or harm
  with an inert product. Nocebo caused or
  aggravated pain is understood enough to also be
  blocked chemically.

14
Overlooked Causes

• Youve had an upset stomach for two days. A
  friend rubs a crystal amulet across your belly,
  and in a few hours your stomach settles. Did the
  crystal heal you?
• Maybe. But there are other possible causes of
  your relief, besides the placebo effect and
  natural fluctuations in your illness.
• People however frequently ignore other
  possibilities and adopt the explanation that
  suits them. This habit is a reliable formula for
  reaching false conclusions.
• For the three reasons just discussed the
  variable nature of illnesses, the placebo effect,
  and overlooked causes (and a few others), our
  principle must always guide us when we try to
  assess anecdotal evidence. Personal experience
  alone generally cannot establish the
  effectiveness of a treatment beyond a reasonable
  doubt.

15
Overlooked Causes

• Its this whirl of possible causes that scientists
  try to control in properly conducted research. By
  controlling these confounding factors, scientists
  hope to narrow down the possibilities to the true
  cause or causes of a condition. This task
  requires a systematic, objective
  approachsomething that personal experiences, by
  definition, isnt.

16
Overlooked Causes

• To fail to consider alternative explanations
  including the variable and self-limiting nature
  of illness, the placebo effect, and the presence
  of hidden causes is to risk committing the
  fallacy of false cause. This common mistake is a
  matter of believing that two events are causally
  connected when in fact no such connection has
  been shown to exist. A common version of this
  error is the assumption that because something
  occurred after something else, it was caused by
  it. This fallacy is known as post hoc, ergo
  propter hoc (After this, therefore because of
  this.)

17
Overlooked Causes

• Its also important to realize that if one persons
  experience generally cant provide reliable
  evidence of a treatments efficacy, neither can
  the personal experiences of many people. If one
  person can commit the fallacy of false cause, so
  can a hundred.
• Its not surprising, then, that numerous claims of
  the effectiveness of treatments, though affirmed
  by many testimonials. Have been shown to be false
  by controlled scientific testing.

18
The Doctors Evidence

• As noted in Chapter 5, appeals to authority can
  indeed give us good reasons for accepting a
  claim. But the authority must be qualified to
  speak on the question at issue. To be qualified,
  an authority must have demonstrated an ability to
  make reliable judgments about the question at
  issue. Its this ability that makes someone an
  authority. Its not the degree behind her name,
  or the school she attended, or her reputation
  among her peers although all these factors may
  be good indicators of whether she possess the
  requisite expertise.

19
The Doctors Evidence

• Medical doctors are, of course, authorities,
  assuming they have shown themselves to be
  effective healers. Theyre authorities in the
  diagnosis of disease and in applying therapeutic
  techniques and technology to their patients.
  They have the requisite ability to make reliable
  judgments in treating human illness.
• But are they authorities on which remedies work
  and dont work? Yes if they have the requisite
  ability to reliably judge the evidence that bears
  on such questions.
• Contrary to what many people believe, a doctors
  work with patients generally cant give us the
  evidence required to assess the effectiveness of
  a therapy.

20
The Doctors Evidence

• Accounts of a doctors observations of individual
  patients are called case reports (also case
  series, case histories, and descriptive studies).
  They can be extremely valuable to other doctors
  and to medical scientists.
• But, as Gehlbach points out, such accounts do
  not provide detailed explanations for the cause
  of disease or offer the kind of evidence we need
  to evaluate the efficiency of a new treatment.
• Though doctors monitor patients and keep records,
  case studies are compiled without the strict
  controls found in scientific studies, so
  confounding factors cant be ruled out.

21
The Doctors Evidence

• Case reports are also vulnerable to several
  serious biases that controlled research is better
  able to deal with. One is called social
  desirability bias. It refers to the patients
  tendency to strongly wish to respond to treatment
  in what they perceive as the correct way. People
  will sometimes report improvement in their
  condition after treatment simply because they
  think thats the proper response or because they
  want to please the doctor.
• Another bias can come from doctors themselves.
  Called investigator bias, it refers to the
  well-documented fact that investigators or
  clinicians sometimes see an effect in a patient
  because they want or expect to see it.

22
The Doctors Evidence

• Case studies alone generally cannot establish the
  effectiveness of a treatment beyond a reasonable
  doubt.
• When claims of a treatments effectiveness are
  based solely on case studies or personal
  experience, you generally cannot know that the
  treatment is effective.

23
The Appeal to Tradition

• The point is not that remedies backed by ancient
  practice or folklore dont work. Some do.
  Indeed, many modern treatments are actually based
  on such treatments. The point is that the mere
  fact that a treatment is supported by such
  revered experiences does not mean it works.
  Maybe it does, maybe it doesnt. Scientific
  testing can often reveal the answer, and ancient
  practice or folklore can sometimes provide leads.

24
The Reasons of Science

• The arguments often sound like this Scientific
  studies show that treatment X works. Or
  Scientific research indicates that treatment X
  alleviates symptoms of condition Y.
• Scientific evidence gained through controlled
  experiments unlike personal experience and case
  studies generally can establish the
  effectiveness of a treatment beyond a reasonable
  doubt.

25
The Reasons of Science

• Unfortunately, this principle alone wont get you
  very far. In books and magazines, in newspapers,
  on radio and television, and in private
  conversation, youre peppered with appeals to
  scientific evidence.
• If youre not a scientist, how are you to
  evaluate these appeals to scientific evidence?

26
The Reasons of Science

• First, understand that its indeed possible for
  nonscientists to make some reasonable judgments
  about medical evidence. Most of the time, of
  course, you must rely heavily on reliable
  authorities for guidance. But even without such
  guidance you can often draw reasonable and useful
  conclusions about medical research if you
  understand some of the peculiar characteristics
  and limitations of this kind of evidence.

27
Medical Research

• Single Studies
• It may seem reasonable to assume that one medical
  study can usually offer conclusive evidence
  because its conducted by scientists who try to
  be objective and contentious and because, after
  all, its science. But this assumption is false.
• To minimize this potential for error, inadequacy,
  or fraud, medical scientists seek replication.
  Several studies yielding essentially the same
  results can render a hypothesis more probable
  than would a lone study.
• Despite the impression often left by the media,
  medical breakthroughs arising out of a single
  study are extremely rare.
• Single medical studies generally cannot establish
  the effectiveness of a treatment beyond a
  reasonable doubt.

28
Medical Research

• Conducting medical research is exacting work, and
  many things can go wrongand often do. Several
  scientific reviews of medical studies have
  concluded that a large proportion of published
  studies are seriously flawed. The mere fact that
  research reports are published even in the most
  prestigious journals, is no guarantee of their
  quality.
• Confounding variables and bias may creep in and
  skew results.
• When the results of studies conflict, scientists
  try to sort things out. They criticize the
  existing studies they do bigger and better
  studies. The process can continue for years until
  the issue is resolved.

29
Medical Research

• Conflicting Results
• When the results of relevant studies conflict,
  you cannot know that the treatment in question is
  effective.
• Studies Conflicting with Fact
• New study results that conflict with
  well-established findings cannot establish the
  effectiveness beyond a reasonable doubt.
• Limitation of Studies
• All medical studies are not created equal.
  Medical studies can vary in more than just
  quality.

30
Types of Studies

• In Vitro Experiments
• Test-tube studies (in vitro means within a
  glass) are the most fundamental kind of study
  though they may or may not actually be done in
  test tubes.
• What transpires in a Petri dish may never happen
  in a living body.
• Generally the chances of test-tube results being
  duplicated in humans is low.
• Test-tube studies alone generally cannot
  establish the effectiveness of a treatment beyond
  a reasonable doubt.

31
Types of Studies

• Animal Studies
• Animal experiments can give scientists important
  leads in understanding human disease. But by
  themselves, animal studies cant show that a
  therapy works in humans, nor can they show that a
  substance is safe for humans.
• It should be no surprise, then, that most
  treatments proven effective in animals usually
  dont pan out in humans. So our guiding
  principle must be the following Animal studies
  alone generally cannot establish the
  effectiveness of a treatment beyond a reasonable
  doubt.

32
Types of Studies

• Observational Studies
• Studies based on observing human subjects are
  also called nonintervention or epidemiology
  studies. They include several kinds of human
  studies whose names you often see in the titles
  of medical articles case-control, cohort,
  cross-sectional, prospective. Their common
  feature is that they dont involve intervening in
  the subjects lives to test something. They
  dont involve administering treatments. (This
  characteristic distinguishes them from
  intervention studies, usually called clinical
  trials, in which scientists do intervene in
  subjects lives.)

33
Types of Studies

• Observational Studies
• The purpose of observational studies is to
  examine the natural course of health and disease.
• The important thing to remember here is that
  observational studies alone no matter how many
  thousands of subjects they include cannot prove
  cause-and-effect relationships. They can only
  show associations and thus hint at possible
  causal connections.
• Observational studies alone generally cannot
  establish the effectiveness of a treatment beyond
  a reasonable doubt.

34
Types of Studies

• Clinical Trials
• Of all the different types of medical studies,
  clinical studies offer the strongest and clearest
  support for any claim that a treatment is
  effective because they can establish cause and
  effect beyond a reasonable doubt. Clinical trials
  allow scientists to control extraneous variables
  and test one factor at a time.
• In clinical trials designed to test treatment
  efficiency, an experimental group of subjects
  received the treatment in question. A control
  group thats as similar to the experimental group
  as possible doesnt get the treatment. (Use of a
  control group makes the study a controlled
  trial.) By comparing results in the experimental
  group to those in the control group, researchers
  can determine whether the experiment treatment
  was more effective then would be expected because
  of these factors alone.

35
Types of Studies

• Clinical Trials
• To minimize confounding factors, subjects in the
  control group often receive a placebo. (Such a
  study is then referred to as a placebo-controlled
  trial.) Scientists compare the results in the
  experiment with that in the placebo-control
  group. If the experimental treatment is truly
  effective and merely placebo itself it should
  perform much better than the placebo.
• Another extremely important element in clinical
  trials is blinding a practice used to ensure
  that subjects (and, if possible, researchers)
  dont know which subjects are getting the
  experimental treatment or the placebo.
  Well-designed clinical trial are double-blind,
  which means that neither the subjects not the
  scientists know whos getting which treatment.

36
Types of Studies

• Clinical Trial Limitations
• Lack of Control Group Without a control group, a
  clinical trial generally prove very little, if
  anything. Clinical trials without a control group
  (called uncontrolled) are about as useful as
  evidence as are testimonials and case studies.
• Faulty Comparisons The experimental group and
  the control group should be as alike as possible
  in all important respects. When they are not
  alike, confounding factors can skew the study
  results. In clinical trials, its frequently
  critical that groups be comparable in health
  status, occupation, race, age, income,
  nationality, and relevant behaviors like exercise
  and smoking. To protect against the problem of
  noncomparable groups, scientists use a technique
  called randomization. Subjects are randomly
  assigned to either the experimental or control
  group.

37
Types of Studies

• Clinical Trial Limitations
• 3. Small Numbers. Some clinical studies may
  have fewer than thirty subjects. These studies
  are generally considered to be pilot studies,
  offering a quick and relatively inexpensive way
  to test a treatments possibilities.
• Clinical trials limited by lack of a control
  group, faulty comparisons, or small numbers
  generally cannot establish the effectiveness of a
  treatment beyond a reasonable doubt.