James E' Haddow, M'D'

1
EDUCATIONAL CHALLENGES PRESENTED BY NEW
CAPABILITIES IN THE LABORATORY NCHPEG Annual
Meeting September 24, 2009

• James E. Haddow, M.D.
• Women Infants Hospital
• Alpert Medical School of Brown University

2
Even though this talks focus will be limited to
DNA testing

• The number of new tests is large
• The range of potential applications is also large
• Some of these tests are complex, with multiple
  SNPs measured
• Collecting data to document their performance
  requires funding and time
• There is economic and consumer-driven pressure to
  put them into practice

3
Within contemporary society, the anticipation is
that these new DNA tests will contribute to a
longer and healthier life, by somehow altering
our destiny, as depicted originally in Greek
mythology by the three fates.
4
Groups in the U.S. using anevidence-based
approach to assessgenomic tests

• United States Preventive Services Taskforce
  (USPSTF), sponsored by the Agency for Health Care
  Quality (AHRQ)
• Evaluation of Genetic Applications in Practice
  and Prevention (EGAPP) working group, sponsored
  by the Office of Public Health Genomics at the
  Centers for Disease Control
• Similar government-sponsored groups exist in
  Canada, Europe and elsewhere

5
Criteria for Evaluating a Screening or Diagnostic
Test (Includes both laboratory tests and other
approaches, such as questionnaires)
Wald N, Cuckle H. Reporting the assessment of
screening and diagnostic tests. Br J Obstet
Gynaecol, 198996389-396
6
Three Critical Elements for Test Evaluation

• The detection rate (sensitivity)
• The false positive rate (1-specificity)
• The odds of being affected, given a positive
  result (OAPR) equivalent to positive predictive
  value

7
Applying these criteria to a Prader-Willi
syndrome question A problem presented to the
New England Regional Genetics Group (NERGG) in
1995
8
Stated Problem Lack of assurance about
manufacturers quality control of molecular
probes used for detection Prader-Willi syndrome
9
Preliminary discussion leads to conclusion that
reagent quality appears satisfactory, but quality
control on a lab-by-lab basis is unacceptable.
10
The evaluation of Prader-Willi testing shifts to
the checklist. The evaluation now begins by
focusing on the disorder.
11
The Medical Disorder Being Sought Prader-Willi
syndrome is well defined and sufficiently serious
to warrant consideration of testing
12
Target Populations that Might be Tested for
Prader-Willi Syndrome, Using the Molecular Probe

• Individuals clinically diagnosed with
  Prader-Willi syndrome.
• Infants suspected clinically to have Prader-Willi
  syndrome.
• Pregnant women having amniocentesis for other
  purposes.
• Obese children whose parents want testing for
  Prader-Willi syndrome.

13
Prevalence of Prader-Willi Syndrome in the Four
Target Populations
Target Population Prevalence Individuals
diagnosed with Prader-Willi 1001 Infants
suspected to have Prader-Willi 110 Women
having routine amniocentesis
115,000 Obese children with no other findings
11,000,000
14

• The test being used is a molecular probe for
  detection of a microdeletion.
• It is considered a diagnostic test.
• It can detect 7 out of 10 cases of Prader-Willi
  syndrome (the other three are caused by
  uniparental disomy).

15

• The false positive rate for the microdeletion
  test is not known.
• Arbitrarily assigned a rate of 1 per 1000 tests
  (0.1)

16

• The prevalence of Prader-Willi in the general
  population is 115,000.
• Knowing the detection and false positive rates of
  the molecular probe, and the prevalence of the
  disorder, we can now calculate the OAPR (or PPV)
  for the four target populations.

17
Reliability of a Positive Test Result
(microdeletion) for Prader-Willi Syndrome in Four
Target Populations
Odds that a positive test result Target
Population is correct (PPV) Individuals
diagnosed with Prader-Willi 7001 Infants
suspected to have Prader-Willi 701 Women
having routine amniocentesis 121 All
obese children 11000
18
ACCEA CDC-Sponsored Project
AIM To develop and test a model system to assess
the available quantity and usefulness of existing
data on DNA-based tests and testing algorithms.
PURPOSE To provide an up-to-date, accurate and
complete summary of available information in
forms that are useful to policy-makers,
professionals and the general public.
19
The Model System is Interpreted in Five Steps

• Defining the disorder and setting

• A nalytic validity

• C linical validity

• C linical utility

• E thical, legal and social implications

The ACCE project was supported by a cooperative
agreement with CDC, Office of Genomics and Public
Health (CCU319352)
20
The ACCE Model System
Effective
Quality
Intervention
Assurance

(Benefit)
Natural
Pilot
Clinical
History
Trials
Sensitivity
Prevalence
Clinical
Specificity
PPV
NPV
Ethical, Legal,
Disorder
Health
Social Implications
Setting
Risks

(safeguards impediments)
Penetrance
Test
Analytic
Assay
Sensitivity
Robustness
Analytic
Quality
Specificity
Control
Economic
Monitoring
Evaluation



Evaluation
Education
Facilities
21
Setting the stage for applying the conceptual
framework

• Is the disorder well defined?
• What is it?
• Is it sufficiently serious to warrant testing?
• Is the prevalence known in the population to be
  tested?
• What is the specific test being applied to the
  population?

22
Analytic validity (ACCE)

• How effectively does the test measure the
  genotype/mutation of interest?

23
Clinical validity (ACCE)

• How closely associated is the genotype/mutation
  with the target clinical disorder in the setting
  to which it is applied?

24
Clinical validity (ACCE)
For a screening test, what is

• The detection rate for the disorder being sought
  (sensitivity)?
• The false positive rate (1-specificity)?
• The odds of being affected, given a positive
  result (positive predictive value)?

25
Clinical validity (ACCE)

• When a diagnostic test follows a positive
  screening test, what is
• The detection rate?
• The false positive rate?
• The odds of being affected, given a positive
  result for that test?

26
Clinical utility (ACCE)

• What harms are associated with testing?
• What therapeutic intervention will follow a
  positive screening or diagnostic test?
• How effective are these interventions in
  reducing risk, or treating existing conditions?
• What are the harms associated with treatments?
• What costs are incurred/avoided?

27
Ethical, legal and social issues (ACCE)

• What ethical, legal, and social issues are
  associated with the screening test or inquiry?
• What safeguards have been described and are these
  safeguards in place and effective?

28
Asking family history questions about
breast/ovarian cancer the best current example

• Aim - identify women at high risk for hereditary
  breast/ovarian cancer caused by mutations in
  BRCA1 and BRCA2 genes.
• Strategy - use a standardized history form
  (computer-based), and interpret the histories
  using a well designed and validated algorithm.

29
What is known?

• The prevalence of disease-causing BRCA1/2 gene
  mutations in the U.S.
• among Caucasian women is about 1 in 350.
• among Ashkenazi Jewish women is about 1 in 50.
• The false positive rate is about 3 for one
  computerized history form. This means that 1 in
  30 women completing the form will be identified
  as candidates for genetic testing but wont carry
  a BRCA1/2 mutation.

30
What is not known?

• The detection rate of family history screening
  for identifying women with a BRCA1/2 mutation
• it is estimated to be 40 60.
• among 3,500 women screened, 10 will carry a
  BRCA1/2 mutation. Four to six of these might be
  identified.
• among 3,500 women screened, about 100 (3) will
  be false positives.
• The overall odds of carrying a mutation, given a
  positive family history would be 5100, or 120
  (a 5 PPV).

31
What is known about the diagnostic sequencing
test for BRCA mutations?

• Detection rate is c. 95
• Variants of unknown significance occur in 5-10
  of cases tested.

32
Why is this a good example?

• A positive screening test leads to a well
  defined diagnostic test.
• The high risk nature of a mutation is well
  understood.
• Risk reducing strategies (chemotherapy, surgery)
  are available.
• Family studies can amplify the potential
  benefits.

33
What implications can be drawn from this example?

• Family history screening applications for other
  disorders with a genetic component ought to be
  assessed via the same/similar framework.
• Validated questions and interpretations should be
  developed for each application.
• Both health benefits and harms need to be
  measured.
• Consistent and transparent policy making should
  be applied to findings from research-based
  clinical applications.

34
Evaluation of Genomic Applications in Practice
and Prevention (EGAPP) www.egappreviews.org
Example of Evidence Reports Outcomes of
Genetic Testing in Adults with a History of
Venous Thromboembolism Example of
Recommendations Genetic Testing Strategies in
Newly Diagnosed Individuals with Colorectal
Cancer Aimed at Reducing Morbidity and Mortality
from Lynch Syndrome in Relatives
35
Analytic Framework
From the EGAPP review entitled Testing for
cytochrome P450 polymorphisms in adults with
non-psychotic depression treated with SSRIs
36
Meeting the Educational Challenges from New
Laboratory Capabilities

• Keep basic evaluative principles in mind
• Understand the characteristics of the population
  being tested
• Ask How much the test will add to existing
  care?
• Look to recommendations from independent panels