The State of Eating Disorders Research Publications 1980-2000. An Empirical Analysis

1
The State of Eating Disorders Research
Publications 1980-2000. An Empirical Analysis
Mary J. Markland, MA, AHIP (Southeast Clinical
Campus Librarian) Stephen A. Wonderlich, PhD
(Associate Chair/Professor Department of
Neuroscience) James E. Mitchell, MD
(Chair/Professor Department of Neuroscience)
Ross D. Crosby, PhD (Director of Biomedical
Statistics Methodology, Department of
Neuroscience/Neuropsychiatric Research
Institute) Martina de Zwaan, MD (Research
Scientist, Department of Neuroscience/Neuropsychia
tric Research Institute)
Research Questions
PsycInfo Search Strategies
Clear Inclusion/Exclusion Criteria
Confidence Intervals Presented
Population/Sample
Interval Estimates

• What is the quality of eating disorders
  publications?
• Has the quality of eating disorders publications
  changed over time?
• How does the quality of eating disorders
  literature compare to publications in anxiety?

Eating Disorders (anorexia nervosa OR bulimia OR
binge eating OR eating attitudes OR body image
disturbances OR eating disorder) AND (treatment
OR therap) Limits journal article,
English (anorexia nervosa OR bulimia OR binge
eating OR eating attitudes OR body image
disturbances OR eating disorder) NOT (treatment
OR therap) Limits journal article,
English Anxiety (anxiety disorder OR phobia
OR obsessive-compulsive disorder OR panic
disorder OR panic OR post-traumatic stress
disorder OR traumatic neurosis OR stress
reactions OR ocd OR ptsd OR post-traumatic stress
disorder) AND (anxiety management OR treatment
OR therap) Limits journal article,
English (anxiety disorder OR phobia OR
obsessive-compulsive disorder OR panic disorder
OR panic OR posttraumatic stress disorder OR
traumatic neurosis OR stress reactions OR ocd
OR ptsd OR post-traumatic stress disorder) NOT
(anxiety management OR treatment OR
therap) Limits journal article, English
The interpretation of the results of any study
depends on the characteristics of the population
included for analysis. Define the population
clearly. If control groups are part of the
design, present how they are defined Describe
the sampling procedures and emphasize any
inclusion or exclusion criteria.
Interval estimates should be given for any effect
sizes involving principal outcomes. Provide
intervals for correlations and other coefficients
of association or variation whenever possible.
APA Task Force on Statistical Inference
Methods

• Raters
• 8 raters with masters or doctoral level training
  in psychology/psychiatry
• Training
• 21 practice articles were reviewed jointly
• Procedure
• Citations were stored in EndNote libraries by
  subject, treatment vs. non-treatment and year
• Citations were randomly selected from EndNote and
  articles obtained
• Article assignments were balanced by field, year,
  content, and rater pairs
• All articles rated independently by 2 raters
• Raters were blind to journal, authors, and author
  affiliation
• Disagreements resolved via consensus
• Statistical Analysis
• Power Analysis n456 articles to detect small
  effect size for ED vs. Anxiety
• Interrater reliability assessed using kappa
  coefficients
• Statistical analysis performed using hierarchical
  log linear analysis

APA Task Force on Statistical Inference
Clinical Significance Data Presented
Interpretation
When you interpret effects, think of credibility,
generalizability, and robustness.
Random Assignment of Subjects
Assignment
Random assignment allows for the strongest
possible causal inferences free of extraneous
assumptionsprovide enough information to show
that the process for making the actual
assignments is random. (If) random assignment is
not feasible, describe methods used to attenuate
sources of bias, including plans for minimizing
dropouts, noncompliance, and missing data.
APA Task Force on Statistical Inference
25.6
15.8
15.8
9.2
9.1
7.8
Conclusions
Note the shortcomings of your studyacknowledging
limitations if for the pupose of qualifying
results and avoiding pitfalls in future research.
APA Task Force on Statistical Inference
Assessor Blind to Condition
Procedures
PubMed Search Strategies
APA Task Force on Statistical Inference
Clearly describe the conditions under which the
measurements are taken (e.g. format, time, place,
personnel who collected data). Describe the
specific methods used to deal with experimental
bias
Eating Disorders (eating disorder OR anorexia
nervosa OR bulimia) AND (therapy OR therapies
OR treatment) Limits human, journal article,
English (eating disorder OR anorexia nervosa
OR bulimia) NOT (therapy OR therapies OR
treatment) Limits human, journal article,
English Anxiety Disorders (anxiety disorder OR
obsessive compulsive disorder OR panic disorder
OR phobia OR agoraphobia OR stress, disorders,
post-traumatic OR post- traumatic stress
disorder OR ptsd) AND (therapy OR therapies OR
treatment) Limits human, journal article,
English (anxiety disorder OR obsessive
compulsive disorder OR panic disorder OR
phobia OR agoraphobia OR stress, disorders,
post-traumatic OR post- traumatic stress
disorder OR ptsd) NOT (therapy OR therapies OR
treatment) Limits human, journal article,
English
Conclusions

• Eating Disorder publications tend to be less
  methodologically rigorous than Anxiety Disorder
  publications in many important areas.
• Structured Interviews Prospective Longitudinal
  Designs
• Random Assignment Blind Outcome Assessment
• Both Eating Disorder and Anxiety Disorder
  publications demonstrate improvement in
  methodological rigor over the last 20 years.
• Sample Size Alpha constraint
• Effect Size Psychometrics
• The majority of Eating Disorder and Anxiety
  publications do not include many of the APA Task
  Force recommendations.
• Confidence Intervals Clinical Significance
• A priori power analysis Alpha constraint

APA Task Force on Statistical Inference
A Priori Power Analysis Mentioned
Power and Sample Size
Study Design
Provide information on sample size and the
process that led to sample size decisions.
Document the effect sizes, sampling and measuring
assumptions, as well as analytic procedures used
in power analysis.
1980 1990 2000 Overall
Eating Disorders 76 (38/38) 76 (38/38) 76 (38/38) 228 (114/114)
Anxiety Disorders 76 (38/38) 76 (38/38) 76 (38/38) 228 (114/114)
Total 152 (76/76) 152 (76/76) 152 (76/76) 456 (228/228)
APA Task Force on Statistical Inference
Results
Effect Size Data Presented
Effect Sizes
Limitations
Research Question/Hypothesis Stated
Design
Always present effect sizes for primary outcomes.

• Literature searches may not have found all the
  articles
• Excludes animal/basic science studies
• Excludes non-English articles
• Low interrater agreement on some ratings
• Raters could deduce field year from articles

Make clear at the outset what type of study you
are doing. Do not cloak a study in one guise to
try to give it the assumed reputation of
anotherbe sure to define and prioritize (the)
goals.
APA Task Force on Statistical Inference
APA Task Force on Statistical Inference