Systematic Review of Research Methodology in Telemedicine Studies

1
Systematic Review of Research Methodology in
Telemedicine Studies

• Pamela Whitten
• Liv Karen Johannessen
• Tove Soerensen
• Deede Gammon
• Michael Mackert

2

• Article in press, Journal of Telemedicine and
  Telecare

3
Rationale

• Review articles struggling to document large
  scale evidence of the positive effect of
  telemedicine on health and cost outcomes
• Previous review articles claim methodology often
  poor in published papers

4
Research Questions

• What is the incidence of theory testing in
  telemedicine literature? What theories are
  actually tested?
• What of studies clearly report specific study
  variables (e.g., objective, hypotheses/RQ, and
  type of subjects, randomization, units of
  analysis, location, time frame)
• What data collection strategies and analyses were
  employed?
• Were results presented in qualitative and/or
  quantitative fashion?

5
Methodology

• Keyword search strategy employed on 15 scientific
  databases
• Initial search yielded more than 10,000 articles
• Articles then assessed against inclusion criteria
  (e.g., published post 1990, included data,
  included some form of a methods section, etc.)
• N1615 articles
• Inter-rater reliability (Cohens Kappa) exceeded
  .80 for three pairs of coder

6
Results

• 5 mention theory or paradigmatic approach
• 96 report overall study aim
• Seek some form of outcome (80)
• Test technical feasibility (13)
• Address hypothetical issues (7)
• 11 provided explicit hypotheses or research
  questions

7
Results (cont.)

• 26 did not provide of subjects
• For those reporting patients as subjects
• Median 47
• 65 had sample size of 100 or less
• For those reporting providers as subjects
• Median 29
• 86 had sample size of 100 or less
• 11 reported randomization for patient studies
  and 4 for provider studies (most papers did not
  even mention subject selection strategies)

8
Results

• 83 reported location of study
• 34 in US
• 24 Europe
• Data collection
• Patients
• 31 multiple data collection strategies
• Image Transfer (35)
• Providers
• 18 multiple data collection strategies
• Surveys (60)

9
Results

• Few significant relationships
• Use of theory and presence of formal research
  questions (r0.25, Plt0.01)
• Research questions and study aim (r0.067,
  Plt0.01)
• Inclusion of theory and study based on an actual
  intervention (r-0.069, Plt0.01)

10
Discussion

• These methodological challenges not owned by
  telemedicine
• Implications of missing methodological details
  should not be underestimated
• Time impacts technologies
• Subject selection and sample size matter
• Lack of explicitly stated RQs prevent true
  critique of study design

11
Kuhn Basic Tenet

• The existence of intertwined theoretical beliefs
  and methodological strategies make it possible
  for a field to evaluate its own body of research