Cognitive Rehabilitation Outcomes for Traumatic Brain Injury

1
Cognitive Rehabilitation Outcomes for Traumatic
Brain Injury

• Nancy Carney, PhD
• Assistant Professor, Division of Medical
  Informatics and Outcomes Research
• Oregon Health Science University
• Hugo du Coudray, PhD
• Professor Emeritus, Department of Psychology
• Portland State University
• Professor, Department of Neurological Surgery
• Oregon Health Science University

2
Cognitive Rehabilitation Outcomes for Traumatic
Brain Injury

• Original systematic review and evidence report
  supported by the Agency for Healthcare Research
  and Quality.

3
Presentation Outline

• I. Evidence Report - Rehabilitation for Traumatic
  Brain Injury (1998)
• process and parameters
• findings - evidence for effectiveness
• findings - limitations of research
• II. Update
• process and parameters
• findings - evidence for effectiveness
• findings - research design
• III. Recommendations

4
Process

• Causal Pathway

Intermediate Outcome
Intervention
Health Outcome
5
Process
Causal Pathway
Scores on Neuropsych. Tests
b
Memory Retraining
c
a
Remembers how to ride bus
6
Process
7
Process

• Direct Evidence
• In the context of a systematic review, direct
  evidence comes from comparative studies that
  examine the effect of CR on measures of real
  health outcomes (arcs 1 and 2 of the causal
  pathway).
• Indirect Evidence
• Causal chain that relies on intermediate
  measures. Does CR improve scores on intermediate
  measures (Arc 3) and does improvement on those
  scores associate with better outcomes (Arcs 4 and
  5)?

8
Process

• 3,098 articles were specified in original search
  (1976 to 1997)
• After final abstract review, 114 full text
  articles were read, 32 were abstracted for
  evidence
• 11 randomized controlled trials
• - 5 measured relevant health outcomes
• - 6 measured intermediate outcomes
• 4 comparative studies
• - 1 measured employment outcomes
• - 3 measured intermediate outcomes

9
Process

• 8 studies of the relationship between
  intermediate tests and employment
• 9 observational studies
• - 1 measured relevant health outcomes
• - 8 measured intermediate outcomes

10
Findings - Evidence for Effectiveness

• Conclusions
• One small RCT (Schmitter-Edgecombe, 1995) and one
  observational study (Wilson, 1997) provide
  direct evidence of the effect of compensatory
  cognitive devices (notebooks, wristwatch alarms,
  programmed reminder devices) on the reduction of
  EMFs for persons with TBI
• Durability of effects are unproven

11
Findings - Evidence for Effectiveness

• Conclusions
• A second RCT (Helfenstein, 1982) provides
  evidence that compensatory cognitive
  rehabilitation reduces anxiety, and improves
  self-concept and interpersonal relationships for
  persons with TBI
• Durability of effects are unproven

12
Findings - Evidence for Effectiveness

• Conclusions
• Three RCTs (Kerner, 1985 Thomas-Stonell, 1994
  Twum, 1994) and two comparative studies (Gray,
  1992 Wood, 1987) provide limited evidence that
  practice and CACR improves performance on
  laboratory-based measures of immediate recall.
• Links between cognitive tests and health outcomes
  are unstudied
• Links between cognitive tests and posttrauma
  employment/productivity are equivocal

13
Findings - Evidence for Effectiveness

• Conclusions
• No studies evaluated the link between cognitive
  tests and health outcomes, and associations
  between performance on cognitive tests and
  post-trauma employment and productivity were
  equivocal.

14
Findings - Limitations of Research

• Classification of Outcomes
• There is no standard set of outcome measures for
  cognitive rehabilitation that can be used across
  clinics to evaluate both patient progress and
  program effectiveness.
• In the 23 studies reviewed, 91 different
  instruments were used to measure outcome.
• Mainly they are intermediate measures, tests
  generated in neuropsychology to probe the
  functioning of a specific performance such as
  attention, memory, etc.

15
Findings - Limitations of Research

• The problem of clinic-specific measures
• 25 of the 91 outcome measures used were
  clinic-specific developed by local clinicians to
  evaluate their particular programs.
• This kind of measure showed the highest
  proportion of positive effects.
• This may reflect more valid evaluation of
  effects under local conditions, or it may be the
  biased result of using an outcome measure that
  has not been cross validated on other
  populations. But which is correct? and to what
  degree?

16
Findings - Limitations of Research

• Spontaneous Recovery
• Either through insufficient follow-up time, or by
  use of comparison groups with large baseline
  differences from treatment groups, the effect of
  spontaneous recovery is not clearly controlled in
  these studies.

17
Findings - Limitations of Research

• General therapeutic effects of stimulation
• Researchers are well aware of the confounding
  effects of stimulation and usually try to control
  for it in tests of CR treatment.
• In general, the studies in this review that did
  not produce a treatment effect compared one form
  of cognitive rehabilitation with another.
• Significant effects were not found when one kind
  of treatment was compared with another, given
  equal levels of stimulation for both groups.

18
Findings - Limitations of Research

• Duration of Intervention
• Many of the interventions in the published
  studies provide minimal treatment hours that do
  not accurately reflect the years of work required
  to rehabilitate the person with brain injury.

19
Results of this Report

• One year after the release of the report AETNA (a
  major medical insurance company in the United
  States) changed its funding policy
• Before the report, their general policy was to
  deny funding for cognitive rehabilitation
• In their announcement of a new policy to fund
  cognitive rehabilitation under specific
  circumstances, they cited the AHRQ evidence
  report, as well as publications by colleagues, to
  justify their decision.

20
UPDATE

• Effectiveness of Rehabilitation for Cognitive
  Deficits
• Cardiff, Wales
• September, 2002

21
Process

• Publication inclusion criteria
• Randomized trials
• Comparative studies
• Systematic reviews or meta analyses
• Traumatic Brain Injury only. Trials were
  excluded if they included non-TBI cases unless
  the data for persons with TBI were reported
  separately.

22
Process

• Electronic search of Medline, PsychInfo, Cinahl,
  and the Cochrane Controlled Trials Register from
  1996 to present
• 1904 abstracts screened
• 27 publications read
• 19 included
• 6 randomized trials
• 2 comparative studies
• 1 meta-analysis
• One study from 1991 which was missed in the
  original review is included in this update.

23
Process

• Limitations of the Update
• Systematic but not yet comprehensive. It has not
  been circulated for the essential peer review to
  ensure that all relevant literature has been
  included.
• Does not include observational studies.
• Does not formally distinguish strength of
  evidence (e.g., Class I, II, III).
• This is a report on an update in progress,
  prepared especially for this meeting.

24
Findings - Evidence for Effectiveness

• Six Randomized Controlled Trials
• 3 with follow-up periods
• Berg et al., 1991
• Fasotti et al., 2000
• Salazar et al., 2000
• Two with effect tests immediately following
  intervention and one with a 1-week follow-up
• Dirette et al., 1999
• Eakman et al., 2001
• Levine et al., 2000

25
Findings - Evidence for Effectiveness

• Due to variability in samples, duration of
  intervention, and research design in general, the
  data across studies cannot be meaningfully
  aggregated, so we report results of each study
  separately.
• Results will be reported as follows
• Sample characteristics
• Intervention
• Measures and results
• Strengths and weakness of the study

26
Findings - Trials with Follow-up

• Berg et al., 1991
• Tested the effect of strategy training for memory
  deficits among 39 adults with moderate to severe
  TBI.
• Minimum 9 months post-injury (average for
  treatment group, 5.3 years placebo group, 6.3
  years control group, 6.8 years).
• At the time of intervention, all participants
  were living independently.

27
Berg et al., 1991

• Intervention was provided 3 days/week, 1
  hour/session, over 6 weekstotal of 18 hours.
• Strategy training (n 17) Cognitive strategies
  for overcoming barriers to memory were explained,
  demonstrated, and practiced. Participants
  completed daily homework. Targets for
  remediation were individualized for each patient.
• Pseudo-Training (placebo group, n 11) Equal
  time practicing and repeating various memory
  tasks.
• Control (n 11) No treatment.

28
Berg et al., 1991

• Objective tests of memory for tasks expected to
  be affected (effect tasksmemory) and those not
  expected to be affected (control tasksreaction
  times).
• Questionnaire - Subjective reports by patient and
  close relative - memory questionnaire.
• Measured 3 weeks after initiation of
  intervention, at post-intervention, and 4 months
  follow-up.

29
Berg et al., 1991

• Effect tasks Strategy group performed
  significantly better than Pseudo and Control
  groups on delayed recall tasks at 3 weeks,
  post-intervention, and follow-up and on memory
  sum and acquisition tasks at post-intervention
  and follow-up.
• Control tasks Pseudo group performed
  significantly better than Strategy and Control
  groups on movement time task at post-intervention
• Questionnaire No differences between Strategy
  and Pseudo groups (control group not tested).

30
Berg et al., 1991
Strengths Weaknesses

• No baseline differences between groups on age,
  education, IQ, PTA, chronicity, and baseline
  memory and control tasks.

• Randomization method and concealment, and
  blinding not specified.

31
Findings - Trials with Follow-up

• Fasotti et al., 2000
• Tested effect of Time Pressure Management (TPM)
  on speed of information processing among 22
  patients with severe closed head injury.
• Minimum 3 months post-injury (mean for treatment
  group 9.8 months, control group 8.3 months).
• Average PTA for sample was about 2 months.

32
Fasotti et al., 2000

• Intervention intensity varied maximum 3
  sessions per week, 1 hour per session, over 2 to
  3 weeks (possible range of 2 to 9 hours total).
• TPM (n 12) presentation of nine videotaped
  short stories of two types
• Story topics likely to be encountered in daily
  life.
• Computer manipulation of a computer program.
• Control (n 10) Concentration training of
  verbal suggestions, 2 to 5 hours per week for 3
  to 4 weeks.

33
Fasotti et al., 2000

• Story task reproduction score - amount
  remembered.
• Computer task number of steps accomplished in
  manipulation of task.
• Behavioral observations scored for preventive
  steps used and time pressure management steps
  used to accomplish tasks.
• Measured at post-intervention and 6 month
  follow-up.

34
Fasotti et al., 2000

• Behavioral observations
• TPM group scores significantly higher than
  control on number of preventive steps taken to
  perform computer task at follow-up.
• TPM group scores significantly higher than
  control on number of time pressure managing steps
  for story and computer tasks at
  post-intervention, and on computer tasks at
  follow-up.
• Performance of tasks
• No significant difference between groups.

35
Fasotti et al., 2000
Strengths Weaknesses

• No baseline differences between groups on
  demographis or neurologic variables.
• Person performing all evaluations was blind to
  group allocation.
• Exclusions clearly defined.
• Used multivariate repeated measures analysis.

• Minimal treatment intensity.
• Equal amounts of stimulation to both groups.

36
Findings - Trials with Follow-up

• Salazar et al., 2000
• Compared the effect of a comprehensive,
  in-patient cognitive rehabilitation program with
  that of a home-administered program on overall
  function to enable return to work and fitness for
  duty among 120 patients in the military with
  severe TBI.
• All had sustained injury within 3 months of
  randomization (average about 1 month).
• All injuries severe, but all ambulated
  independently.

37
Salazar et al., 2000

• Intervention duration varied among patients.
• In-patient program (n 67) milieu-oriented
  program including daily physical, cognitive,
  speech, occupational, and coping skills training.
• Home program (n 53) included educational
  materials, training in cognitive exercises, daily
  physical exercise, and weekly 30-minute telephone
  calls from the psychiatric nurse.

38
Salazar et al., 2000

• Primary outcome measures Return to work and
  fitness for duty at 1 year post-treatment.
• Multidisciplinary tests and psychosocial outcomes
  (cognitive, psychiatric, and neurological
  outcomes, and quality of life) measured at 8
  weeks after randomization, and at 6, 12, and 24
  months.

39
Salazar et al., 2000

• No significant differences between groups in
• return to work (90 in-patient group, 94 home
  group)
• fitness for active duty
• quality of life
• verbal and visual memory or attention,
• general measures of cognitive or psychiatric
  function

40
Salazar et al., 2000

• Randomization clearly defined.
• Patient characteristics and refusal/attrition
  (32) clearly defined.
• No differences between refused or lost patients
  and retained patients on demographics, injury
  severity, and clinical status at entry.
• No differences between experimental groups on
  demographics, injury severity, and
  alcohol-related trauma.

• Blinding not specified.
• Method of analysis not specified.
• For in-hospital group, significantly fewer MVAs,
  more assaults, and fewer patients unconscious for
  gt 1 hour.

41
Findings - Trials without Follow-up

• Dirette et al., 1999
• Tested the effect of computer-aided internal
  compensatory strategy training on visual
  processing among 30 patients with mild, moderate,
  and severe TBI.
• Chronicity varied between 2 and 12 months
  (average 5 months)

42
Dirette et al., 1999

• Intervention provided 1 hour per week over 6
  weeks. First and last weeks were testing only,
  leaving a total of 4 hours of treatment in the
  intervention.
• 5 subjects each in 6 conditions of a 2 x 3
  design
• Treatment vs. control conditions
• TBI mild, moderate, severe
• Treatment group received computer-aided
  instruction in 3 internal compensatory
  strategies verbalizing, chunking, and pacing.
• Control group received four 45-minute weekly
  sessions in remedial computer activities.

43
Dirette et al., 1999

• Weekly visual processing measure
• PASAT (Paced Auditory Serial Addition Task)
• Matching Accuracy Test segments of The Brain
  Game program.
• Pre/Post measures
• Speed and accuracy on two data entry tasks and a
  computerized reading program.
• Post-test 1 week after final intervention session.

44
Dirette et al., 1999

• No significant differences between groups.
• No main effects or interactions among 3 levels of
  severity.
• Regardless of group assignment (or severity of
  injury) 80 of participants used compensatory
  strategies with or without specific training in
  their use.
• Those using internal compensatory strategies did
  better on performance speed and were less erratic
  in performance accuracy.

45
Dirette et al., 1999

• Data collectors blind to subjects group
  allocation.

• Randomization method/concealment not specified.
• Baseline differences not specified.
• Insufficient follow-up time.
• Minimal intervention intensity.

46
Findings - Trials without Follow-up

• Eakman et al., 2001
• Compared effect of hands-on occupational therapy
  with verbal instruction on performance in
  preparing meatballs among 30 patients with TBI
• Mean Rancho 7.2
• Mean Weschler Memory Scale summed score 25.1
• Average chronicity 53.5 months.

47
Eakman et al., 2001

• One session intervention and testing.
• Treatment group received training in meatball
  preparation with 10-step note-card instructions,
  each followed by hands-on task manipulation.
• Control group received only note-card
  instructions.

48
Eakman et al., 2001

• Measure was verbal report in how to prepare
  meatballs. Best total score 38 points.
• Hands-on group mean score 11.8
• 
  median 11
• Instructions-only group mean score 2.3
• 
  median 2
• Hands-on group scored significantly higher than
  instructions-only group.

49
Eakman et al., 2001

• Person scoring was blind to group allocation.
• No baseline differences on age, chronicity,
  Rancho level, and baseline Weschler Memory Scale.

• Randomization method/concealment not specified.
• No follow-up.
• Minimal intervention intensity.

50
Findings - Trials without Follow-up

• Levine et al., 2000
• Compared effect of Goal Management Training (GMT)
  with Motor Skills Training (MST) on disorganized
  behavior among 30 TBI patients.
• 24 - GOS Good Recovery
• 6 - GOS Moderate Recovery
• All 3 to 4 years post injury

51
Levine et al., 2000

• Intervention was one 1-hour session.
• GMT group received verbal definition of goal
  management, concrete examples, and illustrative
  activities. Final activity (setting up an
  answering machine) was partitioned into sub-goals
  and performed by each participant.
• MST group received training in reading and
  tracing mirror-reversed text and designs.

52
Levine et al., 2000

• Three clinic-specific measures administered
  before and after the training session
• Proofreading
• Grouping
• Room Layout
• Neuropsychological tests were administered before
  the training session
• Stroop
• Trails A and B
• WAIS-R Digit Symbol Subtest
• Tested immediately after trainingno follow-up.

53
Levine et al., 2000

• No significant differences between groups.
• Authors reported some within-group differences
  between pre- and post-training.

54
Levine et al., 2000

• Randomization method/concealment, and blinding
  not specified.
• GMT group significantly slower than MST group on
  the Stroop test at pre-training.
• No follow-up measures.
• Minimal intervention intensity.

55
Findings - Evidence for Effectiveness

• Comparative Studies
• Grealy et al., 1999
• Parente et al., 1999

56
Findings - Comparison Studies

• Grealy et al., 1999
• Brain injury rehabilitation unit in Edinburgh,
  Scotland.
• Tested the effect of Virtual Reality Exercise on
  cognitive function
• Chronicity ranged from 1.7 to 178.6 weeks.
• Screened out patients unable to perform Digit
  Span, to carry out simple instructions, or who
  had insufficient language skills to be able to
  express verbal learning.

57
Grealy et al., 1999

• Treatment group patients (n 13) were
  consecutive admissions who volunteered and who
  met criteria.
• Control group data collected from database of 320
  patients admitted to same hospital in previous 2
  years.
• Matched each treatment group patient to 25
  control group patients on age, severity, and
  chronicity.

58
Grealy et al., 1999

• Virtual Reality group participants rode a bicycle
  while viewing virtual environment. Steered a
  course or participated in a race.
• Exercise bouts varied from 13 to 18 sessions.
• Control group data obtained from retrospective
  chart review.

59
Grealy et al., 1999

• Attention and Information Processing
• Digit Symbol
• Trails A and B
• Learning and Memory Functions
• Auditory Verbal Learning
• Visual Learning
• Logical Memory
• Complex Figure

60
Grealy et al., 1999

• Analysis
• Mean scores calculated for each participants own
  group of 25 control patients.
• Scores of each treatment group patient expressed
  in standard deviations from his/her own control
  group mean.
• Compared each patients performance relative to
  control group means before and after intervention.

61
Grealy et al., 1999

• Results
• Significant improvement on Digit Symbol.
• Significant improvement on Verbal and Visual
  Learning for trials 1 to 5 and the delayed trial,
  but not for the interrupted trial.
• No improvement on Logical Memory or Complex
  Figure.

62
Grealy et al., 1999

• Pre-intervention assessor blind to nature of
  study.
• Innovative method for obtaining control group
  data.

• Insufficient follow-up period.
• Treatments for control group patients not
  specified.

63
Findings - Comparison Studies

• Parente et al., 1999
• In- and out-patients of Maryland Rehabilitation
  Center.
• Tested effect of Group Cognitive Skills Training
  on employment.
• Patients were screened prior to referral for
  functional limitations and employment potential.

64
Parente et al., 1999

• Treatment group patients (n 33) were recruited
  from referrals to Division of Rehabilitation
  Services.
• Control group data from 64 patients comparable to
  treatment group, selected from database of 568
  patients with TBI.

65
Parente et al., 1999

• Treatment group received cognitive skills
  training, computer training, prosthetic aid
  training, interviewing skills training. Employs
  clients teaching clients.
• Duration of intervention ranged from 2 months to
  1 year average 4 months.
• Control group data obtained from chart review of
  clients in various training programs during same
  year.

66
Parente et al., 1999

• Measures
• Return to Work
• Job Longevity
• Training Success

67
Parente et al., 1999

• Results
• Return to Work
• At 1 year, 13 clients completed group services
  training. Ten of 13 were working at the time of
  publication (76 employment rate).
• Employment rate for control group during same
  time period was 58.
• No comparison data for Job Longevity and Training
  Success

68
Parente et al., 1999

• Real-life setting and patient-relevant outcomes.
• Control group data obtained from patients being
  treated during same time period.
• Clients in both groups received equal amounts of
  services other than the intervention.
• No baseline differences in demographics.
• Attrition (1 patient) clearly specified.

• Chronicity and severity not clearly specified.
• Statistical significance not specified.
• Evaluation performed by program
  innovators/directors.

69
Findings Meta-analysis

• Two meta-analyses
• (Park Ingles, 2001)
• Loya, 1999

70
Findings Meta-analyses

• Park Ingles (2001) analyzed results of 30
  studies on the effectiveness of attention
  training.
• Their overall finding
• When evaluated by before-after scores, there is
  significant improvement from attention training.
• When evaluated with a control group, the
  treatment group is not significantly different
  than controls.
• However, this meta-analysis also included non-TBI
  participants, so we will not report it fully here.

71
Findings Meta-analyses

• Loya (1999) analyzed results of studies on the
  effectiveness of memory training and memory
  rehabilitation.
• Searched for studies of adolescents to
  middle-aged adults from 1970 to 1999.
• 117 articles found and screened to find 14
  studies for use in the analysis.
• 7 studies were two-group experiments, with
  control groups.
• 7 were one-group studies using change scores

72
Meta-analysis Loya, 1999

• Results (corrected for small sample sizes)
• 2-group
• Grand mean weighted effect size, d .47, k 30
• 1-group
• Grand mean weighted effect size, d .61, k 61
• By Cohens (1988) criterion, this indicates
  treatment effectiveness in the low to high-medium
  range. (Cutoff for medium range d .50)
• Both show statistically significant improvement.

73
Meta-analysis Loya, 1999

• An unusually great homogeneity of effect sizes
  was noted over the different treatments
• Noted limits to generalization of results from
• No control of patient characteristics
• Unreported patient characteristics
• Inconsistent report of statistics used
• Low statistical power (low N in studies)
• Use of unreliable measures
• Confounding of multiple treatments
• Interaction of multi-treatments patient
  selection

74
Meta-analysis Loya, 1999

• Conclusions
• There is an overall significant, positive effect
  of memory training.
• Great homogeneity of effect sizes implies a
  strong, underlying common factor among the
  treatments.
• Inadequacies of present research leave open the
  question of whether
• Treatment of a specified cognitive deficit can be
  isolated, or
• Cognitive rehabilitation is essentially a
  holistic endeavor.

75
Recommendations
76
Efficacy of Cognitive RehabilitationFuture
Research

• Independent Variables - Operational Definitions
• Define overall scope of cognitive rehabilitation
• What are the subcomponents of CR?
• To what extent can cognitive rehabilitation be
  divided into parts for the purpose of evaluating
  the relative effectiveness of its components
  within specific patient groups before division
  into parts renders the evaluation meaningless?

77
Efficacy of Cognitive RehabilitationFuture
Research

• Dependent Variables
• Continuing problem of validity of outcome
  measures.
• Define outcomes that are relevant to patients,
  families, insurance companies.
• Is it possible that the absence of treatment
  effect in some studies is partly a function of
  the lack of relevance in the lives of the people
  being evaluated?

78
Efficacy of Cognitive RehabilitationFuture
Research

• Subject Variables
• With large samples, manage variability arising
  from individual differences with multivariate
  analysis.
• For smaller samples, limit the range of severity,
  age, chronicity, social factors, and other
  sources of individual differences so the study is
  very specific in testing treatments with
  particular categories or types of patients.

79
Efficacy of Cognitive RehabilitationFuture
Research

• Confounding Variables
• Measurement of spontaneous recovery and the
  general therapeutic effects of stimulation.
• In the studies reviewed, treatment effects were
  diminished or not observed when two kinds of
  treatment were compared to each other, with
  levels of stimulation equated for both groups.

80
Efficacy of Cognitive RehabilitationFuture
Research

• The absence of evidence is not evidence of
  absence.
• Not proven is not proven not.