Everyday Evidence: Using Research And Stakeholder Input To Inform Use Of High-Tech Memory Aids

1
WS16 AOTA 84th Annual Conference
Expo Minneapolis, MN May 21, 2004

• Everyday Evidence Using Research And Stakeholder
  Input To Inform Use Of High-Tech Memory Aids

Mary Vining Radomski, MA, OTR/L Susan Newman,
OTR/L Matt White, OTR/L Elin Schold Davis,
OTR/L, CDRS
2
Acknowledgement -

• Funding made possible through the generous
  contributions of donors to the Sister Kenny
  Foundation.

3
Goals of the session

• Describe a method of using best available
  evidence along with consumer and therapist
  perspectives to answer a clinical question
• Share user impressions of PDAs as memory aids
• Propose a learning schema aimed at compensation
  for memory problems

4
Overview of the session

• Identify the research questions
• Provide an overview of the existing literature
• Describe our setting
• Summarize the methods
• Introduce the memory aids used in the study
• BREAK 1050
• Impressions of focus groups
• Proposed OT Process
• Our take-aways

5
Evidence-based practice

• the conscientious, explicit and judicious use
  of current best evidence for making decisions
  about the care of the individual patient. It
  means integrating individual clinical expertise
  with the best available external clinical
  evidence from systematic research.
• Sackett, Richardson, Rosenberg, Haynes (1997)

6
Best practice decisions based on

• Scientific evidence
• Clinical expertise
• Patient preference
• Lee Miller (2003)
• Sackett, Richardson, Rosenberg, Haynes (1997)

7
Context Brain Injury Clinic Sister Kenny
Rehabilitation Institute

• 40 people each year who experience changes in
  memory, concentration, problem solving as a
  result of an acquired brain injury
• Patients participate in occupational therapy to
  learn ways of compensating for often permanent
  cognitive changes.
• They typically participate in 3 to 20 outpatient
  occupational therapy sessions during which they
  learn skills, habits, and routines that center on
  use of a day planner as a cognitive prosthesis

8
Context

• Clients were asking about PDAs
• Various people (physicians, friends, family) were
  recommending PDAs or telling them their
  personal success stories of using a PDA.

9
Our overarching clinical concerns

• When is a high-tech memory aid appropriate for
  cognitively impaired clients?
• What type of device is best?

10
Scientific Evidence
11
Review of Literature

• 2 approaches to addresses cognitive impairments
  restoration of deficits vs. compensation for
  deficits
• Compensation for deficits holds greatest promise
  for improving everyday functioning (Carney et
  al., 1999 Cicerone et al., 2000)

12
Compensation for deficits occurs
through routine use of an external
memory system

• Low tech external memory systems day planners,
  diaries
• High tech external memory systems personal
  digital assistants (PDAs)
• electronic organizers
• electronic pagers

13
Compensation seems to work -
evidence from small pre-post studies
and single case studies

• Persons with ABI who used planners/diaries
  experienced fewer memory failures
  (Schmitter-Edgecombe et al., 1995) and improved
  ability to remember appointments and to dos
  (Zenicus et al., 1990 1991)

14
Compensation seems to work -
evidence from small pre-post studies
and single case studies

• So did persons using PDAs (Giles Shore, 1989
  Kim et al., 1999 Kim et al., 2000 Van Den Broek
  et al., 2000)

15
Most patients use low-tech memory aids

• In a study of almost 100 people with brain injury
  related memory impairment, Evans and colleagues
  (2003) found that most subjects used calendars,
  wall charts, notebooks and only 7.4 used an
  electronic device as memory aid

16

• Evans and colleagues suggested that electronic
  aids may be under-used because clinicians
  themselves do not have a good working knowledge
  of these devices.

17
Problem

• Many clients are motivated to use a
  pocket-computer memory aid (Wright et al., 2001),
  clinicians seem to lack confidence in their
  ability to guide clients in the use of these
  devices (Hart et al., 2003).

18
Evans and colleagues (2003) found that the
following variables predicted use of memory aids

• Age younger more likely to use
• Time since injury the longer since injury, less
  memory aid use
• Number of aids used premorbidly better if more
  premorbid use
• Attention measure of attention and speed of
  information processing proved to have strong
  relationship with use

19
Problem

• There is little guidance from the literature to
  help clinicians match client characteristics with
  memory aid features

20
Research Question

• What compensatory memory tools are preferred by
  occupational therapists and potential users with
  acquired brain injury as means to remember
  appointments, assigned tasks, and time-specific
  actions based on a systematic analysis of
  features, learning time, demand characteristics
  (cognitive, physical, sensory) and user
  satisfaction?

21
Acquiring clinical expertise

• METHODS

22
In order to begin to match the right technology
to specific patients needs and characteristics,
we wanted to

• 1) Acquaint ourselves with the types of devices
  currently available
• 2) Acquire personal expertise in their use
• 3) Obtain impressions of potential users
• 4) Come to some consensus regarding how/when to
  integrate this technology into traditional
  cognitive rehabilitation therapies.

23
Design

• Descriptive study integrating qualitative and
  quantitative methods
• Approved by hospital Institutional Review Board

24
2 Sets of Participants -

• 4 occupational therapists
• All serve outpatients at SKRI
• 3 of 4 work exclusively in cognitive
  rehabilitation-adjustment program
• 16 persons with acquired brain injury recruited
  from local support groups

25
Inclusion Criteria

• Ages 18 - 65
• No motor or communication impairments that
  precluded use of memory aids
• Absence of learning disability or concurrent
  psychiatric illness
• At least 6 months post onset of ABI
• Evidence of mild to moderate memory problems
  (RBMT 10 21)

26
Participants with ABI
Sex 8 men, 8 women
Age 24 62 years (mean 49.4)
RBMT 14 - 20
Time since onset 1 year 32 years (median 4 years)
Nature of ABI 8 - TBI 2 - CVA 3 S/P surgery for brain tumor or cyst 2 Aneurysm 1- Vasculitis
27
Memory aids

• 4 devices were identified for the study
• As representative of
• high tech vs low tech
• simple and more complex.

28
Low Tech Simpler

• Week-at-a-glance Calendar
• www.daytimer.com
• IQ Voice Organizer
• http//www.vpti.com/

29
Higher Tech Specialized Software

• HP Jornada
• With PEAT Software
• Visor
• PDA
• www.hp.com

30
Part 1 Therapists as users Procedure
for each device

• 1) Read manual and create cheat-sheet for how to
  input appointments, input to-dos, set alarms
• 2) Learn to perform 3 functions (defined as
  performance after a 15 minute interference task)
• 3) Use it for a 3-day trial
• 4) Complete adapted version of Quebec User
  Evaluation of Satisfaction with assistive
  Technology (QUEST) to capture immediate
  impressions (Demers et al., 1996)
• 5) Rate cognitive demands of device (LoPresti
  Willkomm, 1997)

31
QUEST example

• Instructions For each of the 14 items, tell me
  the level of satisfaction you experienced with
  your memory tool during the 3-day trial by using
  the following 1 to 5 scale
• 1 Not satisfied at all
• 2 Not very satisfied
• 3 More or less satisfied
• 4 Quite satisfied
• 5 Very satisfied

32
Examples from QUEST

• Satisfaction variables Degree of
  satisfaction
• USEFULNESS
• Degree to which the tool is practical and
  helpful in performing activities in various
  situations and environments 1 2 3
  4 5
• ADJUSTMENTS
• Degree of ease in setting and adjusting the
  components of the tool
• 1 2 3 4 5
• TRAINING
• Degree of skill and experience required before
  being able to use the tool
• 1 2 3 4 5

33
Degree of Cognitive Abilities Required (LoPresti
Willkomm, 1997)

• L Low M Moderate High N Not
  necessary
• P Necessary only for programming
• item added for this project
• Ability to organize and plan
• Ability to problem solve
• Ability to learn and remember
• Ability to be attentive and concentrate
• Ability to see
• Ability to hear
• Ability to feel and touch buttons
• Ability to perform coordinated, fine motor
  activity
• (list continued on next slide)

34
Degree of Cognitive Abilities Required (LoPresti
Willkomm, 1997)

• L Low M Moderate High N Not
  necessary
• P Necessary only for programming
• item added for this project
• Ability to comprehend orally
• Ability to visually read and comprehend
• Ability to process the speed of spoke language
• Ability to speak
• Ability to write
• Accuracy ability required
• Flexibility required
• Ability to self initiate
• Ability to input and respond at an appropriate
  speed

35
Part 1 Therapists as users

• Therapists participated in a 2-hour
  self-lead focus group to discuss impressions
• Discussion was audiotaped, transcribed, and
  analyzed by each therapist for themes.

36
Part Two Persons with ABI as users

• Volunteers were randomly assigned in blocks to 1
  of 4 memory aids
• Received 1 on 1 instruction to criteria
• Participated in 3-day home trial
• Completed QUEST via telephone within 1 day of
  3-day trial
• Participated in focus group (semi-structured)

37
Volunteers per Memory Aid

• Week at a glance planner 3 volunteers
• IQ Voice Organizer 4
• HP Jornada with PEAT software 4
• Handspring Visor - 5

38
Data sources

• Log of learning time for both sets of
  participants
• Log of therapist observations and reflections as
  learners and teachers
• Responses to QUEST and cognitive requirements
  checklist (therapists only)
• Focus group transcripts

39
Analyzing the data

• Attempted to analyze questionnaire data
• Each therapist analyzed the focus group
  transcripts, individually and then as a group.
• To organize the themes we
• assigned codes for key themes
• met as a group to discuss and establish consensus
  around key themes

40
Analyzing the data

• Based on this discussion, we identified/assigned
  key tasks
• 1) create a protocol that addresses key learning
  needs and sequence
• 2) come up with agreement to lend policy and
  procedure
• 3) create a features grid for tools
• 4) create a plan for how we will maintain our
  knowledge of new tools.

41
RESULTS

• What We Learned

42
Acquiring Clinical Expertise and Defining
Patient Preference

• Device Demonstration
• and Impressions

43
Week at a Glance Calendar

• Features
• Inexpensive
• No batteries
• Need to self-initiate
• Need legible penmanship

44
IQ Voice Organizer

• Features
• Clock
• Alarm
• Memo
• Calendar
• Contacts
• Back up with PC

45
Visor Palm Pilot

• Features
• Calendar
• Notes
• To Do List
• Contacts
• Information backed up on computer

46
Pocket PC with PEAT Software

• Features
• Calendar
• Notes
• Contacts
• To Do List
• Information backed up on computer
• Large Text
• Multiple sounds for different alarms

47
Timex Data Link Watch

• Features
• Clock/Alarm
• Contacts
• Appointment/ Schedule
• Countdown timers
• Information backed up on the PC
• Stop watch

48
Time Pad

• Features
• Calendar
• Voice alarm (message up to 72 seconds in
  length)
• Clock

49
Invisible Clock

• Features
• Calendar
• Clock
• Alarm (vibrating)
• Worn like a pager
• Stopwatch

50
MotivAider

• Features
• Vibrating alarm
• User sets interval cycle
• Simple to use
• Can be worn like a pager

51
Watch Minder

• Features
• Calendar
• Vibrating alarm with text
• Up to 16 alarms
• List of universal messages
• Can be worn as watch

52

• BREAK

53

• Findings

54
Clinical expertise and patient
preference Questionnaire data

• Questionnaire data primarily used to enable
  participants to keep track of impressions for
  later discussion
• Small and unequal numbers of participants in
  groups prevented rigorous quantitative analysis

55

• Learning time
• Therapists - Ranged from 5 minutes - 4 hours
  (with first tech-device typically taking the most
  time)
• Persons with ABI Ranged from 60 minutes to 6
  hours
• Overall satisfaction of persons with ABI
• Created an overall satisfaction variable no
  statistically significant difference based on
  type of device

56
Clinical expertise and patient
preference Focus groups

• Device preferences not unlike individual
  differences related to selection of
• SHOES
• Does it Fit?
• Does it make a Statement?
• Is it FUN or Functional?

57
Variables influencing user acceptance
Does it Fit?

• Both therapists and persons with ABI indicated
  that their experience with a given device was
  influenced by their 1) past experience with
  technology and 2) self-conceptions as a high or
  low tech person.

58
Therapist users

• Im a high tech person and I like gizmos and I
  like the alarms. And I think I benefit from
  that.
• Well, my biases are just the opposite. Im a low
  tech person. I like quick access to stuff. I like
  to be able to see something literally visually
  very quickly, often right next to each other

59
Users with ABI

• I found that I didnt particularly care for it
  the planner because I dont like to write
  things down. Its not something I do naturally
• A bias, I guess, I had before learning to use it
  Jornada HP with PEAT was, personally, I had
  been thinking about using one anyhowso I was all
  gung-ho to want it to work

60
Variables influencing user acceptance

• Does it make a Statement about who I am?
  (The Cool factor)
• Therapist So, I think that the appearance
  Visor, its very mainstream. If I whip that out
  at a meeting, no ones going to look at me and
  think, what in the world has he got there?

61

Variables influencing user acceptance

• Does it make a Statement about who I am?
  (The Cool factor)
• User with ABI I had no previous experience with
  it Visor except that my oldest son has been
  using one for 4 years to organize his whole life
  I really like it.I want one for my birthday

62
Variables influencing user acceptance

• Is it FUNctional or functional?
• Therapists tended to evaluate devices based on
  their experience or anticipations of features and
  effectiveness
• Persons with ABI indicated that enjoyment of the
  device was important.

63
User with ABI

• The biggest thing for me was the fun factor. Id
  pretty much convinced myself that my memory was
  just fine and that I didnt need any kind of aid
  at all. Its not true but with that attitude in
  mind, it was difficult for me to remember to pull
  out this planner and write something down.but if
  I had a computer-deal, it would be more fun. Id
  want to do it and it would be something Id want
  to do just for the enjoyment of it

64
Impressions of persons with ABI

• Persons assigned to learn to use technology did
  not think that they couldve done so without
  assistance from therapist
• They needed more time to acclimate to device than
  allowed for in study

65
Impressions of therapists

• Peoples preferences for or impressions of high
  tech devices often are not linked to their
  proficiency or the devices effectiveness.

66
Towards changes in our practice

• Evaluation
• Intervention Planning and Outcomes Expectations
• Intervention

67
Changes to Practice Evaluation

• Self-perceptions as a high tech or low tech
  person shape the learning process.
• We realize that we need to specifically assess
  preferences and history with technology at the
  beginning of our treatment.

68
As part of evaluation processes

• Ascertain clients hopes for therapy
• Obtain self-report of cognitive concerns and
  problem areas
• Ask about past (premorbid) use of information
  management strategies
• Assess self-awareness of deficits
• Ascertain consequences of clients cognitive
  failures/inefficiencies
• Assess adequacy of fine motor and vision skills
  (for use of PDA)
• Determine specific technology preferences

69
Changes to Practice Intervention Planning and
Outcomes Expectations

• Differentiate between
• Avocational Users
• and
• Therapeutic Memory
• Aid Users

70
Avocational user Shorter
episode of care

• Interest in PDA drives interest in intervention
  (more than concerns about cognitive performance)
• Satisfied with reliance on other memory supports
  (spouse)
• Adequate financial resources for purchase of
  technology
• Performance of critical roles not in jeopardy

71
Therapeutic Memory Aid User
Longer episode of care

• Performance of critical roles in jeopardy
• Desire to improve function drives interest in
  intervention
• Enough awareness of cognitive problems to inspire
  participation in therapy

72
Changes to Practice Intervention Planning and
Outcomes Expectations

• 2 Types of Therapeutic Memory Aid Users
• Responder (passive user)
• Initiator (active user)

73
Memory Aid Responder

• Has SO who is committed to the role of inputting
  information for client
• Difficulties with information entry
• Decreased awareness/initiation
• Progressive cognitive disorder
• Profound memory problems

74
Changes to Practice Intervention

• There is a distinction between learning the
  cognitive sub-skills necessary for memory
  compensation and mechanics sub-skills (learning
  to use a specific gadget).
• In general, we need to teach clients the
  cognitive sub-skill before the mechanics
  sub-skill.

75
Cognitive vs. mechanics sub-skills

• Examples of cognitive sub-skills
• Differentiating between an appointment, to-do,
  project
• Understanding that information is stored
  differently based on its type

• Examples of mechanics sub-skills
• Locating and inputting information in
  function-specific components of memory aid

76
Cognitive vs. Mechanics Sub-skills

• Examples of cognitive sub-skills
• Attentiveness to detail to ensure accuracy of
  information entry
• Estimating how much time is needed for to dos
  with reasonable precision

• Examples of mechanics sub-skills
• Following procedure to inputting information into
  device
• Locating and following a checklist for daily
  planning

77

• SEE HANDOUT
• COMPARISON OF DEVICES (GRID)

• For successful memory compensation, there must be
  a fit between user characteristics and
  technology features/characteristics.
• We drafted a features grid of the devices in the
  study.

78
Take-away from the project

• Technology is constantly changing and in order to
  continually find the best tools for patients, we
  need a systematic way of staying current.

79
Take-away from the project

• Try the devices you recommend to patients
• Open yourself to technology as a treatment tool
• Using technology devices provides valuable
  insight for teaching your client
• Learn specific device features to facilitate
  appropriate recommendations to clients
• Expanding your treatment options enhances the
  quality of service you provide.

80
Take-away from the project

• Be MINDFUL of the seductiveness of technology
• Technology graveyard proportional to income and
  access to funds
• What works for one may not work for another My
  doctor uses a PDA and recommends I get one
• this XYZ will fix the problem
• Focus on finding the right gadget can sidetrack
  all from focus on skill sets

81
Take-away from project

• Take advantage of existing philanthropic
  resources in your setting and community for
  research and technology.

82
Questions and Answers

• Thank you
• And enjoy Minneapolis ?