Joseph Shuluk, BA Jeannine Melly, MPH

1
http//socialworkleadership.org
Presentation Prepared for theState Society on
Aging of New York Conference October 15-17,
2009 Rochester, New York Building an
Evidence-Based Practice Database Care
Coordination for Older Adults

• Joseph Shuluk, BA Jeannine Melly, MPH
• Edward Mullen, DSW Patricia J. Volland, MSW,
  MBA

2
The New York Academy of Medicines Social Work
Leadership Institute
The New York Academy of Medicine (NYAM) The New
York Academy of Medicine is an independent,
non-partisan, non-profit institution that has
been advancing the health of people in cities
since 1847.
The Social Work Leadership Institute (SWLI) A
national program that supports healthy aging by
ensuring America's older adults receive the care
they need to stay independent and in charge that
of their lives.
3
SWLIsEvidence-Based Database
? Developed by an interdisciplinary team of
professionals based at the New York Academy of
Medicines Social Work Leadership Institute ?
Designed to provide access to available evidence
on social work intervention outcomes
particularly as related to serving the aging
population ? Its targeted user audiences are
public policy makers, academics, researchers,
students, clinicians, advocacy groups ? This
work is funded by a generous grant to NYAM-SWLI
from the Atlantic Philanthropies
4
SWLIs Evidence-Based Database

• GOALS
• Assist in translating research into policies and
  procedures
• Expand publicly accessible evidence-based
  databases
• Assess the overall strength of a body of evidence
  
• Research available for White papers
• Research available for Synthesis studies

TOPICS While the database focus is specific to
interventions with older adults we considered it
important to first examine what is known about
the general effectiveness of social work
interventions The second topic populating the
database, care coordination, is specific to
interventions with older adults
5
A Blueprint for Building the Database

• Step 1 Develop a topic question
• Step 2 Construct search strategies
• Step 3 Select databases and perform searches
• Step 4 Manage and filter the search results
• Step 5 Assess the study designs
• Step 6 Code the studies
• Step 7 Review and record the studies
• Step 8 Disseminate the results
• Step 9 Evaluate the work

6
Building the Database Step 1 Develop a Topic
Question

• A topic based on policy questions was identified
  with the assistance of the Advisory Board
• A narrative was developed to frame the topic
• The topic narrative was refined to formulate a
  concise investigative question
• Current SLWI Topics
• Topic 1 What is known about the general
  effectiveness of social work interventions? Nine
  review articles, along with their 308 primary
  studies, were identified by the Advisory
  Committee for inclusion.
• Topic 2 What are the outcomes and costs of care
  coordination (care management, transition-in-care,
  discharge planning) for older adults?

7
Building the Database Step 2 Construct a
Search Strategy

• Developed a search strategy by
• breaking down the investigative question into
  keywords
• defining population, intervention, comparisons,
  and outcomes (PICO)
• defining search limits, such as date of
  publication,language, or study design
• Engaged a librarian to help conduct the search

8
Building the Database Step 3 Select Databases
and Perform Searches

• Identified relevant databases to capture the
  evidence
• Created search strings using the keywords that
  were developed along with using synonyms
• Adjusted search strings for each database
• Recorded the search strategy, the search strings,
  date of search, and number of results, for each
  database to provide transparency and replication
• Designed feeds to retrieve new articles as they
  are published
• Strived for sensitive and efficient searches

9
(No Transcript)
10
Building the Database Step 4 Manage and Filter
the Search Results

• Loaded the search results into the reference
  program RefWorks and created a spreadsheet in
  Excel
• Eliminated duplicate results from the searches
• Reviewed all of the abstracts for relevancy and
  excluded those that fell outside of the topic
• Recorded the origin (database) and status
  (included or excluded) of each study
• Labeled the citations with unique numbers to
  assist tracking
• Created folders with author name, labels, id
  numbers etc. to maintain a paper record

11
Building the Database Step 5 Assess the Study
Designs

• Constructed pre-screening questions that were
  relevant to the topic to aid in identifying
  studies for inclusion or exclusion
• Used an on-line system to channel the various
  research designs by constructing 3 review forms

• Experimental
• Non-experimental
• Synthesis Review

12
Building the Database Step 6 Code the Studies

• Reviewers needed a strong background in
  methodology
• Developed training for consistency
• Used senior reviewers to oversee training
• Used style guidelines (APA)
• Sought skills in extracting information to fit
  into predefined categories
• Developed a glossary
• Developed a manual
• Reminded reviewers not to interpret, but report
• Used 2 reviewers to code and a 3rd reviewer to
  assist with resolving discrepancies

13
Building the Database Step 6 Code the Studies
Variables captured in each review

• Research Methods
• -Research design and type
• -Methods of assignment
• -Country where sample studied
• -Service settings
• Standardized Assessment Tools
• Dominant topics of study
• Sampling method
• Sample size
• Participant size and flow
• Demographics
• -Age
• -Gender
• -Racial/Ethnic group
• Recruitment Strategy
• Outcomes
• -Primary outcomes
• -Program Outcomes
• Interventions

14
Building the Database Step 7 Review and Record
the Studies
The review forms captured the information
necessary to make informed assessments of the
relevance, quality, and outcomes of the
interventions reported and to allow the user to
assess the internal and external validity

• Sample Characteristics
• Participant Flow
• Research Methods
• Randomization
• Follow-up periods
• Study limitations

15
Building the Database Step 7 Review and Record
the Studies
Post finished review online
16
Building the Database Step 8 Disseminate the
Results

• Free evidence-based database
• Publicized the website
• -Marketing
• -Emails
• -Newsletters
• Collaborated with outside agencies
• Seminars, Conferences

17
Building the Database Step 9 Evaluate the work

• SWLIs evaluation
• Initial website launched in 2005
• Evaluated in 2006
• Redesign completed in 2008, using the Advisory
  Committee and outside reviewers
• Re-evaluated in 2009
• 10 interviews
• Survey-152 completes
• Web statistics

18
Population of SWLIs EBD

• Currently available on SWLI EBD website
• Social Work Effectiveness
• 2103 studies retrieved from searches
• 19 systematic reviews identified
• 317 abstracts posted
• 115 reviews completed and posted
• Care Coordination
• 4767 studies retrieved from searches
• 986 abstracts posted
• 145 reviews completed and posted

19
Supplementary Features

• ? Links to NYAMs Grey Literature collection
• ? Links to web sites contributing to
  evidence-based practices that are relevant to
  aging and social work outcomes
• Resources from conferences and proceedings
• ? Basic or advanced search options
• ? Print and Export functions
• ? A glossary of key terminology

20
Features Under Development Mapping

• Adds the ability to investigate the distribution
  (scope) of current available evidence under the
  topic in the database
• ? Facilitates information retrieval by user
  selected content
• Produces frequency of user selected content
• Produces cross-tabulation tables bases on user
  selected comparisons
• Produces condensed review and/or citation list
• Produce reports with the features mentioned based
  upon captured content such as research design,
  outcomes, service settings, demographics, sample
  size/method, recruitment, follow-up, participant
  flow, effect size.

Adopted from concept developed by the Evidence
for Policy and Practice Information and
Co-ordinating Centre (EPPI-Centre). (Bates, 2007
Peersman, 1996 Oakley, 2005)
21
Features Under Development GRADE-Quality/Strength
The assessment of the quality of a body of
evidence through the GRADE approach This system
awards a grade of high, moderate, low or
very low quality for each outcome, taking into
account ? Limitations in design
implementation of available studies (i.e. risk
of bias) ? Indirectness of evidence (indirect
population, intervention, control, outcomes)
? Unexplained heterogeneity (inconsistency) of
results (including problems with subgroup
analyses) ? Imprecision of results (wide
confidence intervals) ? High probability of
reporting bias
Standardized assessment software (GRADE Pro
available from Cochrane) The GRADE Working Group
http//www.gradeworkinggroup.org
22
2009 EBD SWLI Group

• SWLI Group
• Patricia Volland, MSW, MBA, Director of SWLI
• Jeannine Melly, MPH, Project Director
• Edward Mullen, DSW, Chair, Advisory Committee
• Joseph Shuluk, BA, Project Coordinator and
  Reviewer
• Elizabeth Taylor, MLS, Reference Librarian and
  Reviewer
• Lin Fang, PhD, Reviewer
• Jennifer Manuel, PhD, Reviewer
• Colleen McGinn, MSSW, Reviewer  
• Carin Tinney, LMSW, Reviewer  
• Jarmin Yeh, MSSW, MPH, Reviewer

http//socialworkleadership.org
23
2009 EBD Advisory Committee

• Standing Advisory Committee
• Robert P. Connolly, MSW, Centers for Medicare
  Medicaid Services
• Edward J. Mullen, DSW, (Chair) Columbia
  University
• Victoria M. Rizzo, PhD, Columbia University
• Aron Shlonsky, PhD, University of Toronto
• Haluk Soydan, PhD, University of Southern
  California
• Joan Zlotnik, PhD, Institute for Advancement of
  Social Work Research
• Founding Advisory Committee
• Candyce S. Berger, PhD, Stony Brook University
• Robert P. Connolly, MSW, Centers for Medicare
  Medicaid Services
• Kristin Day, LCSWM, VA Central Office
• Scott Geron, PhD, Boston University
• Julia Littell, PhD, Bryn Mawr College
• Edward J. Mullen, DSW, (Chair) Columbia
  University
• Enola Proctor, PhD, Washington University in St
  Louis
• Victoria M. Rizzo, PhD, Columbia University
• Aron Shlonsky, PhD, University of Toronto
• Haluk Soydan, PhD, University of Southern
  California
• Ronald Toseland, PhD, State University of New
  York at Albany