DISCUSSION

1
Developing a Psychometrically Sound Instrument of
Best Practices Specific to the PBRN Context
Victoria Neale, Kendra Schwartz, Monina
Bartoces, Jason Booza, Joseph Monsur, Marilyn
Wayland Wayne State University and Brenda
Hudson Indiana University

DISCUSSION
ABSTRACT
RESULTS

• Analytic Strategy
• 1) Essential Phase
• Content Validity Coefficients (CVCs)
  identified the items that a majority of 76 PBRN
  experts rated as essential research best
  practices.
• 12/59 items had negative CVCs and were
  eliminated 47 items were retained.
• 2) 360 Phase
• a) Cronbachs coefficient alpha computed by role
  of respondent
• Directors/Research Directors (n 33) a 0.89
• Staff/Research Assistants ( n 34) a 0.93
• Community-based Staff ( n 37) a 0.94
• All respondents combined (n 104) a 0.93
• b) Factor Analysis
• 31 items loading on 5 factors retained based on a
  priori criteria eigenvalue gt 1 factor
  loading gt 0.45 accounting for gt 5 of
  variance.
• Factor 1 Data Management (10 items) overall a
  0.88
• Director/Research Director a 0.65
• Staff/Research Assistants a 0.83
• Community-based staff a 0.89

• Objective To develop a self-assessment tool for
  PBRNs to evaluate and improve their research
  culture.
• Design Multi-step formative process to develop
  consensus on best practices.
• Setting Internet-based survey data collection.
• Participants 3 respondents per 42 PBRNs
  estimated the extent to which the PBRN followed
  59 research best practices.
• Outcome A psychometrically-sound
  self-assessment tool of research best practices
  specific to the PBRN context.

• Innovation and Significance
• To date, the national research enterprise has
  functioned in a decentralized fashion,
  resulting in duplicative or undocumented
  processes, without acknowledged best
  practices.
• This study used collective wisdom of
  experienced survey researchers and practice-
  based researchers to identify essential
  PBRN-specific research best practices.
• The final product will be a self-assessment
  checklist of best practices that PBRNs can
  use to assess their current research culture.
• The checklist has the potential to improve PBRN
  research culture by encouraging PBRNs to
  prioritize activities that strengthen the
  research professionalism of their
  organization.
• By promoting research professionalism, this
  project has the potential to contribute to
  greater public willingness to participate in
  research that can build the evidence- base
  of primary care.
• Next Steps
• 1) Develop Standard Operating Procedures (SOPs)
  for Each Best Practice
• 2007-2008 funding from CTSA Community
  Engagement Supplemental Award Laura Mae
  Baldwin is Co-Investigator Victoria Neale and
  Rowena Dolor are consultants.
• We will conduct a PBRN survey in early 2009 to
  a) determine routine procedures and SOPs
  currently used by PBRNs and b) ask PBRNs for
  insights into how SOPs could best be
  implemented in their setting.
• 2) Disseminate and Evaluate the Utility of the
  Best Practices and their SOPs
• Recruit PBRNs as test sites to implement and
  evaluate the SOPs.
• Evaluate utility of the best practices and SOP
  model for improving PBRN research
  professionalism.

DISCUSSION
RATIONALE
INTRODUCTION

• The practice-based research network (PBRN)
  movement has resulted in decentralized
  community-based research, which raises particular
  challenges to research integrity.
• While practice-based primary care research offers
  the advantage of greater external validity, it
  also presents challenges that include
  implementing standardized research protocols in
  decentralized settings training and monitoring
  of ethical research practices among the
  community-based clinic staff and data management
  and quality control issues.
• To date, there is no PBRN-specific infrastructure
  to promote the integrity of studies conducted in
  decentralized settings.

METHODS
Progress to Date 1) An extensive list of
research integrity best practices was developed
from the published literature and suggestions
from PBRN experts. The wording for these
PBRN-specific best practices was reviewed by 20
survey research experts recruited from the NAPCRG
Consultants Directory. 2) 59 research best
practices were organized into 7 domains (study
launch study orientation study supervision
data management analysis dissemination
professional training/development and PBRN
management). 3) Essential Phase 76 PBRN
experts rated the 59 items as essential or
not. 4) 360 Phase 3 respondents from 42
PBRNs evaluated the use of 59 research best
practices in their network. Respondents were in
the role of a) PBRN director/research
director b) network coordinator or experienced
research assistant and c) community-based
staff person.