Expert Committees

1
Expert Committees

• R. F. Shangraw, Jr., Ph.D.
• Chief Executive Officer
• Project Performance Corporation

Methods and Approaches in Screening Proprietary
Drug Names Public Meeting Renaissance Washington
DC Hotel June 26, 2003
2
Topics

• Framing the Problem
• Identifying Relevant Research
• Addressing the Key Questions
• Raising Two Other Concerns

3
Framing the Problem
Bound the Decision
Identify Alternatives
Evaluate Alternatives
Consensus Decision

• Using expert committees to review proposed
  proprietary drug names for potential sound alike
  and look alike confusion with existing drug names
• Using expert committees to review multifactoral
  studies on potential sound alike and look alike
  confusions
• Using expert committees to provide input to a
  single factor study

4
The Key Questions

• Is an expert committee necessary to review
  information from studies?
• How many people should staff an expert committee?
• What credentials are important for expert
  committee members?
• Should expert committees meet in person, via
  videoconference, teleconference, email?

5
Identifying Relevant Research

• Psychology
• Sociology
• Law
• Jury Deliberations
• Expert Witness
• Science Court
• Policy Sciences
• Forecasting
• Game Theory
• Organizational Behavior

• Medicine
• Appropriate and Necessary Care
• Group Decision-Making Methods
• Delphi Method
• Nominal Group Technique
• NIH Consensus Development Program
• RAND/UCLA Appropriateness Method

6
Necessity of Expert Committees Research

• Empirical research suggests group decisions are
  more consistent individual judgments are prone
  to personal bias (McDonnell et al, 1996)
• There is some empirical evidence that groups make
  quicker and better decisions (Blinder Morgan,
  2000 Hogarth, 1978)
• However, some empirical research on group
  decision making suggests a systematic bias in the
  outcomes (Janis Mann, 1977)
• But, a recent study found that groups are better
  than individuals in interpreting letters and
  number substitutions (Laughlin, et al., 2002)

7
Necessity of Expert Committees Practice

• Use an Expert Panel When
• Historical data does not exist or is limited
• Future events are likely to invalidate the
  results of historical analysis
• Ethical and moral factors are sufficiently
  important to the decision

8
Optimal Size of Expert Committees Research

• Assuming specific voting rules, committee size is
  negatively correlated with decision accuracy
  (Gabel Shipan, 2000)
• Communication quality and complexity is greater
  in small groups (n5) than in larger groups
  (n10) (Fay, et al., 2000)
• Limit to 12 15 members for effective
  functioning of the group (Shekelle, et al., 1999)

9
Optimal Size of Expert Committees Practice
Group Decision-Making Method Recommended Expert Committee Size
RAND/UCLA Appropriateness Method 7 15 Panelists
Nominal Group Technique 8 12 Participants
NIH Consensus Development Program 9 16 Panelists
10
Credentials for Committee Participation Research

• Need a baseline level of expertise but additional
  expertise does not improve accuracy (Armstrong,
  1980)
• Committees should include participation from a
  variety of relevant disciplines (Fitch et al.,
  2001)
• Participants status affects group dynamics
  (Murphy, et al., 1998)
• Participants must be justifiable as "expert" on
  the matter under discussion (Jones Hunter,
  1995)

11
Credentials for Committee Participation Practice

• Establish baseline qualifications
• Review conflicts of interest
• No advocacy or promotional position
• No financial interests
• Minimize domineering personalities
• Increase diversity

12
Optimal Media for the Expert Committee Research

• Recent research focuses on decision quality,
  satisfaction, and media richness
• Computer-mediated group decision systems
  decreases group effectiveness, increases time to
  complete task, and decreases member satisfaction
  in the process (Baltes, et al., 2002)
• However, groups with a history can be just as
  effective with a computer-mediated system (Alge,
  et al., 2003)
• Group decision support systems lead to more
  complex communications than simple chat systems
  (Brandy Young, 2002)

13
Optimal Media for the Expert Committee Research

• Adding video to an audio-only system improves
  decision quality (Baker, 2002)
• Participants are less likely to be committed to
  computer-mediated processes (Shangraw Bozeman,
  1989)
• After reviewing 200 studies, no difference
  between face-to-face meetings and specific
  collaborative technologies (Fjermestad Holtz,
  1998)

14
Optimal Format for the Expert Committee Practice

• Combine computer-mediated and face-to-face
  discussions if economically feasible
• Match media to the type and structure of the
  decision

15
Addressing the Concerns with Groupthink

• Facilitator should be impartial
• Facilitator should assign the role of critical
  evaluator to all committee participants
• One or more participants should be rotated
  through the role of devils advocate
• If feasible, sub-divide the group to work under
  different facilitators
• Immediately following preliminary consensus, hold
  a second chance meeting to express any residual
  doubts

16
Using a Structured Approach The Nominal Group
Technique

• Briefing provided on topic and method
• Ideas are silently generated on paper
• Participants share one idea from his/her list
• Review and consolidate ideas
• Voting is done privately
• Votes are tabulated

17
Direction of Future Research

• Intense focus on the value of computer-mediated
  group decision-making system
• Specific interest in web-based, distributed
  computer-mediated group decision support systems
• Increasing interest in the value of combining
  expert panels with empirical, data-driven models