Title: Expert Committees
1Expert Committees
- R. F. Shangraw, Jr., Ph.D.
- Chief Executive Officer
- Project Performance Corporation
Methods and Approaches in Screening Proprietary
Drug Names Public Meeting Renaissance Washington
DC Hotel June 26, 2003
2Topics
- Framing the Problem
- Identifying Relevant Research
- Addressing the Key Questions
- Raising Two Other Concerns
3Framing the Problem
Bound the Decision
Identify Alternatives
Evaluate Alternatives
Consensus Decision
- Using expert committees to review proposed
proprietary drug names for potential sound alike
and look alike confusion with existing drug names - Using expert committees to review multifactoral
studies on potential sound alike and look alike
confusions - Using expert committees to provide input to a
single factor study
4The Key Questions
- Is an expert committee necessary to review
information from studies? - How many people should staff an expert committee?
- What credentials are important for expert
committee members? - Should expert committees meet in person, via
videoconference, teleconference, email?
5Identifying Relevant Research
- Psychology
- Sociology
- Law
- Jury Deliberations
- Expert Witness
- Science Court
- Policy Sciences
- Forecasting
- Game Theory
- Organizational Behavior
- Medicine
- Appropriate and Necessary Care
- Group Decision-Making Methods
- Delphi Method
- Nominal Group Technique
- NIH Consensus Development Program
- RAND/UCLA Appropriateness Method
6Necessity of Expert Committees Research
- Empirical research suggests group decisions are
more consistent individual judgments are prone
to personal bias (McDonnell et al, 1996) - There is some empirical evidence that groups make
quicker and better decisions (Blinder Morgan,
2000 Hogarth, 1978) - However, some empirical research on group
decision making suggests a systematic bias in the
outcomes (Janis Mann, 1977) - But, a recent study found that groups are better
than individuals in interpreting letters and
number substitutions (Laughlin, et al., 2002)
7Necessity of Expert Committees Practice
- Use an Expert Panel When
- Historical data does not exist or is limited
- Future events are likely to invalidate the
results of historical analysis - Ethical and moral factors are sufficiently
important to the decision
8Optimal Size of Expert Committees Research
- Assuming specific voting rules, committee size is
negatively correlated with decision accuracy
(Gabel Shipan, 2000) - Communication quality and complexity is greater
in small groups (n5) than in larger groups
(n10) (Fay, et al., 2000) - Limit to 12 15 members for effective
functioning of the group (Shekelle, et al., 1999)
9Optimal Size of Expert Committees Practice
Group Decision-Making Method Recommended Expert Committee Size
RAND/UCLA Appropriateness Method 7 15 Panelists
Nominal Group Technique 8 12 Participants
NIH Consensus Development Program 9 16 Panelists
10Credentials for Committee Participation Research
- Need a baseline level of expertise but additional
expertise does not improve accuracy (Armstrong,
1980) - Committees should include participation from a
variety of relevant disciplines (Fitch et al.,
2001) - Participants status affects group dynamics
(Murphy, et al., 1998) - Participants must be justifiable as "expert" on
the matter under discussion (Jones Hunter,
1995)
11Credentials for Committee Participation Practice
- Establish baseline qualifications
- Review conflicts of interest
- No advocacy or promotional position
- No financial interests
- Minimize domineering personalities
- Increase diversity
12Optimal Media for the Expert Committee Research
- Recent research focuses on decision quality,
satisfaction, and media richness - Computer-mediated group decision systems
decreases group effectiveness, increases time to
complete task, and decreases member satisfaction
in the process (Baltes, et al., 2002) - However, groups with a history can be just as
effective with a computer-mediated system (Alge,
et al., 2003) - Group decision support systems lead to more
complex communications than simple chat systems
(Brandy Young, 2002)
13Optimal Media for the Expert Committee Research
- Adding video to an audio-only system improves
decision quality (Baker, 2002) - Participants are less likely to be committed to
computer-mediated processes (Shangraw Bozeman,
1989) - After reviewing 200 studies, no difference
between face-to-face meetings and specific
collaborative technologies (Fjermestad Holtz,
1998)
14Optimal Format for the Expert Committee Practice
- Combine computer-mediated and face-to-face
discussions if economically feasible - Match media to the type and structure of the
decision
15Addressing the Concerns with Groupthink
- Facilitator should be impartial
- Facilitator should assign the role of critical
evaluator to all committee participants - One or more participants should be rotated
through the role of devils advocate - If feasible, sub-divide the group to work under
different facilitators - Immediately following preliminary consensus, hold
a second chance meeting to express any residual
doubts
16Using a Structured Approach The Nominal Group
Technique
- Briefing provided on topic and method
- Ideas are silently generated on paper
- Participants share one idea from his/her list
- Review and consolidate ideas
- Voting is done privately
- Votes are tabulated
17Direction of Future Research
- Intense focus on the value of computer-mediated
group decision-making system - Specific interest in web-based, distributed
computer-mediated group decision support systems - Increasing interest in the value of combining
expert panels with empirical, data-driven models