Administrator interface: The administrator grants access to the system after confirming that the request has come from a faculty member. The interface shows active and pending faculty accounts, keeps track of each faculty

1
The Comprehensive Assessment of Team Member
Effectiveness A New Peer Evaluation Instrument
Matthew W. Ohland, General Engineering, Clemson
University Hal R. Pomeranz, Deer Run Associates,
Inc. Harlan W. Feinstein, Deer Run Associates,
Inc.
A multi-university research team has designed a
peer evaluation instrument that is simple to use.
The system gathers data from students through a
web interface, ensuring the confidentiality of
the peer ratings. The system analyzes the data to
calculate suggested grade adjustments for
equitably distributing a teams grade among the
teams members and provides extensive feedback to
faculty as to certain dynamics of student teams
that can be discerned from the peer evaluation
data. The scale from which the present instrument
is derived is previously published, and a variety
of reliability and validity studies are under
way. The paper in the proceedings includes a
description of the behaviorally anchored rating
scales, and the electronic interface and the
feedback it provides.
Administrator interface The administrator grants
access to the system after confirming that the
request has come from a faculty member. The
interface shows active and pending faculty
accounts, keeps track of each facultys last
login date and time. This interface also provides
access to the raw data for surveys released for
research purposes. Faculty interface faculty
enter information about classes, populate a class
with students, populate teams with students, and
set up surveys for team activities. Faculty
control the instructions given to the students,
the factors surveyed, whether consent is
required, and what data will be reported. Faculty
select from five factorsContributing to the
Team's Work, Interacting with Teammates, Keeping
Team on Track, Expecting Quality, and Having
Task-Related Knowledge/Skills/Abilities, in
addition to optional follow-up questions. All
survey settings are stored for potential reuse.
Some features are automated, such as sending
reminder emails to students who still need to
take the survey. Student interface students
receive an email when they are first added to the
system and secure their account. After login,
students find a list of active and completed
surveys. Since the teams are assigned, the
interface shows each student a list of their
teammates being evaluated when they are taking
the survey.
Exceptional Conditions The feedback provided by
exceptional conditions is an important
contribution of the system. Faculty confirm that
the system accurately predicts which students and
teams will struggle, so early intervention is
possible. The following conditions are flagged
Low (a student performing poorly who knows it),
Over (a student performing poorly who is
overconfident), High (a student performing well
who knows it), Under (an underconfident student
who is performing well who rates themselves lower
than their teammates), Conf (a conflict where
there is significant disagreement about the
performance of an individual student), and Cliq
(where a clique appears to have formed in which a
subset of the team appears pitted against another
subset of the team and vice versa).