Case study:

1
Case study

• Experience with a Peer Evaluation Tool
• for Group Work
• Kenneth Johns, Université de Sherbrooke
• Québec Canada

2
Canadian program reform in Engineering

• Objectives
• Integration of subjects and of learnings
• Contextualisation
• Professionalism
•  Soft skills   (communication, teamwork,
  interpersonal )
• Design as a high priority

3
Examples of Canadian reforms

• Queens University, Kingston  Integrative
  learning centre 
• Sherbrooke, Electrical and computer Eng 100
  Project and problem-based learning
• Sherbrooke, Mechanical and Civil Course-based
  curriculum with innovations

4
Sherbrooke Mechanical and Civil program features

• Just-in-time teaching of science and math done by
  engineering staff
• Integrative projects within sessions
• Design taught right from first year
• Teamwork, communication, ethics and humanities
  integrated into curriculum
• Sessional themes, team meetings, student feedback
  systems
• Major design projects at different terms

5
First year Civil Engineering course Integration
to Professional Practice

• Writing skills and teamwork skills
• Visiting engineers and real-world professional
  issues
• Preparation for first job-placement term
• Major documentary research project done in teams

6
Research project onMajor Civil Engineering
Projects (in teams of four or five students)

• Insights
• Social, and cultural dimensions
• Impact of finances and management strategies
• Political debates and problems

• Competencies
• Researching and interpreting information
• Writing, presenting skills
• Professional maturity and confidence

7
Examples of projects studied

•   The Panama Canal, French failure and American
  success, the role of Yellow fever
•   The Eurotunnel project major financial
  difficulties
•    The first James Bay Hydro development in
  northern Quebec environmental impact and
  displacement of Aboriginal peoples
•    The Canadian Offshore Hibernia project in the
  North Atlantic, weathering storms, icebergs,
  environmental activism and fluctuating oil
  prices.

8
Deliverables

• Project Proposal, October
• Annotated bibliography, November
• Preliminary report, December
• First oral presentation, January
• Final oral (5 minute per person) presentation,
  March
• Final written report, April

9
Evaluating individual performance within a team

• A peer evaluation tool, used twice, anonymously
• Divide 10,000 among your team members
  proportional to the Value of each ones
  contribution  (C. Lovas, School of Engineering.
  Southern Methodist University, Dallas, Texas)
• Justify any difference of more than 20
• Equal amounts will be rejected
• The combined attributions are used to
  redistribute the instructors group mark

10
Results(for two applications in the same year)

• Students always write comments!
• Majority of students give higher marks to others
• 80 , then 91 unanimity in identifying the
  highest-value member
• 84, then 86 unanimity in identifying the
  weakest member
• 71, then 57 of weakest contributors
  self-identify
• Students will reward good leadership

11
Some results, 2002-2003
12
Some results, 2002-2003
13
Some Analysis 2002-2003

• Stronger students tend to under-estimate the
  value of their own contribution
• Weaker students tend to over-estimate the value
  of their contributions
• Some weak students have quite unrealistic ideas
  about their own contributions
• (an honest false impression or a
  defence mechanism?)

14
Student Participation and opinion

• Experience shows high student competence in using
  the tool, and good ethics.
• Student opinions satisfaction that
• Good contributions will be rewarded, the opposite
  detected and penalised
• A strong leader in a weak group can still get a
  good mark

15
Conclusions

• Appears to give credible, reliable results
• Further development potential Possible use for
  feedback to students?
• (students not sure)
• Corroboration of peer scores with some other
  measure would increase confidence