Integrated assessment of disciplinary, personal and interpersonal skills in a design-build course

1
Integrated assessment of disciplinary, personal
and interpersonal skillsin a design-build course
Kristina Edström, Khalid El Gaidi, Stefan
Hallströmand Jakob Kuttenkeuler June 8, 2005
2
Integrated assessment of disciplinary, personal
and interpersonal skillsin a design-build course
Kristina Edström, Khalid El Gaidi, Stefan
Hallströmand Jakob Kuttenkeuler June 8, 2005
3
Integrated assessment of disciplinary, personal
and interpersonal skillsin a design-build course
Kristina Edström, Khalid El Gaidi, Stefan
Hallströmand Jakob Kuttenkeuler June 8, 2005
4
The design-build course

• Idea to realisation (verifiable!)
• Large project groups (8-15 students)
• Final year
• Full year - 1/4 time
• Open-ended project specifications

5
Project task example

• Bring 2 kg of payload up in the air and keep it
  there using the sun as the only source of power
• a few constraints

6
Project examples
2001-2002 Solar powered aircraft 2002-2003 W
aterbike 2003-2004 Subskimmer
7
Solar powered aircraft
8
The course format

• Project based
• Some activities predefined and fixed others to
  be defined
• "A smörgåsbord syllabus" different students do
  different things
• Conceptual analysis
• Deep analysis
• Project management
• Manufacturing
• Experiments
• PR sponsoring
• Planning and follow-up

Typically all students do most of the things but
no student does everything
9
Course goals

• After the course the participant is expected to
  be able to work efficiently as an engineer in a
  project environment, i.e.
• take personal responsibility for small tasks and
  the project as a whole
• analyse technical problems from a holistic point
  of view
• handle technical problems which are incompletely
  stated and subjects to multiple constraints
• develop strategies for systematic choice and use
  of available engineering methods and tools
• make estimations and appreciate their value and
  limitations
• make decisions based on acquired knowledge
• communicate engineering orally, in writing and
  graphically

10
Integrated learning

• What does it mean to
• communicate engineering orally, in writing and
  graphically ?

• to describe and present ideas
• to develop ideas through collaborative sketching
  and engineering reasoning
• if necessary, to argue for or against conceptual
  ideas and solutions
• to communicate technically, both with experts
  and laypersons
• to nurture confidence in expressing oneself
  within a certain field of work

Communicating engineering is contextual
11
Assessment challenges and approach

• We want to assess individual achievements in a
  group setting How?!
• As personal and interpersonal skills are
  explicitly addressed
• personal and interpersonal skills should be
  assessed!
• We could only assess what we could observe or
  measure but here the main part of the work is
  carried out behind the scenes!
• In order to assess the work behind the scenes one
  has to
• either be there or ask those who are!
• The students trust in us is sometimes worrying

12
Assessment strategy

• Introduction
• Faculty
• communicate course goals
• instruct students to collect evidence
  (portfolios)
• instruct students to log own and others
  activities
• Students
• express personal goals
• plan own time
• declare anticipated working experiences

START
END
13
Assessment strategy

• Mid-course poll
• Faculty
• recap course goals
• teach giving/receiving feedback
• Students
• write narratives
• read narratives, write feedback, suggest peer
  grades
• read feedback, write reflective documents
• revisit/revise personal goals
• follow up with reflective exercises

START
END
14
Assessment strategy

• Final assessment
• Faculty
• recap course goals
• recap reflections from poll
• Students
• write narratives
• write feedback, suggest grades
• follow up with discussions on the contribution
  from each student
• follow up with reflective exercises
• write reflective documents

START
END
15
Grading

• The grades are finally set based on a holistic
  assessment of
• portfolios (reports, protocols, presentations,
  sketches, hardware, )
• with respect to course goals
• given feedback
• received feedback recommended grades
• reflective documents
• participation
• and guided by
• logged time
• continuous observations

16
Some general observations

• Students attitudes to knowledge are challenged
• Applied knowledge differs from theoretical
  knowledge
• Decision making is new to students
• Distinction between project goals and course
  goals is essential
• Peer assessment is delicate to compose not
  enough to use the right instruments, one has to
  play them careful too

17
Conclusions and final remarks

• Open-ended project implies limited number of
  predetermined activities
• assessment scheme should be generic
• In order for peer assessment to be useful and
  constructive
• students must be comfortable in the process
• students must rely on the process
• The experience from developing the peer
  assessment scheme is that
• it is not only what you do that matters but also
• how you do it and
• how you present it to the students