ASSESSING STUDENT LEARNING AND OVERALL PROGRAM EFFECTIVENESS

1
ASSESSING STUDENT LEARNING AND OVERALL PROGRAM
EFFECTIVENESS Summary Doris R. Brodeur CDIO
External Review 19 June 2003
2
PRESENTATION GOALS

• To highlight the development and use of new tools
  and procedures for assessing student learning and
  program effectiveness
• To show evidence of CDIO learning outcomes that
  can be measured with these tools
• To provide samples of data collected to date that
  give information about student learning and
  program effectiveness
• To demonstrate ways in which assessment and
  evaluation data are used in continuous program
  improvement processes

3
CONTINUOUS IMPROVEMENT PROCESS

Ongoing Data Collection
Course Level
Incentives for Change
Review of Results

Recommended Actions
Implementation of Changes
Program Level
4
TRANSFERABLE RESULTS

• Documentation of our experiences with assessment
  and evaluation
• Starter Kit
• Tools and procedures for assessing student
  learning and satisfaction within courses
• Tools and procedures for collecting data for
  overall program evaluation
• Procedures for using data for continuous
  improvement of programs and processes

CDIO Partners
Engineering Programs
Engineering
External Stakeholders
Education
Higher
5
STARTER KIT TOOLS
Course Level
Program Level

Peer and Self Assessment
Senior Exit Interviews
Baseline Interviews
Oral Exams

Course Evaluations

Oral Presentations
Baseline Survey
Reflective Memos
Reflective Portfolios

6
STARTER KIT REPORTS
Course Level
Program Level

Senior Exit Interviews
Baseline Interviews
Conceptual Learning

Program Portfolios
Senior Exit Surveys
Reflective Memos
Reflective Portfolios
Longitudinal Studies
7
PUBLICATIONS

• Using Portfolios for Exit Assessment in
  Engineering Programs, D. Brodeur, 32nd ASEE/IEEE
  Frontiers in Education Conference, Boston, MA,
  November 2002.
• Using Concept Maps and Concept Questions to
  Enhance conceptual Understanding, D. Darmofal,
  D. Soderholm, D. Brodeur, 32nd ASEE/IEEE
  Frontiers in Education Conference, Boston, MA,
  November 2002.
• International Collaboration in the Reform of
  Engineering Education, D. Brodeur, E. Crawley,
  I. Ingemarsson, J. Malmqvist, S. Ostlund, ASEE
  Conference, Montreal, Canada, June 2002.
• Problem-Based Learning in Aerospace Education,
  D. Brodeur, P. Young, K. Blair, ASEE Conference,
  Montreal, Canada, June 2002.
• The CDIO Capstone An Innovation in
  Undergraduate Systems Engineering Education, D.
  Miller, D. Brodeur, ASEE Conference, Montreal,
  Canada, June 2002.

8
PRESENTATIONS

• Student Portfolios as Exit Assessment in
  Professional Education, D. Brodeur, American
  Association of Higher Education Conference,
  Seattle, Washington, June 2003.
• Assessing the Teaching and Learning of
  Mathematics in the Mechanical Engineering Program
  at Chalmers Technical University, T. Lingefjärd,
  Psychology of Mathematics Education Conference,
  Athens, Georgia, October 2002.
• Teaching for Conceptual Understanding, D.
  Brodeur, Linköping University, October 2001.
• Expectancies and Realities, E. Edvardsson
  Stiwne, M. Rosander, D. Stiwne, ECER Conference,
  Lille, September 2001.
• Educational Objectives in Math and Engineering
  Courses, D. Brodeur, Kungi Tekniska Högskolan
  (KTH), March 2001.
• Educational Objectives and Intended Learning
  Outcomes, D. Brodeur, Chalmers University of
  Technology, December 2000.

9
BENEFITS OF OUR COLLABORATION

• Shared understanding of the importance of
  assessment and evaluation
• Recognition of the role of assessment in the
  teaching and learning process
• Learning from each others experiences in
  introducing new assessment methods
• Opportunities to collaborate in workshops,
  research and publication on topics related to
  assessment and evaluation

10
IMPACT

• Impact on students
• Increase in learning and satisfaction because of
  continuous improvement processes now in place
• More willing participation of students in
  assessment processes
• Impact on instructors
• Increasing competence and confidence in using new
  assessment methods
• Eagerness to address new assessment challenges
• Impact of stakeholders
• Tangible evidence of student learning and program
  effectiveness
• Recognition of their roles in continuous
  improvement processes
• Impact on engineering education
• Expanded knowledge base on educational evaluation
  methods and procedures
• Emphasis on overall program effectiveness beyond
  individual course effectiveness

Students
Instructors
Engineering Education
Stakeholders
11
PLANS FOR YEAR 4

• Refinement of assessment tools with a focus on
  project-based courses
• Guidance on statistics for reporting results at
  the course level and program level
• Investigation of the consequences to students,
  faculty, and programs of introducing new
  assessment tools

Consequences
Project-Based Courses
Reports
12
PLANS FOR YEARS 5 AND 6

• Development and piloting of tools and procedures
  to measure more-difficult-to-measure CDIO
  learning outcomes, e.g. risk-taking, creativity,
  professional ethics
• Continued assessment of students until at least
  two cohort groups have completed the full program
• Expansion of program evaluation to include
  stakeholders, other engineering programs,
  industry, the field of engineering.

Exit Data
Program Evaluation
CDIO Outcomes