AssessmentDriven Design Education: A CrossDisciplinary Collaborative Approach

1
Assessment-Driven Design Education A
Cross-Disciplinary Collaborative Approach

• Deans Summit on Education for a Technological
  World
• Baltimore, MD
• October 1-2, 2001
• EDUCATION Michael Trevisan, Judy Mitchell (dean)
• ENGINEERING Denny Davis, Anjan Bose (dean)

2
Growing Collaboration
TIDEE II
TIDEE
Joint engineering and education project funding
from NSF since 1995

• Washington Council for Engineering and Related
  Technical Education
• 26 community colleges, 5 state universities, 5
  private colleges

Northwest Roundtable for Engineering Education
engineering-related university, industry,
government leaders
Collaborators
Mid-Program design education and assessment
End-of-Program design education and assessment
Project Focus
3
Focus of Collaboration
Mid-ProgramOutcomes
End-of-ProgramOutcomes
Educational Objectives
Junior Senior Education Process
Freshman Sophomore Education Processes
Mid-Program OutcomesAssessment
End-of-Program OutcomesAssessment
Assessment Processes
4
Statewide EndorsementA Basis for TIDEE
Collaboration
(Approved by WCERTE 25 October 1996)

• The foundations fordesign education mustbe
  incorporated intothe first two yearsof
  engineering andengineering technologycurricula.
  
• This includes development of competence in
  communication, teamwork, and the creative problem
  solving or engineering design process.

Washington Council for Engineering and Related
Technical Education
5
Keys to Quality AssessmentsA Basis for
Assessment

• Clear purposes for assessment
• Clear and appropriate achievement targets
• Target and method match
• Appropriate content sampling
• Bias and distortion are eliminated

Stiggins, R.J. 1995. Assessment Literacy for the
21st Century. Phi Delta Kappan, 77(3) 238-245.
6
Mid-Program Assessment of Team-Based Design--
Three components
Component 1 SHORT ANSWER (15 minutes)
Component 2 GROUP ACTIVITY (45 minutes)
Component 3 2-PAGE ESSAY (take-home)

• Define essential elements of
• Design process
• Teamwork
• Design Communication

• Team project
• Organize team
• Record process
• Produce required deliverables

• Reflect on project
• What was done
• What produced desired results
• How to improve performance

Basic Knowledge
Application of Knowledge
Critical Analysis
Increasing Level of Learning
7
Assessment Scoring Reliability Study Fall 2000
Sampling of Population

• Two institutions, five class sections, five
  instructors
• One administrator of assessments
• Four disciplines
• Mechanical engineering
• Electrical engineering
• Computer engineering
• Biological systems engineering
• n 139 students

8
A 3-Step Scoring Process
Multi-Disciplinary Evaluation Team

• .

Standard Administration of Assessment
Evaluation Team
1
2
3
Establish Scoring Criteria
Establish Decision Rules
Score Assessment Results
Subsets of Student Work
Population of Student Work
Reliability for Group Performance (mean of 3
raters)
0.43 0.85
9
Approach to End-of-Program Assessment for
Engineering

• Use capstone project as assessment exercise
• Team-based design projects (1 or more terms in
  length)
• Required part of all engineering curricula
• Provide students a project guide to
• Communicate achievement targets expected
• Elicit student responses used for each outcome to
  be assessed (match target and method)
• Establish scoring scales and decision rules for
  use in scoring student work
• Engineering intern
• Entry-level engineer
• Project engineer

10
End-of-Program Assessment Using Capstone Design
Projects
11
Outcomes of Collaboration

• Defined Educational Outcomes
• Workshops with industry and academic engineers
• Mid-program and end-of-program outcomes defined
• Developed Mid-Program Assessment Instrument
• Instrument pilot tested and disseminated
• Scoring process defined and reliability
  determined
• Developed Curriculum Materials
• Activities for design process, teamwork, design
  communication
• Workshops used to disseminate materials and
  methods
• Materials cataloged and posted on project web
  site
• Created Web-Based Survey of Alumni and Employers
• Survey developed and used by multiple programs,
  institutions

12
Challenges and Opportunities in Collaboration

• Challenges
• Establishing relationships of trust and mutual
  goals
• Funding graduate students in engineering
• Creating publications valued by both disciplines
• Opportunities
• Meets need in engineering for educational
  expertise
• Provides education faculty ready laboratory
• Results contribute to scholarship in both
  disciplines

13
Next Steps in TIDEE Engineering and Education
Collaboration

• Build on Combined Assessment Strengths
• Identify best practices in engineering design
  assessment
• Bridge from Mid-Program Assessment to reliable,
  transferable End-of-Program Assessment
• Build on Combined Curriculum Strengths
• Expand curricular materials to develop critical
  capabilities in students
• Extend curricular materials and methods to
  support generic problem solving and critical
  thinking
• Enhance student learning through guided
  multi-disciplinary capstone projects