ECUE MEETING

1
ECUE MEETING

• Monday, May 24, 2004

2
ECUE AGENDA MAY 24, 2004

• Final review of engineering workload study data
• Discussion of results and recommendations
• Discussion of possible next steps dissemination,
  website, workshop?
• ESD Undergraduate Potential Initiatives
  (Professor Joseph Sussman)
• Next year Input for AY 2004-05 ECUE theme
• Ideas Alumni follow up study on Seering/Wolfe
  study, 2004 NAE study of engineering, community
  service and engineering
• Summer 2004
• ECUE follow-up on workload study/
• ECUE preparation for AY 2004-05 theme
• ABET Mid-point Accreditation Cycle website and
  workshop?

3
ENGINEERING WORKLOAD STUDY SUMMARY 1

• Goal Examine undergraduate engineering workload
  with respect to learning, assessment, teaching
  methods, and study habits.
• Hypothesis
• There is a measurable workload hours issue.
• There is a perceived workload hours problem that
  impacts student motivation and learning.
• Study
• Review baseline workload data from standard MIT
  student surveys (COFHE Senior Exit Survey,
  Enrolled Student Survey, EBI Senior Engineering
  Survey) and subject evaluation surveys
• Hold focus groups with engineering seniors to
  establish issues related to engineering workload.
• Test and distribute engineering workload survey
  to seniors in 3 largest departments (ChemE, ME,
  EECS).

4
ENGINEERING WORKLOAD STUDY SUMMARY 2

• Findings
• Workload hours/ week varies for the three
  departments examined (ME, ChemE, EECS). Juniors
  in ChemE and juniors and seniors in EECS report
  highest average academic workload outside of
  class.
• Students perceived high workload is exacerbated
  by poorly structured teaching/ learning
  experiences in key engineering subjects.
  Perceptions of high workload can demotivate some
  students and impact performance and study habits.
• Teaching/ learning factors that can lead to a
  sense of high workload are clarity of subject
  learning goals amount of material to be covered
  in a subject teaching methods structure of
  psets, projects and exams assessment frequency
  feedback on assignments relationship with
  instructors.
• Best-practice teaching/ learning experiences that
  permit students to handle the high workload of
  engineering study identified in each department.
  Best-practice experiences highlight use of key
  teaching/learning factors in subject design.
• Recommendations
• Create teaching/ learning website that permits
  faculty to easily access information on designing
  a subject that balances key teaching/ learning
  factors with workload.
• Support use of website with Summer 2004 seminars.
  

5
Workload and learning Recent literature

• Student perceptions of workload are related to
  (Hounsell 84, Ramsden 92)
• Teaching approaches (didactic/ active)
• Assessment (grades) of exams and assignments
  (frequency)
• Relevance of coursework to student interests
• Work hours (class independent study hours)
• Perceived workload can shape students learning
  approaches (Ramsden and Entwistle 81, Entwhistle
  and Ramsden 83)
• High workloads can lead to students adopting
  reproducing learning approach over meaning
  learning approach
• Reproducing learning approach
• A study of a mechanical engineering program
  supports this finding (Kember and Leung 98) can
  particularly be the case in high workload
• Methods for studying student workload perceptions
  among engineering students
• Study process inventory permits some
  identification of student learning approaches
  when faced with high workload (Biggs, Kember and
  Leung 01)
• Workload diary of class and independent study
  hours (Kember et al., 95)
• Interview or focus groups to examine study
  strategies and workload (Gow and Kember 95)

6
IS THERE A MEASURABLE WORKLOAD ISSUE?
Data from ECUE Engineering Workload Survey
Spring 2004
7
IS THERE A PERCEPTION OF HIGH WORKLOAD?
8
WORKLOAD PERCEPTION FACTORS- RANKED 1-5
9
WORKLOAD PERCEPTION FACTORS- RANKED 6-9
10
Other workload factors often mentioned by students

• Subjects try to achieve too many learning
  objectives in one class
• The department has too many educational goals
  that they try to achieve in one class
  (professional, technical theory, communication,
  design)
• Crowding in subject content does not lead to a
  solid learning experience for many students.
• Instructors move at too fast a pace.
• Problem sets shouldnt be just one more grading
  opportunity
• Problem sets should have more instructional than
  evaluative purpose.
• Sometimes the issue is just one of too much work
  for number of units assigned to subject
• Units assigned and workload should reflect one
  another in a subject.
• Students sometimes feel that they just dont know
  the problem solving processes in their
  discipline.
• You need to know the tricks to complete
  assignments or tests and do well.

11
WORKLOAD FACTORS- SUMMARY

• Key factor mentioned by students is structure of
  assignments (psets, projects). Poorly structured
  assignments (wording, length, lack of connection
  to lecture, lack of clarity of problem solving
  process required for given problem) coupled with
  frequency of such lead to high workload.
• Lack of curriculum coordination across subjects
  means that faculty are not balancing high
  workload across subjects.

12
Reflection and learning student perceptions

• Students reported that they seldom review subject
  material during the term except before an exam
• On average, most students reported only reviewing
  subject material before an upcoming exam ( 61
  ME respondents, 74 ChemE respondents, 69 EECS
  respondents)
• On average, few students reported reviewing
  subject material before and during problem set
  completion 14 ME respondents, 9 ChemE
  respondents, 11 EECS respondents)
• Student comment No time to review! Its on to
  the next assignment!
• When workload is lower, students agreed that they
  will spend more time reviewing material

13
WILL FEWER PSETS OR EXAMS SOLVE THE WORKLOAD
PROBLEM? YES. WILL DRAMATICALLY CHANGING THE
SYSTEM OF FREQUENT ASSESSMENT SOLVE THE PROBLEM?
NO.
14
MIT STUDENTS ARE USED TO A SYSTEM OF FREQUENT
ASSESSMENT AND THAT MEETS THE NEEDS OF MANY TYPES
OF STUDENTS. MANY STUDENTS EXPRESS NEED FOR
VARIED ASSESSMENT METHODS FOR DISPLAYING
PERFORMANCE.
15
SOME STUDENTS PERCEIVE WORKLOAD DECREASES AND
LEARNING IMPROVEMENT IN SUBJECTS WHERE THERE IS
AN ENGAGED LEARNING PROCESS AND AN INSTRUCTOR WHO
KNOWS THEM.
16
STUDENTS IDENTIFIED BEST-PRACTICE SUBJECTS WHERE
DESIGN OF SUBJECT PERMITS STUDENT ACHIEVEMENT OF
LEARNING OBJECTIVES WITH A RELATIVELY HIGH
WORKLOAD!

• 2.005/2.006 (THERMAL FLUIDS I AND II) IN
  MECHANICAL ENGINEERING
• 6.004 (COMPUTATION STRUCTURES) IN EECS
• NO CLEAR WINNER IN CHEMICAL ENGINEERING
• ABOVE SUBJECTS HAD MUCH IN COMMON AS TEACHING/
  LEARNING EXPERIENCES

17
COMMON FEATURES OF BEST-PRACTICE IN HIGH WORKLOAD
SUBJECTS

• Lecture and learning goal clarity
• Clarity and continuity of learning goals from
  beginning of subject (or series of subjects) to
  end
• Clear connection of lecture, pset, lab, and
  project content and learning goals
• One year of continuity helped student absorb
  and reinforce learning of complex
• Lectures presented concepts verbally and
  numerically
• Labs visually reinforced concepts with visual,
  hands on representations
• Though sometimes too much was due at one time,
  problem sets and lab write-ups were clearly
  connected
• Carefully structured, not too long psets that
  illustrate concepts and problem solving methods.
  In 2.005/2.006, students were given many types of
  problems and examples to illustrate concepts.
  They were given many opportunities to try out
  problem solving in psets and labs.
• Sense of fair grading. Range of types of
  assessment methods (pset, project, exams) so that
  students could perform in range of formats.
• A design project that was not just an add-on,
  waste of time! You really had to design
  something!
• Engaged, approachable instructors made me want
  to work really hard. I felt that they really
  cared if I learned the material!

18
BOTTOM LINE - 1

• A well-designed subject is one where high
  workload can be achieved by careful balancing of
  subject learning objectives, content level,
  teaching methods, consideration of student
  learning styles and study habits, assessment
  methods and assessment frequency, feedback, and
  instructor-student interaction.

19
BOTTOM LINE - 2.

• Student study habits of seniors show that most
  MIT students are motivated, deep learners. High
  workload and poorly structured assignments
  frustrates students in achievement of deep
  learning.
• Kember et al. 97 suggests that Biggs study
  inventory be used to evaluate progress in
  educational program improvement.

20
RECOMMENDATIONS

• A fun, easy to use website that permits
  instructors to click on subject design features
  of interest.
• Supplement website with short, hands on seminar
  in which instructors work on own subjects. Leave
  with clear direction on how to improve subjects.
• Use Biggs study process instrument to
  periodically check on MIT student study habits.

21
Is subject trying to achieve too many objectives?
Is subject providing students with abilities to
use content in engineering work or just content?
Is subject carefully related to overall program
curriculum learning objectives?
Are psets carefully structured to provide
instruction in both content and problem solving
methods? Is exam structure and timing appropriate
for most students? Are open ended projects
structured so that workload is not overwhelming?
Are teaching methods (lectures, labs, online
material, muddy point, discussions, etc.) and
learning methods (assignments, feedback loop) and
staffing appropriate to achieve the objectives?

Subject objectives, outcomes







Assessment of learning outcomes (psets, exams,
projects, etc)




Teaching/ learning methods












Do all types of students have some time to
reflect on learning or just work hard? Are the
incentives for learning (as opposed to just
getting good grades) right?


Changes to subject


Instructor review of assessment data


Are assessment methods providing sufficient
information to know how students are doing at
different points in the term? Are instructor and
TA sharing info?






How often?




SUBJECT STRUCTURE
22
NEXT STEPS

• ECUE follow up on workload study?
• Suggestions?
• Support website/ seminars on subject design?
• Speaking of websitesfeedback on structure and
  user friendliness
• http//web.mit.edu/engineering/ecue
• Would like to improve use of site as part of
  ABET accreditation readiness. Were now at the
  mid-point in cycle.
• Support mid-point department review during Summer
  04?
• Next year Input for AY 2004-05 ECUE theme
• Ideas Engineering core abilities? Establish
  school-wide core abilities.
• Alumni follow up study on Seering/Wolfe study
  (available upon request)
• 2004 NAE study of engineering http//www.nap.edu
• Summer 2004 ECUE study group?

23
NAE STUDY

• What will or should engineering be like in 2020?
• What global issues will shape technology in 2020?
  In what societal, geopolitical, professional
  context will engineers work?
• What attributes will define the graduate of 2020?
  Strong analytical skills, creativity,
  professionalism, leadership

24
Other ideas

• Community service and engineering your thoughts