A Methodology for Evaluating Education Programs

1
A Methodology for Evaluating Education Programs
Leyla Conrad PRC, GaTech Bethany Bodo, Joseph
Hoey Office of Assessment, GaTech Director of
Assessment, Radford University Jill
Auerbach ECE, GaTech
Supported by the NSF under grant numbers
EEC9810227 and EEC9402723
2
Approach

• Work with a professional assessment person
• Determine which activities to evaluate
• Courses (hands-on, system-level)
• Curriculum
• Graduating students (UG and graduate)
• Participants (short courses, conferences,
  workshops)
• Summer Research (REU and high school students)
• Teachers (RET program)
• Determine expected outcomes for each activity

3
PRC Education Programs
4
Curriculum
Certificate Program Major ECE, ME or MSE Minor
Management
5
Comprehensive Assessment Areas

• System-level course
• Management course
• DBO courses
• POM program
• Graduating students (exit surveys)
• REU
• RET

6
Methodology for POM

• Continuous dialogue with industry
• Compare POM with similar programs update it
• Evaluate POM curriculum and courses
• Exit interviews with students
• How many job offers they receive
• Role of packaging certificate in securing
  employment
• Role of management courses in securing employment
• Role of internship in securing employment
• Keep track of students after graduation
• -Percent employed in packaging industry
• Contact graduates internship supervisor
• -Do POM graduates undergo extra training?
• -In what areas do their skills need to be
  improved
• What are the significant advantages of a POM
  background?

7
Used Assessment Methods

• Quantitative
• - Tests
• - Rubrics course projects
• - Course surveys
• - POM exit surveys
• Qualitative
• - Employer surveys
• - Open ended course surveys
• - Interviews with students

8
ABET Criterion 3. Program Outcomes and
Assessment Engineering programs must demonstrate
that their graduates have 2004-2005 Criteria
for Accrediting Engineering Programs (a) an
ability to apply knowledge of mathematics,
science, and engineering (b) an ability to design
and conduct experiments, as well as to analyze
and interpret data (c) an ability to design a
system, component, or process to meet desired
needs (d) an ability to function on
multi-disciplinary teams (e) an ability to
identify, formulate, and solve engineering
problems (f) an understanding of professional and
ethical responsibility (g) an ability to
communicate effectively (h) the broad education
necessary to understand the impact of engineering
solutions in a global and societal context (i) a
recognition of the need for, and an ability to
engage in life-long learning (j) a knowledge of
contemporary issues (k) an ability to use the
techniques, skills, and modern engineering tools
necessary for engineering practice.
9
Educational Outcomes DBO1

• Perform simple PCB layout using the Cadence APD
  software package.
• Operate the following apparatus
• - spin coater
• - exposure tool
• - alignment tool
• - development station
• - electroplating baths
• - curing oven
• 3) Explain the significance of the following
  process steps
• - dielectric deposition
• - via formation
• - metallization
• - dry film lamination
• 4) Characterize and test
• - base substrate materials
• - dielectric layers
• - feature dimensions
• - metal connectivity

10
Course Survey DBO1

• I. Faculty Contributions
• To what extent did the instructor of this course
  do each of the following?
• Incorporate viewpoints from multiple disciplines
  into the learning process
• Incorporate teamwork in classroom activities
• Encourage you to see learning as a lifelong
  process
• Introduce you to new technologies and equipment
• Please rate the overall quality of instruction
  provided by the instructor

11

• II. Knowledge and Skills
• To what extent did this course contribute to your
  knowledge, skills, and personal development in
  the following areas?
• Technical report writing skills
• Verbal communication skills
• Presentation skills
• Leadership skills
• Interpersonal skills
• Ability to learn using the Internet
• Ability to learn using computer-based instruction
  
• Ability to using the library as a tool for
  research
• Understanding of contemporary issues related in
  microelectronics packaging
• Understanding of the global scope of the
  microelectronics packaging industry
• Ability to work with individuals from diverse
  backgrounds
• Ability to function as part of a
  multidisciplinary team
• Ability to synthesize and integrate knowledge
  across disciplines while learning
• Familiarity with the engineering systems approach
  
• Provide a broad education in primary
  microelectronics packaging disciplines
• Provide a system-level understanding of
  microelectronics packaging
• Provide a multidisciplinary engineering education

12

• Provide an understanding of science and
  engineering fundamentals relevant to electronics
  packaging
• Provide in depth research experience from
  fundamental science to commercial Application
• Ability to define microelectronics packaging
  problems
• Ability to define the steps needed to solve new
  problems
• Ability to develop solutions to microelectronics
  packaging problems
• Provide hands-on practical experience with
  electronics packaging systems and prototypes
• Ability to design experiments
• Ability to conduct experiments
• Ability to analyze data
• Ability to interpret data
• Understanding of technology applications relevant
  to microelectronics packaging
• Training on state-of the art research equipment
• Depth of technical understanding
• Breadth of technical understanding

13

• Course Objectives
• To what extent did this course enable you to
• Perform simple PCB layout using the Cadence APD
  software package
• Gain competence in operating the following
  apparatuses
• Spin coater
• Exposure tool
• Alignment tool
• Development station
• Electroplating baths
• Curing oven
• Gain an understanding of the following process
  steps
• Dielectric deposition
• Via formation
• Metallization
• Dry film lamination
• Gain competence in the characterizing and testing
  of
• Base substrate materials
• Dielectric layers
• Feature dimensions

14
Rubrics DBO2
15

• To what extent did the students report
  demonstrate an understanding of
• The principles of thermal management
• The sequence necessary for packaging assembly
• How to select materials for packaging
• How to inspect the quality of a packaging process
  
• The experimental methods used for cooling devices
  on chips / dice
• The methods used to conduct a functional test
• The methods used to evaluate the result of a
  functional test
• The methods used to conduct thermal reliability
  tests
• The methods used to evaluate the result of a
  thermal reliability tests
• The methods used to inspect the specimens using
  moiré technique, Sonoscan or x-ray systems
• How to evaluate the reliability of chip assembly
  using ANSYS
• How to evaluate the reliability of chip assembly
  using experimental methods
• How to conduct a manufacturing process for flip
  chip component placement
• How to conduct a manufacturing process for
  Surface Mount Technology component replacement
• How to conduct a functional test on a chip using
  a high-speed tester
• How the packaging will be performed for flip chip
  components
• How the packaging will be performed for Surface
  Mount Technology
• The continuity test

16

• To what extent did the students report
  demonstrate that the student learned how to MAKE
  research level specimens using
• Thermal Cycle
• Shear Testing
• Defect evaluation using Sono Scan equipment
• Optical Profiler
• Defect evaluation using X-Ray equipment
• Underfill property evaluation
• Surface inspection using microscope
• Electrical continuity evaluation
• To what extent did the students report
  demonstrate that the student learned how to TEST
  AND EVALUATE research level specimens developed
  using
• Thermal Cycle
• Shear Testing
• Defect evaluation using Sonoscan equipment
• Optical Profiler
• Defect evaluation using X-Ray equipment
• Underfill property evaluation
• Surface inspection using microscope
• Electrical continuity evaluation
• To what extent did the students report
  demonstrate a knowledge of

17
Open Ended Survey DBO2

• Have your experiences in the DBO(2) course helped
  to enhance your communication skills? If so, in
  what ways and by what means?
• (a) Specifically, has the program helped to
  enhance your oral communication, technical
  writing, or presentation skills?
• (b) Can you give me an example of an activity
  that helped to develop your communications
  skills?
• To what extent have you worked in
  multidisciplinary teams to complete the
  coursework for DBO(2)?
• (a) How many group projects have you
  participated in during the past semester?
• (b) Can you comment on your learning experiences
  in these group work situations?

18

• 3) Please describe your level of proficiency thus
  far in science and engineering fundamentals
  related to microelectronics packaging.
• (a) Do you feel as if the DBO(2) course has
  helped to increase your proficiency in
  microelectronics packaging fundamentals? If so,
  how?
• (b) What educational experience was most
  beneficial to your learning of microelectronics
  packaging fundamentals?
• 4) To what extent has the DBO(2) offered you the
  opportunity to personally use equipment related
  to microelectronics packaging systems and
  prototypes? (Hands-on experience)
• (a) How much hands-on exposure have you had to
  microelectronics packaging equipment and
  instruments?
• (b) From what classroom activity did you gain
  the most hands-on experience?

19

• Overall, how would you rate the DBO(2) course? (5
  excellent and 1 poor)?
• (a) What aspects of the course did you find the
  least useful?
• (b) What aspects of the course did you find the
  most useful?
• 6) Please feel free to comment on any other
  aspect of the DBO(2) course.

20
Student Interview Questions(POM program)

• How did the program help to enhance your
  communication skills?
• Have you been able to work in multidisciplinary
  teams?
• What exposure the program provided you in
  professional, ethical or contemporary issues in
  MSP?
• Describe your proficiency in science and
  engineering fundamentals related to MSP.
• What is your comfort level and capabilities in
  tackling MSP problems?
• How much opportunity is offered to use equipment?
• To what extent the program has prepared you to
  contribute to industry.
• To what extent management courses prepared you to
  tools, techniques, concepts relevant to
  management?
• Comment on the MSP courses most and least
  useful.
• Provide three major strengths of the program.
• Provide three areas for improvement.

21
Responses

• Strengths of the program
• - multidisciplinary nature
• - hands-on courses
• - management component
• - internship component
• Areas of improvement
• - structure and organization of courses

22
Industry Respondents Perceptionon Abilities of
POM Interns
23
Professional Issues
24
Work Attitudes and Skills
25
Career Skills
26
PRCs Dissemination Methods

• Partnered together with professional societies
  (IEEE/CPMT ASME) to create education sessions in
  major conferences
• Organize annual international academic
  conferences
• Undergraduate textbook
• Web based course modules
• Submit papers to major engineering education
  conferences
• Maintain an updated the web site