Girard School of Business

1
Girard School of Business International
Commerce Merrimack College
AACSB CONTINUOUS IMPROVEMENT SEMINAR How
Small Schools Assess Learning
While Youre Waiting for the Session to
Start Please Fill Out An Assessment Survey (If
you havent already sent one in)
2
Girard School of Business International
Commerce Merrimack College
How would we assess how much this MBA student
learned?
3

Girard School of Business International
Commerce Merrimack College
How Small Schools Assess
Session Goals

• Provide a benchmark of the ways small business
  schools assess learning.
• Provide different outcome assessment strategies.
• Evaluate your own outcome assessment strategies.
• To explore whether there is an opportunity for a
  small school assessment consortium.

AACSB CIS September 23, 2002
Ed Popper
4

Girard School of Business International
Commerce Merrimack College
Sample Characteristics

• 53 Survey Responses (164 Distributed)
• 32.3 Response Rate
• AACSB Status
• 50 Accredited
• 20 Candidacy
• 30 Member - Unaccredited

Ed Popper
AACSB CIS September 23, 2002
5
Types of Assessment Measures
Girard School of Business International
Commerce Merrimack College

• Course Embedded
• Course Evaluations
• Comprehensive Exam
• Capstone Course
• Student Portfolios
• Surveys

AACSB CIS - September 23, 2002
Ed Popper
6

Girard School of Business International
Commerce Merrimack College
Use of Course Embedded Measures

• Final Grades -- Percent of Schools Using Final
  Course Grades As Indicators of
  Achieving Program Outcomes
• 62.7 For Undergraduate Programs
• 57.9 For MBA Programs
• Downstream Course Measures -- Percent of
  Schools Using Retention of Prerequisite
  Material in Subsequent Courses As
  Indicators of Achieving Program Outcomes
• 35.3 For Undergraduate Programs
• 29.7 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
7

Girard School of Business International
Commerce Merrimack College
Use of Course Evaluation

• Percent of Schools Using Course Evaluations As
  Indicators of Achieving Program Outcomes
• 69.2 For Undergraduate Programs
• 70.3 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
8
Course Evaluation Type Use
Girard School of Business International
Commerce Merrimack College

• Types of Course Evaluations Used
• 21.1 Use Nationally Normed Instrument
• 50.0 Use Institutionally Normed
  Instrument
• 39.5 Use Business Unit Instrument
• 10.5 Use Department Instrument
• 7.9 Use Individual Faculty
  Members Instrument

• Who Sees Evaluation Results
• 92.1 Individual Instructors
• 97.4 Dean/ Department Chair
• 15.8 Other Department Members
• 21.1 Entire Business Unit
• 7.9 Entire College/ University
• 2.6 Students

As a percent of Schools that use course
evaluations for program assessment
Ed Popper
AACSB CIS September 23, 2002
9

Girard School of Business International
Commerce Merrimack College
Use of Comprehensive Examinations

• Percent of Schools Using Comprehensive Exams As
  Indicators of Achieving Program Outcomes
• 71.4 For Undergraduate Programs
• 30.8 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
10
Undergraduate Use Of Comprehensive Exams
Girard School of Business International
Commerce Merrimack College

• 71.4 Use A Nationally Standardized Exam
• 89.3 Use ETS Major Field Exam
• 3.6 Use EBI
• Exam Includes
• 94.3 Measures of Knowledge of Core
  Material
• 66.7 Measures of Core
  Competencies
• 67.9 Measures of Knowledge of Majors
  Material
• 69.2 Measures of Majors
  Competencies

• When Exam is Given
• 42.9 Before Graduation as a Separate
  Test
• 48.6 During Final Semester as Part of a
  Course
• 8.6 Earlier in Program for Pre- Post
  Measure
• 38.2 Taking Exam is a Graduation Requirement
• 9.4 Passing Exam is a Graduation Requirement

(As a percent of Schools that use a comprehensive
exam for undergraduate program assessment)
Ed Popper
AACSB CIS September 23, 2002
11
Graduate Use Of Comprehensive Exams
Girard School of Business International
Commerce Merrimack College

• 30.7 Use A Nationally Standardized Exam
• 15.3 Use ETS Major Field Exam
• Exam Includes
• 87.5 Measures of Knowledge of Core
  Material
• 75.0 Measures of Core Competencies
• 60.0 Measures of Knowledge of Majors
  Material
• 60.0 Measures of Majors
  Competencies

• When Exam is Given
• 30.0 Before Graduation as a Separate
  Test
• 60.0 During Final Semester as Part of a
  Course
• 10.0 Earlier in Program for Pre- Post
  Measure
• 40.0 Taking Exam is a Graduation Requirement
• 10.0 Passing Exam is a Graduation Requirement

(As a percent of Schools that use a comprehensive
exam for Graduate program assessment)
AACSB CIS September 23, 2002
Ed Popper
12

Girard School of Business International
Commerce Merrimack College
Use of Capstone Course Activities

• Percent of Schools Using Activities in a
  Capstone Course As Indicators of Achieving
  Program Outcomes
• 84.3 For Undergraduate Programs
• 81.8 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
13
Assessment Activities Used In Capstone Course
Girard School of Business International
Commerce Merrimack College

• For Undergraduate Programs
• 80.5 Use Individual Projects
• 81.4 Use Group Projects
• 62.2 Use Competitive Simulation
• For MBA Programs
• 77.8 Use Individual Projects
• 79.2 Use Group Projects
• 55.6 Use Competitive Simulation

(As a percent of Schools that use Capstone Course
Activities for program assessment)
AACSB CIS September 23, 2002
Ed Popper
14

Girard School of Business International
Commerce Merrimack College
Use of Student Portfolios

• Percent of Schools Using Student Portfolios As
  Indicators of Achieving Program Outcomes
• 10.4 For Undergraduate Programs
• 2.9 For MBA Programs
• Items included in Portfolios
• Repeated business plan and case done four times
  in the undergraduate program (Freshman intro to
  Business, Accounting, Marketing, and capstone)
• The portfolio is information from the internship.
  
• Selected samples of work from all eight
  undergraduate semesters regarding core
  competencies
• Writing, Computer Oral Presentation Math
  Competencies
• Personal Development

Ed Popper
AACSB CIS September 23, 2002
15

Girard School of Business International
Commerce Merrimack College
Use of EBI Benchmarking Surveys

• Percent of Schools Using EBI Surveys As
  Indicators of Achieving Program Outcomes
• 42.9 For Undergraduate Programs
• 37.8 For MBA Programs
• EBI Surveys used
• 34.4 Use Undergraduate Exit Survey
• 11.3 Use Undergraduate Alumni Survey
• 26.4 Use MBA Exit Survey
• 5.7 Use MBA First Year Survey
• 5.7 Use Undergraduate Exit Surveys

Ed Popper
AACSB CIS September 23, 2002
16

Girard School of Business International
Commerce Merrimack College
Use of Student Exit Survey

• Percent of Schools Using Exit Surveys As
  Indicators of Achieving Program Outcomes
• 75.0 For Undergraduate Programs
• 50.0 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
17
Use Of Exit Surveys
Girard School of Business International
Commerce Merrimack College

• For Undergraduate Programs
• To Assess Student Satisfaction With
• 65.8 The University
• 74.4 The Business Unit
• 64.1 Their Major
• For Student Self-Assessment of Their Learning of
• 51.3 Overall General Education
• 64.1 The Business Core
• 48.7 Their Major

• For MBA Programs
• To Assess Student Satisfaction With
• 47.4 The University
• 89.5 The Business Unit
• 42.1 Their Major
• For Student Self-Assessment of Their Learning of
• 47.4 Overall
• 73.7 The Business Core
• 36.8 Their Major

(As a percent of Schools that use a exit surveys
for program assessment)
Ed Popper
AACSB CIS September 23, 2002
18

Girard School of Business International
Commerce Merrimack College
Use of Other Surveys

• Percent of Schools Using Alumni Surveys As
  Indicators of Achieving Program Outcomes
• 65.3 For Undergraduate Programs
• 65.7 For MBA Programs
• Percent of Schools Using Surveys of Their
  Graduates Employers As Indicators of Achieving
  Program Outcomes
• 34.0 For Undergraduate Programs
• 23.5 For MBA Programs

Ed Popper
AACSB CIS September 23, 2002
19

Girard School of Business International
Commerce Merrimack College
Number of Assessment Methods Used

• 82 Of Undergraduate Programs Use Four or More
  Assessment Methods
• 71 Of MBA Programs Use Four or More Assessment
  Methods

Ed Popper
AACSB CIS September 23, 2002
20

Girard School of Business International
Commerce Merrimack College
Other Assessment Tools Used

• For Undergraduate Programs (35.3)
• Surveys / Feedback Internship/Coop Employers
• Outside Assessors for Final Project
• Current Student Surveys/Focus Groups
• Job Placement Data
• Enrollment Management Data
• Subject Specific Evaluations (Writing,
  Accounting, Quant.)
• Modified Job Diagnostic
• For MBA Programs (21.6)
• MBA Directors Forum Consortium Survey (Widener
  University)
• Current Student Surveys/Focus Groups
• Surveys / Feedback Internship/Coop Employers
• Alumni Correspondence, Employer Feedback,
  Advisory Council
• Modified Job Diagnostic

Ed Popper
AACSB CIS September 23, 2002
21
Use Of Outcome Assessment Results
Girard School of Business International
Commerce Merrimack College

• Who Sees Assessment Results
• 37.7 University Administration
  (President, Provost)
• 92.5 Deans Department Chairs
• 90.6 Business School Faculty
• 43.4 Advisory Boards
• 11.3 Students

• Impact of Outcome Assessment
• 88.7 Course Revisions
• 79.2 Minor Curriculum Change
• 56.6 Major Curriculum Change
• 35.8 Faculty Assessment (tenure,
  promotion, salary)
• 17.0 Faculty Recruiting

(As a percent of Schools responding)
Ed Popper
AACSB CIS September 23, 2002
22

Girard School of Business International
Commerce Merrimack College
Interest in Learning Outcome Assessment
Consortium 82.6 of Schools Responding
Ed Popper
AACSB CIS September 23, 2002