McCoy College of Business Administration (MBA Program)

1
McCoy College of Business Administration (MBA
Program)

• Why Assessment?
• We give grades dont we?
• What is an Assessment Program?
• What can we do?
• How do we do it?
• Special thanks to Dr. Doug Eder and Dr. Kathryn
  Martell for much of the content in this
  presentation.

2
The First and Only Goal

• To teach for long-term retention of information
  and application to new situations
• --after Halpern Hakel

3

• Deep (expert) learning is a curriculum function.
• It is not a course function.

Universities are systems
4
Direct vs. Indirect Assessment

• Direct assessment acquires evidence about student
  learning and the learning environment Exams,
  projects, logs, portfolios, observations....
• Indirect assessment acquires evidence about how
  students feel about learning and their learning
  environment Surveys, questionnaires, interviews,
  focus groups, reflective essays....

5
Whatever Assessment Is...

• ...Its Multiple Measures Over Time.

6
Why Assessment Wont Go Away

• Limits on money
• Shifts in public view of Higher Education
• State laws and education standards
• Accountability to accrediting agencies to
  maintain operating credentials
• Result We must demonstrate quality of outcome
  and return on investment.
• Through assessment educators meet
  responsibilities to themselves, to students, and
  to the public.

7
What is Program-Level assessment as specified
by AACSB?

• The college or school must specify program-level
  learning goals for each separate degree program.
  Generally, such goals are anticipated for each
  degree, not for separate majors or concentrations
  within a degree. For example, the McCoy College
  offers a Bachelor of Business Administration
  (BBA) degree with defined majors in accounting,
  CIS, economics, finance, management, and
  marketing. A set of program-level learning goals
  for the BBA degree must be provided and assessed
  goals for each major will not be required for
  AACSB program review accreditation. The MBA
  program has program-level learning goals
  reflecting graduate-level learning outcomes.

8
Strongest LinksThe Scholarship of Teaching

• Clear Goals
• Adequate Preparation
• Appropriate Methods
• Significant Results
• Effective Presentation
• Reflective Critique

9
Assessment vs. Evaluation

• Assessment focuses on the student and the
  learning environment.
• Evaluation focuses on the professor and the
  teaching performance.

10
We Give Grades, Dont We?

• Arent grades
• (by themselves)
• enough?

NO!
11
When used alone, grades are no longer regarded as
adequate indicators of student learning.

• Professor autonomy Grades in one course or
  section may be recorded using a different
  standard than grades in another.
• General accountability Much of the current
  public annoyance with higher education comes from
  a lack of skill and content mastery by students
  who have received good grades.

12
Grades may reflect many things besides student
mastery of course objectives

• Verbal ability
• Participation
• Cooperation
• Extra credit
• Attendance
• Effort
• Criterion Performance vs. Value Added

13
Good Assessment Asks Good Questions

• Do we have a curriculum or a set of common
  courses? Which of these do our students
  experience?
• What do we want our students to be or to have
  when they have completed the program? What are
  our curricular goals?
• What do our students do to demonstrate
  accomplishment of curricular goals (i.e. course
  objectives)? How much and what kinds of writing?
  What kinds of math? Oral Presentations? Analysis
  and Reports? Are these relevant to our curricular
  goals?
• How do we demonstrate to ourselves that students
  are achieving or have achieved curricular goals?

14
Graduate Common Core Courses

• 1. MGT 5314 Organizational Behavior and Theory
• 2. ECO 5316 Managerial Economics
• 3. MKT 5321 Marketing Management
• 4. QMST 5334 Advanced Statistical Methods for
  Business
• 5. ACC 5361 Financial and Managerial Reporting
  and Analysis
• 6. FIN 5387 Managerial Finance
• 7. CIS 5354 Decision Support Models in
  Management
• 8. MGT. 5313 Administrative Policy

15
GRADUATE LEARNING GOALS (MBA)Upon completion of
the Master of Business Administration (MBA), a
student will be able to

• 1. Integrate knowledge of fundamental business
  disciplines to effectively manage domestic and
  global organizations in a dynamic environment.
• 2. Integrate appropriate information technologies
  for managing business data for decision making,
  enhancing productivity, and communicating with
  others.
• 3. Evaluate analytical skills and critical
  thinking processes used in business decision
  making.
• 4. Evaluate the issues associated with ethical
  leadership and conducting business in an ethical,
  legal, and socially responsible manner.
• 5. Demonstrate the ability to communicate
  effectively, both orally and in writing, in new
  and unfamiliar circumstances.
• 6. Apply the skills needed to effectively lead
  and contribute to dynamic workgroups.

16
An Assessment Question How Do You Know...

• ...that students walk out your door looking like
  you want them to? What behaviors have they
  exhibited or products have they produced? What
  are the indicators for your goals?

17
Goals and Objectivesfor Students

• Goals express what we want our students to be.
• Objectives describe what we want our students to
  do.
• .....

• Objectives are indicators of goals.

18
Graduate Faculty Teaching Common Core Courses
Should

• Talk to each other (if multiple faculty teach the
  course) . . .
• Agree on specific objectives for the course . . .
• Make sure course objectives are relevant to
  Program-Level goals . .

19
To be assessment friendly, objectives should

• Focus on students
• Make the learning goals visible (serve as
  indicators)
• Describe behaviors or products (doing, making)
  that can be captured by assignments.

20
Blooms Taxonomy

• LEVEL SOME COGNITIVE BEHAVIORS
• Evaluation Appraisal of an Analysis
• or Synthesis
• Synthesis Assembly of Application
• Analysis Disassembly of Application
• Application Use of Understanding
• Understanding Management of Knowledge
• Knowledge Memorization of facts, language,
  concepts, principles, theories

21
The Next Step . . .

• Graduate Faculty Should Agree on How Best to
  Measure Student Performance on each Objective
  Contributing to Program-Level Goals.

22
Some Assessment Ways and Means

• Assessment days or centers
• Case studies
• Classroom assessments
• Completion and retention studies
• Content analyses
• Debates
• Direct observations
• Focus groups
• Graduate success
• Internships and service learning
• Interviews (including videotapes)
• Exams for certification and licensure

• Matrices
• Performances
• Portfolios of several kinds
• Projects (Primary Trait Analysis)
• Questionnaires and surveys (Direct and telephone
  employer, alumni, and student attitude and
  satisfaction)
• Reflective essays
• Study and activity logs
• Tests
• (Locally-developed and standardized)
• Transcript analyses

23
Another Step . . .

• Depending on the method (ways and means) used to
  assess a course objective, develop a rubric or a
  method of measurement.
• Evaluate student performance on the specific
  objective on three levels Exceeds Expectation
  Meets Expectation Does Not Meet Expectation.
• Report your findings.
• Discuss the result(s), determine any actions to
  take (if necessary).

24
Example Rubric for MGT. 4335
25
Responsibilities of Core Course Coordinators

• 1. completing the Course Alignment Matrix which
  specifies which program-level learning goals are
  covered in a specific core course,
• 2. developing course learning objectives that
  lead to fulfillment of program-level learning
  goals,
• 3. collecting data from faculty teaching course
  sections to support or measure student learning
  of program-level learning goals (including a
  representative sample of measurement
  devices/techniques/assignments to be maintained
  in the office of the core course coordinator),
• 4. reporting collective results on program-level
  learning goals to the McCoy College Assessment
  Committee, and
• 5. discussing and acting on specific
  recommendations made by the college assessment
  and curriculum committees.

26
(No Transcript)
27
(No Transcript)
28
What documentation must be retained by the Core
Course Coordinators?

• Core course coordinators should retain copies of
  instruments used for direct assessment of student
  learning, such as assignments, written products,
  test questions, rubrics, scoring grids, etc. The
  course coordinators also must retain sample
  student products relating to the instruments
  used. Aggregate results of direct assessments
  for various program-level learning goals gathered
  and/or submitted to the assessment committee
  should be retained by the course coordinator by
  semester or year for review. Minutes of core
  course faculty meetings should be retained with
  special emphasis given to items concerning
  improvements in instruments used to assess
  student learning and acknowledgement and review
  of data.

29
Must all students be assessed?

• For the purpose of meeting AACSB-International
  standards relating to Assurance of Learning (16,
  18 and 20), sampling may be utilized as long as
  it is representative. Faculty teaching college
  core courses for various programs, as scholars in
  their respective fields and under the leadership
  of the Core Course Coordinator, have the
  responsibility for determining appropriate
  representative sample sizes. Graduate core
  course coordinators may use census data from
  graduate core courses for assessment purposes.

30
Should college core course faculty and
coordinators gather data and submit results every
year?

• Program-level learning goals for undergraduate
  (BBA) and graduate (MBA) programs are reviewed on
  a scheduled rotational basis. At least two
  program goals are reviewed each year, and college
  core courses covering that particular program
  goal are required to submit the results of
  course-embedded assessment to the assessment
  committee during the scheduled review year.
  Since the McCoy College views assessment as a
  continuous process, core course coordinators and
  core course faculty are encouraged, but not
  required, to gather data each semester for the
  reporting of assessment results.

31
Example Undergraduate Goal Rotation

• Program-Level Goal Rotation Sequence
• Academic Year 2005-2006
• Goal 1 Conceptualize a complex issue into a
  coherent, persuasive written or oral statement.
• Goal 5 Understand the importance of group
  dynamics in achieving organizational goals and
  use the skills needed for effective teamwork.
• Academic Year 2006-2007
• Goal 2 Use critical thinking skills to evaluate
  information, solve problems, and make sound
  decisions.
• Goal 3 Use information technology skills in
  decision-making.
• Academic Year 2007-2008
• Goal 4 Apply general concepts of ethical
  behavior in dealing with stakeholders.
• Goal 6 Understand the importance of culture and
  diversity.

32
We have never been asked...

• ...to assess everything all at once.

33
Who sets performance standards? How should Core
Course Coordinators report student performance on
various college-level learning goals?

• For each program-level learning goal assessed in
  a college core course, the faculty will determine
  their minimum expectation or standard. For
  reporting data on any specific program-level
  learning goal, core course coordinators should
  aggregate data and report the results by three
  levels Exceeds Expectations Meets Expectations
  or Does Not Meet Expectations. In order to
  preserve the integrity of the assessment process,
  core course coordinators should release aggregate
  data by course (all sections) and not individual
  results by the professor teaching the core
  course. Program Assessment in the McCoy College
  is not the evaluation of faculty performance in
  the classroom it concerns the aggregate level of
  student learning of various program-level goals.

34
Example Report on Program Goal
35
An Important Lesson from the Farm

• A pig doesnt get any fatter merely by weighing
  it.

36
How many students must meet or exceed the
performance standard on a college-level program
goal?

• There is no prescribed percentage of students
  that must meet the standard articulated by
  faculty. AACSB and other constituent groups will
  look at whether and how these data are used for
  continuous improvement purposes. Thus, a poor
  showing of student mastery of a given
  program-level learning goal would only be a
  concern if the curriculum was not subsequently
  modified to improve student skills/performance of
  that learning goal. A second emphasis in the
  Assurance of Learning Standards is that
  program-level learning goals are used to
  communicate competencies of graduates to
  students, employers and other constituents.
  Thus, faculty have the responsibility to develop
  and modify curricula to insure that program-level
  goals are achieved by most graduates and not
  just a portion.

37
What Assessment Asks of You!

• One day of initial development time to determine
  course objectives and agreeing on a grading
  rubric for specific course objectives.
• Two to three hours per semester (depending on the
  goal rotation) to extract student performance on
  course objectives leading to the fulfillment of
  program-level goals.
• Writing a short report (1 page) concerning
  student performance.
• Maintaining examples of student work, grading
  rubrics or other examples demonstrating student
  achievement of course objectives.
• Talking to other faculty about continuously
  improving classroom processes and products to
  improve student learning of program level goals
  over time.

38
Closing Thought

• The enemy of the good is the perfect.