Effective Assessment: Goal- and Objective-Setting and Curriculum Mapping

1
Effective Assessment Goal- and Objective-Setting
and Curriculum Mapping
2
Some Important Assessment Basics

• Establishing congruence among institutional
  goals, programmatic and course objectives,
  learning opportunities, and assessments
• Distinguishing between goals and objectives
• Using a variety of measures, both quantitative
  and qualitative, in search of convergence
• Value of course-embedded assessment

3
Goals and Objectives Whats the Difference?

• Goals are broader, more general, and
  non-measurable as stated
• Students will gain an understanding of the
  scientific method.
• Objectives are narrower, more specific, and at
  least translatable into a measure
• At the program level Students will demonstrate
  the ability to formulate hypotheses, analyze
  data, and reach conclusions.
• At the course level Students will be able to
  perform an analysis of variance and interpret its
  results.

4
Advantages of Course-Embedded Assessment

• Least time- and labor-intensive
• Direct, necessary involvement of faculty
• Student motivation assured
• Face validity of measures assured (i.e.,
  authentic assessment)
• And, most important, its implications for
  immediate and direct feedback to individual
  faculty (and, therefore, for closing the loop)

5
Assessments Four Steps

• Setting objectives What you say you do
• Curriculum mapping How you do what you say you
  do
• Assessment How you know you are doing what you
  say you do
• Closing the loop What you do next based on
  results

6
Programmatic Objectives
7
Developing Programmatic Objectives Some General
Suggestions

• Involve all faculty teaching in program
• Program objectives should reflect institutional
  and program Mission Statement
• Best objectives result from faculty-negotiated
  agreement about what students in the program
  should be like upon completing program
• Focus on five or so core objectives to begin with
  thats plenty!

8
Developing Program Objectives Basic Questions
in Getting Started

• What do you expect of students in terms of
  knowledge, skills, behavior, and attitudes?
• What achievements do you expect of graduates in
  your field?
• What profiles of your alumni do you have, or can
  you develop in terms of issues you believe are
  important?

9
Developing Program Objectives Some Typical
Areas of Interest

• Knowledge of content
• Communication ability (written and oral)
• Information literacy ability (library use and
  computer proficiency)
• Quantitative reasoning
• Critical thinking
• Analytic and interpretative ability

10
Specific Guidelines for Setting Program
Objectives

• Three Basic Rules

11
Rule 1 Identify Overarching Concepts, Not
Course-Level Objectives

• Good Example Students will demonstrate the
  ability to formulate hypotheses, analyze data,
  and draw conclusions.
• Poor Example Students will demonstrate the
  ability to perform an ANOVA.

12
Rule 2 State Objectives Using Concrete
Language and Action Verbs

• Good Example Students will acquire and
  demonstrate knowledge and skills necessary to
  solve complex business problems in one or more
  areas of emphasis.
• Poor Example Our objective is to enhance
  students intellectual growth.

13
Rule 3 Focus on Results, Not Process

• Good Example Students will demonstrate clear
  and effective oral communication skills.
• Poor Example Students will successfully
  complete four Oral Intensive courses.

14
Course Objectives
15
Course Objectives Should

• Reflect program goals and objectives
• Be developed within the context of the program
  and, ideally, involve all faculty who teach the
  course
• Be more concrete and specific than program
  objectives
• Be measurable
• Be included on course syllabi

16
Questions To Ask

1. Which program objectives are most appropriately
   covered in your course? (no course can or need
   do everything!)
2. How can you effectively translate the program
   objectives into course-level objectives?
3. What specific activities do you provide to your
   students that enable them to achieve these
   objectives?
4. What specific assignments enable you to determine
   their level of achievement?
5. Based on the outcomes, what can you conclude
   about that level of achievement, and do you need
   to do anything differently the next time?

17
Categories of Course Objectives

• Cognitive what do you want students to know?
• Behavioral what competencies do you want them
  to demonstrate?
• Attitudinal are their particular values you
  want them to adopt?

18
Sample Program Objectives Sociology Department

• Student understands and can explain major
  theories of social behavior.
• Student understands the nature and purposes of
  social research and understands different
  methodological techniques.
• Student can apply theories and research methods
  to real-world situations.
• Student can describe these issues effectively in
  oral and written form.

19
Translating Programmatic Objectives into Course
Objectives (Handout 1)

• Value of fairly broad student learning goals that
  can then be operationalized more specifically at
  the course level (especially when different
  courses involved)
• Importance of making sure that compound
  objectives can be broken down into discrete
  parts, each of which, when assessed, yields a
  distinct sub-score (holistic scoring less useful)
  
• Again, individual courses need to do everything
  what is important is that, when all courses are
  considered overall, all programmatic objectives
  are covered adequately
• Common technique for determining this
  Curriculum mapping

20
Curriculum Mapping Assessments Second Step

• Matching Programmatic Objectives to Curricular
  Activities

21
Multiple Benefits of Curriculum Mapping

• Increased clarity as to extent to which and
  where programmatic objectives are being covered
  and accomplished
• Increased awareness by faculty of their and
  others responsibilities in delivering the
  curriculum, as well as a better understanding of
  the entire program
• Multiple opportunities for establishing consensus
  about the curriculum as well as faculty ownership
  and contributions
• Positive implications for developing a
  comprehensive assessment database

22
Basic Steps in Curriculum Mapping

• Involve all faculty teaching in program
• Survey faculty with respect to their coverage of
  learning objectives
• Share information with faculty for review and
  discussion
• Reach consensus regarding extent to which program
  is addressing objectives adequately and develop
  strategies for change as necessary

23
Maximizing Information Gained Through Curriculum
Mapping

• Have faculty indicate the extent to which they
  cover the learning goals for each course they
  teach
• And, while youre at it, survey if they are
  assessing students mastery of the objectives
• If so, have them indicate the type of measure
  they are using, and even the specific assessment
  activity being utilized

24
Sample Form for Collecting Course Information
(Handout 2)
25
Hypothetical Example Bachelor of Music
Composition Program (Handout 3)

• Grid reveals much useful information about
  curriculum
• Are all goals being covered adequately?
• Are there gaps in coverage of goals?
• Are some goals being covered excessively?
• Are different sections of courses providing the
  same emphases?
• Process also yields rich assessment database
  for determining program effectiveness

26
Summarizing the Benefits of Curriculum Mapping

• Effective tool for consensus- and
  community-building in a department or program
• Promotes holistic perspective of a curriculum
• Clarifies relationships between courses (e.g.,
  course sections, prerequisites)
• Can result in prolific assessment database
  through extraction of assessment products