Developing CourseLevel Learning Outcomes: Enhancing Learning Through Shared Expectations

1
Developing Course-Level Learning Outcomes
Enhancing Learning Through Shared Expectations

• A Workshop Hosted by
• The College of Arts and Science,
• the Thompson Center for Learning and Teaching,
  and
• the Assistant Provost for Assessment and
  Institutional Accreditation
• December 1, 2006
• University of Michigan-Flint

2
Agenda

• Introduction (Definitions and Objectives)
• Working Lunch (Critique and Share)
• Writing Well Stated Outcomes
• One Framework Blooms Taxonomy
• Independent Work 1
• Articulating your course objectives and outcomes
• Course Alignment
• Independent Work 2
• Specifying Teaching Learning Activities
• Stating Assessments of Student Learning
• Beyond Course Alignment

3
Why transform to a language of assessment?

• Specific learning outcomes lead to
• More measurable outcomes
• Better assessment
• Higher quality feedback
• Improved courses and programs
• Improved student learning and achievement

4
Alignment Within Courses
5
The Language of Assessment

• Outcomes Based Assessment A process by which
  you
• determine the indicators of an effective program,
  
• use those indicators as criteria for assessing
  the program, and
• apply the results of the assessment toward the
  ongoing and continuous improvement of the
  program.

6
Objectives vs. Outcomes1

• Program/course objectives are general goals that
  define what it means to be an effective
  program/course. They are general, indefinite,
  and not intended to be measured. They set the
  overall agenda for the program/course.
• Student learning outcomes are specific results
  the program/course seeks to achieve in order to
  attain the general goals defined in the
  objectives. Outcomes are definite and intended
  to be measured. They establish the particular
  means by which the agenda (as defined by
  objectives) is achieved. The achievement of
  outcomes is evidence that our students are
  learning.

7
Direct vs. Indirect Assessment1

• Direct assessment of learning gathers evidence,
  based on student performance, which demonstrates
  the learning itself.
• Examples most classroom testing for grades or
  evaluation of a research paper on specific
  criteria
• Indirect assessment of learning gathers
  reflection about the learning or secondary
  evidence of its existence.
• Examples student, alumni, employer surveys

8
Assessment is not always an add-on1

• Embedded assessment a means of gathering
  information about student learning that is built
  into, and is a natural part of the
  teaching-learning process.
• Example as part of a course, expecting each
  senior to complete a research paper that is
  graded for content and style, but is also
  assessed for advanced ability to locate and
  evaluate Web-based information (as part of a
  program level, or a college-wide outcome to
  demonstrate information literacy).

9
Formative vs. Summative Assessment 1

• Formative assessment the gathering of
  information about student learning - during the
  progression of a course or program and usually
  repeatedly - to improve the learning of current
  students.
• Summative assessment the gathering of
  information at the conclusion of a course,
  program, or undergraduate career to improve
  learning of the next cohort of students or to
  meet accountability demands.

10
Accountability and Improvement1

• Assessment for accountability assessment of some
  unit (could be a department, program or entire
  institution) to satisfy stakeholders external to
  the unit itself. Results are often compared
  across units, compared to state and national
  norms, and always summative.
• Assessment for improvement assessment that feeds
  directly, and often immediately, back into
  revising the course, program or institution to
  improve student learning results. This can be
  formative or summative.

11
Levels of Assessment1

• Assessment of individuals uses the individual
  student, and his/her learning, as the level of
  analysis.
• Assessment of programs uses the department or
  program as the level of analysis. Ideally
  program goals and objectives would serve as the
  basis for the assessment.
• Assessment of institutions uses the institution
  as the level of analysis. Ideally,
  institution-wide goals and objectives would serve
  as a basis for the assessment. At this level it
  is essential to examine institutional documents
  such as mission and vision statements, as well as
  strategic plans.

12
Lunch Activity

• Share your experiences with writing learning
  outcomes for courses or programs with those at
  your table.
• Describe questions/challenges you face
• Review the syllabi provided
• What do you like/not like with respect to the
  learning outcomes in each syllabus?
• Use Worksheet 1 (blue) to make individual
  observations and the same worksheet (buff) for
  collective observations

13
Characteristics of Well Stated Learning Outcomes2

• student-focused rather than professor focused
• focused on the learning resulting from an
  activity rather than on the activity itself
• focused on skills and abilities central to the
  discipline and based on professional standards of
  excellence
• general enough to capture important learning but
  clear and specific enough to be measurable
• focused on aspects of learning that will develop
  and endure but that can be assessed in some form
  now

14
Common Problems with Learning Outcomes

• Using vague terms, such as
• Appreciate
• Become aware of
• Become familiar with
• Develop
• Know
• Learn
• Understand
• Describing action taken by someone other than the
  learner.
• The program will... or
• The course will

15
A Comparison of Poorly and Well Stated Outcomes

• Students will understand Eriksons developmental
  stages.
• Students will be familiar with the major
  sociological perspectives and how they relate to
  their daily lives.
• Students will develop the skills necessary for
  conducting research in the natural sciences.

• Students will identify and summarize each of
  Eriksons stages of development.
• Students will describe each of the major
  sociological perspectives and will illustrate how
  each perspective relates to events in their daily
  lives.
• Students will design, conduct, and analyze a
  research project using appropriate scientific
  theory and methodology

16
Why Classify Learning Outcomes?

• All learning outcomes are not developed,
  delivered, or measured equally

17
Learning Domains

• Three primary domains for classifying educational
  goals
• Cognitive (knowledge)
• Affective (attitudes)
• Psychomotor (skills)

18
Blooms Taxonomy of Educational Objectives3
(Cognitive domain)

• A now classic system that classifies educational
  goals to facilitate the development and
  evaluation of college and university curricula.
• A hierarchical taxonomy of student behaviors that
  reflect the development of increasingly complex
  cognitive abilities and skills as a result of
  instructional experiences.

19
Blooms Taxonomy of Cognitive Levels

• There are 6 categories, listed hierarchically
  from simplest to most complex
• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

20
Blooms Taxonomy Knowledge4

• Definition
• ability to remember information from simple
  (facts, terminology) to more complex/abstract
  (theories, principles)
• Student Learning Verbs
• List, name, identify, show, define, recognize,
  recall, state, describe, label, match, outline,
  reproduce, select

21
Blooms Taxonomy Knowledge4

• Examples Knowledge of dates, events, places,
  major ideas, and mastery of subject matter
• The student will
• Define the 6 levels of Blooms taxonomy of the
  cognitive domain

22
Blooms Taxonomy Comprehension4

• Definition
• ability to understand material at a level
  sufficient for grasping its meaning and inferring
  its implications
• Translating, comprehending, or interpreting
  information based on prior learning
• Student Learning Verbs
• Summarize, explain, interpret, describe, compare,
  paraphrase, differentiate, demonstrate, restate,
  illustrate

23
Blooms Taxonomy Comprehension4

• Examples Translates knowledge into next context,
  interprets facts, compare and contrast, order,
  group, infer, predict
• The student will
• Explain the purpose of Blooms taxonomy of the
  cognitive domain

24
Blooms Taxonomy Application4

• Definition
• ability to correctly and independently bring to
  bear abstractions (e.g., theories, principles,
  methods) in solving concrete problems
• The selection, transfer, and use of data and
  principles to complete a task with a minimum of
  direction
• Student Learning Verbs
• Solve, illustrate, calculate, compute, use,
  interpret, relate, manipulate, apply, classify,
  modify, demonstrate, construct, discover, predict

25
Blooms Taxonomy Application4

• Examples Use information, methods, concepts or
  theories in new situations, solve problems using
  required skills or knowledge
• The student will
• Write an instructional objective for each level
  of Blooms taxonomy of the cognitive domain

26
Blooms Taxonomy Analysis4

• Definition
• ability to parse information into is constituent
  elements and to identify the relationships
  between those elements
• Student distinguishes, classifies, and relates
  the assumptions hypotheses, evidence, or
  structure of a statement or question
• Student Learning Verbs
• Analyze, organize, categorize, deduce, choose,
  contrast, compare, distinguish, separate,
  differentiate, discriminate

27
Blooms Taxonomy Analysis4

• Examples Seeing patterns, organization of
  parts, recognition of hidden meanings,
  identification of components
• The student will
• Compare and contrast the cognitive and affective
  domains as specified by Bloom

28
Blooms Taxonomy Synthesis4

• Definition
• ability to combine elements into new wholes
  (e.g., ideas, plans of action, abstract
  relations) that are more than the sums of their
  respective parts
• Student originates, integrates, and combines
  ideas into a product, plan, or proposal that is
  new to him or her.
• Student Learning Verbs
• Design, create, hypothesize, invent, develop,
  support, schematize, write, report, discuss,
  plan, devise, compare, construct, compose,
  generate

29
Blooms Taxonomy Synthesis4

• Examples Use old ideas to create new ones,
  generalize from given facts, relate knowledge
  from several areas
• The student will
• Design a classification scheme for writing
  educational objectives that combines the
  cognitive, affective, and psychomotor domains.

30
Blooms Taxonomy Evaluation4

• Definition
• Ability to offer quantitative and qualitative
  judgments about the value of ideas and methods
• Student appraises, assesses, or critiques on a
  basis of specific standards and criteria
• Student Learning Verbs
• Evaluate, choose, estimate, judge, defend,
  criticize, justify, recommend, critique,
  interpret, support

31
Blooms Taxonomy Evaluation4

• Examples Compare and discriminate between
  ideas, assess value of theories or presentations,
  make choices based on reasoned argument, verify
  value of evidence, recognize subjectivity
• The student will
• Judge the effectiveness of writing objectives
  using Blooms taxonomy

32
Learning Outcomes by Blooms Taxonomy
Bloom's Taxonomy of Cognitive Categories
Student Learning Outcomes
Unit/Lesson Learning Outcomes
Course Goals/Objectives
Eval-uation
Appli-cation
Com-prehen-sion
Know-ledge
Syn-thesis
Anal-ysis
1.a.1. Define the three tenets of the Central
Limit Theorem
 
 
 
 
 X
1. Introduce students to inferential statistics
1.a learn the conceptual foundations of inference
 
 
 
 
 
 X
 
1.a.2. Describe three key distributions
 
 
 
 
 
 X
1.a.2. Combine to explain the relationship
between the three distributions
 
 
 
 
 
 
1.b.1. Outcome
 1.b Apply to confidence intervals
 
 
 
 
 
 
1.c.1. Outcome
 1.c Test for difference between means
 
 
 
 
 
 
 2. Introduce students to descriptive statistics
 2.a Outcome
2.a.1. Outcome
 
 
 
 
 
 
 
2.b.1. Outcome
 2.b Outcome
Note While this worksheet accommodates 7
learning outcomes, your specific course will most
likely have more than this single worksheet can
accommodate. The purpose of the worksheet is to
provide a framework and not set parameters.
33
Alignment within a Given Course
Course-Level Learning Outcomes
Teaching and Learning Activities
Assessments of Student Learning
34
The Next Step

• Given your student learning outcomes, what
  specific tasks or activities will you have
  students complete to promote learning?
• Given these student learning outcomes, how will
  you know when your students have achieved the
  outcomes for that lesson or course (what
  assessments will you use)?

35
Teaching Learning Activities by Outcomes and
Blooms Taxonomy
Student Learning Outcomes
Bloom's Taxonomy of Cognitive Categories
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
Class lecture and students will read assigned
chapter.
1.a.1 Define the three tenets of the Central
Limit Theorem
Class lecture and students will read assigned
chapter.
1.a.2. Describe three key distributions
In class, students will calculate sample means
and construct a sampling distribution. Homework
will reinforce lesson.
1.a.3 Combine to explain the relationship
between the three distributions
1.b.1 Outcome
1.c.1 Outcome
2.a.1 Outcome
2.b.1 Outcome
36
Assessments by Outcomes and Blooms Taxonomy
Student Learning Outcomes
Bloom's Taxonomy of Cognitive Categories
Evaluation
Synthesis
Analysis
Application
Comprehension
Knowledge
1.a.1 Define the three tenets of the Central
Limit Theorem
Homework and examination.
1.a.2. Describe three key distributions
Homework and examination.
1.a.3 Combine to explain the relationship
between the three distributions
Homework and examination.
1.b.1 Outcome
1.c.1 Outcome
2.a.1 Outcome
2.b.1 Outcome
37
Beyond Course Alignment

• Thinking Globally Alignment of course level
  learning outcomes
• Sequentially
• How do my outcomes fit with those of more
  advanced courses?
• What do I expect students to have learned by the
  time they when they enter this class?
• Programmatically
• How do all of the courses within a programs
  curriculum fit together?
• How do the program curricula contribute to the
  institutional outcomes?

38
Alignment Between Course Outcomes and
Institutional Outcomes
39
Want to learn more?

• Plan to participate in the Assessment Summit,
  January 12, 2007

40
References

• 1 Leskes (2002) Beyond Confusion An Assessment
  Glossary, Peer Review.
• 2 Huba and Freed. 2000. Learner-Centered
  Assessment on College Campuses.
• 3 Bloom.1956. Taxonomy of Educational
  Objectives, Handbook I The Cognitive Domain
• 4 From Don Clark, http//www.nwlink.com/dpmcarl
  /hrd/bloom.html, the Learning Skills Program at
  the University of Victoria (htttp//www.coun.uvic.
  ca/learn/program/hndouts/bloom.html) , the
  Faculty Roles and Rewards Program at Portland
  State University (http//edtech.clas.pdx.edu/prese
  ntations/frr99/blooms.htm), and W. Huitt,
  http//chiron.valdosta.edu/whuitt/col/cogsys/bloom
  .html.