Learning Outcomes Workshop

1
Creating a Plan for Measuring Outcomes
2

• Learning Outcomes Workshop
• Crafton Hills College
• November 21, 2003
• Rick Hogrefe
• Assistant Professor, Speech
• Crafton Hills College

2
What is Outcomes Assessment?

• An ongoing process aimed at understanding and
  improving student learning.
• formative rather than summative in nature
• focuses on learning that endures, but that can be
  measured in some form now
• directed toward an assessment of a specific
  course or entire program

3
Process of Outcomes Assessment
1. Identify Learning Outcomes
5. Use Results for Improvement
2. Identify/Create Assessment Tools
Student Learning

• Choose/Develop
• Evaluation Criteria

4. Conduct Assessment
4
ChoosingAssessment Tools (Part 1)

• Use of embedded assessment, assignments and
  methods of evaluation already part of a course or
  programs curriculum used as a means of measuring
  student learning.
• OR
• Creation of new techniques for measuring student
  learning.

5
ChoosingAssessment Tools (Part 2)

• Assessment techniques which provide direct
  evidence of student learning.
• OR
• Assessment techniques which provide
  indirect/supportive evidence of student learning.

6
ChoosingAssessment Tools (Part 3)

• Assessment techniques which measure cognitive
  outcomes.
• OR
• Assessment techniques which measure attitudinal
  outcomes.
• OR
• Assessment techniques which measure behavioral
  outcomes.

7
Characteristics of an Assessment Plan

• The plan comprehensively assesses all learning
  outcomes.
• The tools chosen are appropriate to the learning
  outcomes.
• The evaluation criteria are clear and
  consistently applied regardless of section or
  instructor.
• Multiple methods of assessment are used.

8
Assessment Tools

• Standardized Tests
• Common Exams
• Common Questions
• Portfolios
• Projects
• Writing Samples
• Oral Presentations
• Capstone Exercises

• Classroom Assessment Techniques
• Student Surveys and Interviews
• Focus Groups
• Employer Evaluations
• Alumni Surveys
• University Faculty Surveys
• Licensure and Certification Information

9
Embedded Assessment

• CHEMISTRY
• Kim Salt, Ph.D.

10
The Assessment ofthe Outcomes

• No single method is perfect
• For any outcome, multiple indices can be used
• Assessment methods can differ radically across
  disciplines.
• Not all students need to be assessed. In large
  classes, a random sampling of students is
  sufficient

11
Course Embedded Assessment

• Shared midterms or finals
• Shared embedded questions within certain exams
• Random selection of student essays at beginning
  and end of courses to measure gains in skills
• Collection of data and analysis of what it means

12
Assessment Plan Development

• Specify the intended outcome (results oriented,
  easily understood, measurable).
• Identify related course(s).
• Describe assessment measure(s) and technique(s).
• Establish assessment criteria (challenging, yet
  attainable).

13
Course EmbeddedAssessment Process

• Choose tools that evaluate the intended outcomes
• Choose tools that yield manageable information.
  Dont aim for perfection, aim for a little
  knowledge that can help you.
• Use or modify existing course assignments
• Choose tools that assess more than one outcome at
  a time
• The local measure of grades by themselves do
  not ensure that a student has achieved a
  particular outcome however, grades on tests or
  papers can be used if they are designed to assess
  a particular competency.

14
Assessment Measures/Techniques

• Measure
• Questions Embedded in Exams
• Technique
• Evaluation of Common Exam Questions Across Course
  Sections
• Measure
• Essay, Research Paper, Oral Presentations
• Technique
• Use of Rubrics to Evaluate Student Performance in
  Various Knowledge/Skills Areas

15
Program Assessment Plan Shell

• Intended Outcome
• Students will be able to solve stoichiometry
  problems involving amounts of one or two
  reactants in solid, gaseous and aqueous states.
• Related Course(s)
• Chemistry 101
• Chemistry 150
• Assessment Measure(s) and Technique(s)
• 5 problems involving the solving of stoichiometry
  problems will be embedded in the final
  examination of these courses.
• Assessment Criteria
• At least 85 of students completing the final
  examination will respond correctly to 80 or more
  of the embedded questions.

16
Chemistry 101 Outcomes
17
Chemistry 101 Outcomes
18
Assessment Measures

• Scoring Rubrics
• scoring tool that lists criteria
• specify the level of performance expected for
  different levels of quality
• levels of quality given ratings (eg excellent,
  good, poor) or numerical values (eg 5, 4, 3 etc)

19
Rubric Development

• Identify the intended outcome.
• Specify the indicator for successful performance.
  Include various performance levels from poorest
  to best.
• Assign ratings to performance levels.
• Devise a scoring procedure.

20
Program Assessment Plan Shell

• Intended Outcome
• Students will demonstrate the ability to design a
  laboratory experiment, collect and analyze data
  for the experiment, and report the results in a
  formal scientific report format.
• Related Course(s)
• Chemistry 151

21
Program Assessment Plan Shell

• Assessment Measure(s) and Technique(s)
• As a course requirement, students will devise an
  experimental procedure, use the procedure to
  collect laboratory data, analyze the data and
  write formal scientific report following a set of
  guidelines. Reports will be evaluated using a
  checklist determining the students development
  of an appropriate experimental procedure,
  collection of data, analysis of data, and use of
  appropriate report format.
• Assessment Criteria
• 80 of students evaluated will have used an
  appropriate experimental procedure, collected
  appropriate data, analyzed the data in an
  appropriate manner, and used the appropriate
  report format.

22
Portfolios

• CHILD DEVELOPMENT
• JoAnn Jones, M.Ed.

23
Portfolios

• Educators of teachers have two essential ethical
  and legal responsibilities. One is to support
  the development of the teachers with whom they
  work. The other is fundamentally one of
  accountability and plays itself out in policy
  arrangements between the state and teacher
  education institutions. These two
  responsibilities of any teacher education program
  contain an inherent tension How to provide
  supportive opportunities for learning while
  simultaneously being accountable to standards set
  forth by licensing agencies and the profession
  they represent.
• -Snyder, Lippincott Bower, 1998

24
Portfolio Types

• Child
• Undergraduate college students
• Student teachers
• New teachers
• Tenured teachers
• University level educators

25
Portfolios as a Concept

• Facilitate the development of reflective thinking
• Present a holistic view of your achievements
• Provide an ongoing record of your accomplishments
• Place responsibility on the individual to to
  develop a plan for their goals
• Documents valid achievements
• Enhances professionalism and interview processing

26
Website Assistance for Portfolios

• http//amby.com/kimeldorf/sampler/html
• http//www/bradley.edu/otefd/Library/Teacher-Portf
  olio.html
• http//www.cll.wayne.edu/fls/Teachptf.htm

27
Portfolios at CHC

• Portfolios begin in CD 100.
• Resumes, letter of intent, etc. are included.
• Portfolios are a collective effort from many
  classes (Philosophy from CD 112)
• Curriculum CD 132, CD 134 etc.
• Portfolios are given a final grade in CD 205x2

28
Portfolios

• Can represent individuals
• Can represent particular courses
• Can represent programs

29
Surveys

• SPEECH
• Rick Hogrefe, M.A.

30
Exercise

• In groups, brainstorm a preliminary plan about
  how you could already next semester begin a
  process of measuring student learning. Base your
  responses on the specific learning outcomes of
  your course or program.

31
Thank You!

• For more information, please contact
• Rick Hogrefe
• Assistant Professor, Speech
• Crafton Hills College
• [phone]
• rhogrefe_at_crafton.sbccd.cc.ca.us