Creating Effective, Viable Plans to Assess Student Learning

1
Creating Effective, Viable Plans to Assess
Student Learning
Pittsburgh Regional Assessment Workshops September
7-8, 2005

• Linda Suskie, Executive Associate Director
• Middle States Commission on Higher Education
• 3624 Market Street, Philadelphia PA 19104
• Web www.msche.org E-mail LSuskie_at_msche.org

2
Middle States Assessment Conference!

• September 29-30, 2005
• Baltimore Marriott Inner Harbor at Camden Yards
• Featured speakers
• Ginny Anderson
• Doug Eder
• Peter Gray
• Jeff Seybert
• Randy Swing
• Wendy Troxel
• Visit www.msche.org click on Events

3
This morning...

• What is assessment?
• Four steps of program assessment
• Learning goals
• Curricular alignment
• The assessment toolbox
• Setting targets for assessment results

4
What is Assessment of Student Learning?

• Deciding what we want our students to learn
• Making sure they learn it!
• --Jane Wolfson, Director, Environmental Science
  Studies Program, Towson University

5
The Teaching-Learning-Assessment Cycle
1. Learning Goals
4. Using Results
2. Learning Opportunities
3. Assessment
6
Learning Goals
7
What is a good learning goal?

• Outcomes what graduates should be able to DO
• Observable action words
• Understand
• Clear no fuzzy terms
• Demonstrate (how?), critical thinking,
  communication skills
• Skills and/or attitudes/values
• Important - meet student/employer needs

8
Examples of Learning Goals

• Chemistry Design an experiment to test a
  hypothesis or theory.
• Communication Apply knowledge of media history
  and the principles of media criticism to
  understand and evaluate media messages.
• Finance Identify financial problems, analyze
  their impact, and design solutions that will
  withstand critical examination.
• Theatre Express through clear writing and
  analysis a unique point of view on theatre
  subjects.
• Environmental Science Critically evaluate the
  effectiveness of agencies, organizations and
  programs addressing environmental problems.

9
More Examples

• Demonstrate effective communication skills across
  disciplines.
• Demonstrate understanding of other cultures.
• Demonstrate ability to work effectively with
  computer components.

10
Time to Think!

• Why do most colleges have a General Education
  requirement in the social sciences?
• Why is it a good idea for college students to
  study the social sciences?
• How should our students be different 5-10 years
  from now, because they completed this
  requirement?
• Draft three learning goals for a General
  Education requirement in the social sciences.

11
Time to Think Again!

• Draft three learning goals for one of your
  programs or Gen Ed requirements.
• Critique edit the goals of someone sitting next
  to you.
• Clear? No fuzzy words?
• Observable? Action words?
• Outcomes?

12
Align Your Goals Curriculum

• Will every degree student, regardless of course
  choices, have ample opportunity to achieve each
  goal?

13
Aligning goals curricula

• Review syllabi for learning outcomes.
• All course syllabi must include the courses
  learning outcomes!
• Create a grid of program goals x courses.
• Use syllabi or have faculty check off goals in
  their courses.
• Look for underemphasized overkilled goals.
• Consider modifying syllabi.
• Include relevant program-level learning outcomes.
• Grid of assignments x learning outcomes

14
The Assessment Toolbox
15
Look for information already on hand.

• Tests
• Papers, projects, performances
• Especially from capstones
• Field experience supervisors evaluations
• Surveys self-ratings

16
Create embedded tools.
17
Useful and not too much work

• Rubrics (rating scales/grading criteria)
• Test blueprints
• Reflective writing on attitudes values

18
Useful but more time work

• Additional tests
• Additional surveys
• Additional focus groups
• Additional portfolios

19
Understanding Assessment Results by Setting
Appropriate Targets
20
Michael earned 65 points on the midterm.

• Did he do well on the midterm?

21
To decide if Michael did well, we must compare
his 65 against something else.

• The something else depends on what we want the
  test to tell us.

22
Suppose 55 is passing and 70 is a perfect score.

• Standards-based perspective
• aka competency-based
• aka criterion-referenced
• Question answered
• Are my students meeting my standards?
• Challenge
• Establishing sound performance standards

23
Suppose the class average is 75.

• Benchmarking perspective
• aka peer-referenced
• aka norm-referenced
• Question answered
• How do my students compare to peers?
• Challenge
• Identifying appropriate peers collecting info
  from them

24
Suppose the university average is 75, but the
average at South University is 95.

• Best practice perspective
• aka best-in-class
• Question answered
• How do my students compare to the best of their
  peers?
• Challenges
• Commitment to improving teaching learning
• Identifying best-practice peers

25
Suppose Michael scored 35 a year ago.

• Value-added perspective
• aka growth, change, improvement, pre-post
• Question answered
• Are my students improving?
• Challenges
• Imprecise assessments mask growth.
• Motivating students on pre-test
• Is growth due to us?

26
Suppose class average is 75 now and 40 three
years ago.

• Longitudinal perspective
• Question answered
• Is my teaching curriculum improving?
• Challenge
• Using the same assessment

27
Suppose Michael is tone-deaf.

• Capability perspective
• aka potential
• Question answered
• Are my students doing as well as they can?
• Challenge
• Determining potential

28
Recap Six Questions that Assessments Can Answer

• Are my students meeting my standards?
• How do my students compare to their peers?
• How do my students compare to the best of their
  peers?
• Are my students improving?
• Is my teaching curriculum improving?
• Are my students doing as well as they can?

29
Which perspective should you use?

• Each has advantages and disadvantages.
• Each gives a somewhat incomplete picture.
• Multiple perspectives give the most balanced
  picture of student learning.