Closing the Assessment Loop

1
Closing the Assessment Loop

• Susan Hatfield
• Winona State University
• SHatfield_at_winona.edu

2
Closing the Loop

• Starts with a Pattern of Evidence
• Seeks to close the gap between actual and desired
  performance

3
Does not meet
Meets
Exceeds
Verbal Delivery
14
81
5
14
Nonverbal Delivery
21
74
79
Organization
14
7
Evidence
9
72
19
Transitions
7
85
8
Program Summary
4
Interpreting Data
ITEM 5
homegrown
transfer
2005
2006
2007
5
Student Learning Outcome
Components
Test Questions
1
1
2
3
4
2
2
5
6
7
8
9
3
3
10
11
12
3
4
17
13
14
15
16
4
6
Interpreting Patterns of Evidence

• Consistency - over time
• Consensus - different populations
• Distinctiveness - different situations/ variables
  / items

7
Consistency

• Examines the same practice of and individual or
  group over time
• Key question
• Has this person or group acted, felt, or
  performed this way in the past / over time?

8
Consistency
How well are students performing on
the departmental learning outcome measures?
High performance
Low performance
04
05
03
06
07
08
9
Consensus

• Comparison to or among groups of students
• Variation between disciplines, gender, other
  demographic variables
• Key questions
• What is the general feeling, outcome, attitude,
  behavior?
• Do other groups of people act, perform or feel
  this way?

10
Consensus
How well are students performing on
the departmental learning outcome measure?
High performance
Low performance
Females
Transfers
1st Generation
Males
11
Distinctiveness

• Examines individual or cohort perspectives across
  different outcomes
• Key Question
• Does a person or group perform equally as well on
  different outcomes?

12
Distinctiveness
How well are our students performing on the
learning outcomes?
S P E A K I N G
High Performance
A N A L Y S I S
T H I N K I N G
R E S E A R C H
E THICS
W R I T I N G
Low Performance
13
Interpreting Data
MALE
MEAN
FEMALE
CONSISTENCY
2004
2005
2006
14
Interpreting Data
C O N S E N S U S
MALE
MEAN
FEMALE
2004
2005
2006
15
Interpreting Data
D I S T I N C T I V E N E S S
16
Patterns of Evidence

• Who was involved in the discussion of the data?

17
Patterns of Evidence

• Based upon your experience, does the data
  surprise you?

18
Patterns of Evidence

• Does our students performance on the outcome
  meet our expectations?

19
Patterns of Evidence

• How can this data be validated? What else can we
  do to find out if this is accurate?

20
Closing the Assessment Loop
21
The Assessment Process
Cycle 2
Cycle 3
Cycle 1
O U T C O M E
New / Revised Activity 1 New / Revised Activity
2 New / Revised Activity 3
New / Revised Activity 1 New / Revised Activity
2 New / Revised Activity 3
Learning Activity Learning Activity Learning Act
ivity
Compare results against Benchmarks, Standards, T
argets,Past Performance
Measure Measure Measure Measure BASELINE
Measure Measure Measure Measure
Measure Measure Measure Measure
22
Closing the Loop

• DANGER!!!!!
• Dont be in a rush to close the loop no matter
  how soon the accreditors are coming!
• At the same time, dont let the process become
  gaseous

23
Closing the Loop

• What plans were implemented to address the
  concerns identified?

24
Closing the Loop

• Development
• Faculty, Staff, Student
• Infrastructure
• Policy, Process, Planning
• Curriculum
• Learning Opportunities

25
The Seven Principles for Good Practice in
Undergraduate Education

• 1. Student-Faculty Contact
• 2. Cooperative Learning
• 3. Active Learning
• 4. (Prompt) Feedback
• 5. Time on Task
• 6. High Expectations
• 7. Respect for Diverse Talents and Ways of
  Learning

26
Seven Objections to the Principles

• There are seven of them
• They were written 1987
• They are in a list that is practical, portable,
  and memorable
• If the principles are used, the material wont be
  covered

27
Seven Objections to the Principles

• They cant be implemented in large section
  courses
• Students are already learning without the seven
  principles
• They challenge assumptions about how students
  learn and how teachers teach

28
The Seven Principles for Good Practice

• 1. Good Practice encourages Student-Faculty
  Contact
• Frequent student-faculty contact in and out of
  classes is the most important factor in student
  motivation and involvement. Faculty concern
  helps students get through rough times and keep
  on working. Knowing a few faculty members well
  enhances students' intellectual commitment and
  encourages them to think about their own values
  and future plans.

29
The Seven Principles for Good Practice

• 2. Good practice encourages cooperation among
  students
• Learning is enhanced when it is more like a team
  effort than a solo race. Good learning, like
  good work, is collaborative and social, not
  competitive and isolated. Working with others
  often increases involvement in learning. Sharing
  one's own ideas and responding to others'
  reactions improves thinking and deepens
  understanding.

30
The Seven Principles for Good Practice

• 3. Good practice encourages active learning.
• Learning is not a spectator sport. Students do
  not learn much just sitting in classes listening
  to teachers, memorizing pre-packaged assignments,
  and spitting out answers. They must talk about
  what they are learning, write about it, relate it
  to past experiences, and apply it to their daily
  lives. They must make what they learn part of
  themselves

31
The Seven Principles for Good Practice

• 4. Good practice gives prompt feedback.
• Knowing what you know and don't know focuses
  learning. Students need appropriate feedback on
  performance to benefit from courses. In getting
  started, students need help in assessing existing
  knowledge and competence. In classes, students
  need frequent opportunities to perform and
  receive suggestions for improvement. At various
  points during college, and at the end, students
  need chances to reflect on what they have
  learned, what they still need to know, and how to
  assess themselves.

32
The Seven Principles for Good Practice

• 5. Good practice emphasizes time on task.
• Time plus energy equals learning. There is no
  substitute for time on task. Learning to use
  one's time well is critical for students and
  professionals alike. Students need help in
  learning effective time management. Allocating
  realistic amounts of time means effective
  learning for students and effective teaching for
  faculty. How an institution defines time
  expectations for students, faculty,
  administrators, and other professional staff can
  establish the basis for high performance for all.

33
The Seven Principles for Good Practice

• 6. Good practice communicates high expectations.
  
• Expect more and you will get it. High
  expectations and important for everyone--for the
  poorly prepared, for those unwilling to exert
  themselves, and for the bright and well
  motivated. Expecting students to perform well
  becomes a self-fulfilling prophecy when teachers
  and institutions hold high expectations of
  themselves and make extra efforts.

34
The Seven Principles for Good Practice

• 7. Good practice respects diverse talents and
  ways of learning.
• There are many roads to learning. People bring
  different talents and styles of learning to
  college. Brilliant students in the seminar room
  may be all thumbs in the lab or art studio.
  Students rich in hands-on experience may not do
  so well with theory. Students need the
  opportunity to show their talents and learn in
  ways that work for them. Then they can be pushed
  to learning in ways that do not come so easily.

35
Closing the Assessment Loop

• Susan Hatfield
• Winona State University
• SHatfield_at_winona.edu