Formative Evaluation Using Student Focus Groups

1
Formative Evaluation UsingStudent Focus Groups

• Heidi M. Anderson, Ph.D.
• University of Kentucky
• College of Pharmacy (UKCOP)

2
Student Focus Groups

• UKCOP uses student focus groups as part of the
  formative curricular evaluation process
• Goals
• Systematic method to collect information about
  the curriculum
• Assure confidential timely exchange between
  professors and students on their perceptions of
  course in progress

3
How it Developed

• D. Joseph Clark, Washington University
• Small Group Instructional Diagnosis (SGID)
• 5-step process
• Conducted once mid-semester
• Adapted a hybrid version of SGID
• Student Liaison Committee (SLC)

4
Who Participates in SLC?

• 10 Students from each class for Fall/Spring
• 9 members randomly selected from the 10 small
  groups
• 1 member is the class Vice President
• Facilitators from the Office of Education
  Innovation (OEI)
• Assistant Dean
• Assessment Coordinator
• Recorder
• Faculty observers periodically invited

5
Obtaining the Participants

• Beginning of Fall semester students receive 1st
  electronic correspondence, that
• Explains purpose of Student Liaison Committee
  (SLC)
• Describes the random selection process
• 2nd correspondence, is sent
• Those 10 selected members, informed of dates,
  training, etc.
• Entire class announcing the 10 individuals

6
Initial Training Meeting

• Training on Providing Feedback
• Purpose UKCOP process and assessment
• Focus group methodology
• Guidelines on offering constructive feedback
• Relevant and Appropriate
• Specific descriptions
• Offer both positive and negative
• Explain how to improve

7
Meeting Process

• After each set of block exams the Student Liaison
  Committee meets to review the learning in each
  course
• Block Exams approximately every 3 wk period
• Each course is given 10-12 minutes for discussion
  
• PY1 (7) PY2 (5) PY3 (3)
• SLC members represent their entire class
• Solicit feedback prior to meetings

8
During the Meeting

• Use a systematic 3-question format
• Responses are discussed, consensus is reached and
  recorded
• Data is organized into a report for the
  instructor
• Report is sent electronically to the instructor
  and course coordinator 48 hours post meeting

9
Feedback Questions

• What elements in the course helped students learn
  the material in this course?
• What elements in this course hindered the
  learning?
• What specific suggestions do you have to improve
  the course?

10
Post Meeting

• The instructor takes a few minutes of subsequent
  class to respond to student feedback in the SLC
  report and to describe any changes that will
  occur to improve learning
• Facilitator will arrange a follow-up session with
  the instructor prn

11
Evaluation of SLC Process
12
Evaluation of SLC

• Spring 2005, conducted a survey to determine
  student perceptions about SLC
• Purpose
• Comfort providing input
• Effectiveness of process
• Accuracy of process
• Benefits
• Areas for improvement
• Specific questions for the SLC members

13
Survey Response Rate

• All 3 classes combined 85
• PY1 90
• PY2 80
• PY3 85

14
I understand the purpose of the SLC
Mean
SD1 D2 A3 SA4
15
I feel comfortable providing information to the
SLC member(s) for discussion at the next SLC
meeting
Mean
SD1 D2 A3 SA4
16
The SLC is an effective way to communicate class
perceptions to faculty and administration
Mean
SD1 D2 A3 SA4
17
The SLC accurately reflects the perceptions of
the entire class
Mean
SD1 D2 A3 SA4
18
As a member of the SLC, the time required to
participate was worthwhile
Mean
SD1 D2 A3 SA4
19
As a member of the SLC, I understood what was
expected of me.
Mean
SD1 D2 A3 SA4
20
Benefits

• Opportunity to voice class opinions to
  communicate concerns to faculty
• The faculty response to the SLC feedback

21
Areas for Improvement

• Feedback gathered from students by their SLC
  members before the meetings
• Concern about selecting their own members to the
  SLC
• Feedback from SLC members about the meeting

22
Suggestions

• More effective methods to obtain student input
  before these meetings
• Email
• Class announcement
• Form
• Summary of the meeting results to the class

23
References

1. Clark, J and Redmond, M. Small Group
   Instructional Diagnosis Final Report. Settle
   Department of Biology Education, University of
   Washington, 1982. (ED 217-954).
2. Coffman, SJ Improving Your teaching Through
   Small-Group Diagnosis. College Teaching, 1991,
   39(2), 80-82.
3. Angelo, TA., and Cross, KP. Classroom Assessment
   Techniques A Handbook for College Teachers. San
   Francisco CA. Jossey-Bass, 1993.
4. Wimer, M. Improving College Teaching Strategies
   for Developing Instructional Effectiveness. San
   Francisco CA. Jossey-Bass, 1990.