Student Affairs Assessment Strategies: Connecting Learning Outcomes to Practice

1
Student Affairs Assessment Strategies
Connecting Learning Outcomes to Practice

• Grace Frenzel, Consultant to Student Affairs
• Pamela Rohland, Director of Disability Services
• Leslie Williams, Director of Judicial and
  Commuter Affairs
• University of Rhode Island
• International Assessment Retention Conference
• St. Louis, MO
• June 8, 2007

2
Process Flowchart
Timeline
Feedback
Feedback
Learning Focus Learning assessment
integrated into ongoing traditional assessment.
3
BOG as Prime Motivators

• Outside Assistance
• Davis Foundation Grant
• Student Learning Outcomes Assessment Office
• Student Learning Improvement Assessment Advisory
  Committee
• General Education Committee
• SA VP Assistance
• Financial Support
• Ongoing endorsement

4
Assessment Committee as Prime Motivators

• Assessment Committee Initiatives
• Education Motivation
• Develop Materials/Resources
• Learning Domains and Sub-domains
• Cross-Division SLOs Measurements
• Assessment Library
• Workshops Consultations
• Planning, Measurement, Data Analysis
• Coordinate timing of assessments (avoid subject
  fatigue)
• Compile Report aggregate results to broader
  institution administration
• Continuing Education for new staff and new ideas
  for continuing staff
• Revisions, Adjustments

5
Departments as Self Motivators

• Department Initiatives
• SLO Definition
• SLO Measurement
• Evaluation of Outcomes
• Decisions about Changes in Measurement and/or
  department functioning

6
Assessment Committee

• Members of Assessment Committee
• Division of Student Affairs at the University of
  Rhode Island
• Melissa Boyd-Colvin, Assistant Director, Student
  Leadership Development
• Stacey Bush, Grad Student, College Student
  Personnel Grad Assistant, Greek Affairs Grad
  Assistant, Substance Abuse Prevention
• Grace Frenzel, Consultant to Student Affairs
• Kevyn Goodrich, Grad Student, College Student
  Personnel Grad Assistant, Honors Program
• Pamela Rohland, Assistant Director, Disability
  Services for Students
• Lora Scagliola, Grad Student, Psychology Grad
  Assistant, Commuter Housing
• Leslie Williams, Assistant Director, Judicial and
  Commuter Affairs
• Chip Yensan, Assistant Vice President, Student
  Affairs Director, Housing and Residential Life
• For two of these individuals, division level
  assessment activities are in their job
  description.
• For the other three staff members, the activities
  have been arranged with cooperation of the
  individuals supervisor.
• The three grad students are involved on a
  volunteer basis one has received academic credit
  for being involved.
• Committee meetings every two weeks.

7
Assessment Premise

• At this point, we are not looking for cause and
  effect by setting up controlled research.
  Rather, we are trying to document what students
  know, for purposes of
• Accountability
• So students can better identify what they know
• So staff can help students expand their knowledge

8
Getting Staff Involved/Buy-InTypes of Efforts,
Examples

• Creation of Student Affairs Assessment Committee
• All members of division invited to join,
  repeatedly
• Membership from several different departments
• Students welcome
• Committee creates documents that are designed to
  ease the involvement process for members of
  division
• Statement of purpose learning domains and
  sub-domains Student Employee Learning Outcomes,
  and rubrics for measuring
• Committee educates division directors and staff
• Visits and presentations at vice presidents
  directors meetings one committee member is also
  a member of the directors group.
• Road Show taken to each department to explain
  concepts and provide reassurance
• Workshop for division to explain use of Student
  Employee Learning Outcome instrument
• Attempts to make presentations invitingeg,
  incorporate into other events, provide food

9
Getting Staff Involved/Buy-InTypes of Efforts,
Examples (cont)

• Consultant Peggy Maki provides education and
  consultation
• Division workshop to explain basic concepts
• Consultation with each department in Student
  Affairs re their outcome statements and measures
• Support from Vice President for Student Affairs
• Discussion in VPs directors meetings
• Presentation by College Student Personnel faculty
  member at divisions annual fall kickoff meeting
• Input and feedback solicited and incorporated at
  all stages
• Workshop to elicit each departments ideas about
  what students learn from interactions within the
  department
• Pilot trial of Student Employee Learning Outcomes
  instrument
• Both explicit/overt and behavioral feedback noted
  and taken into account
• Importance of repetition, keeping conversation
  going
• Frequency with which staff are exposed to
  concepts
• Variety of formats for exposureeg, division list
  serve

10
Handouts Instruments Created

• Domains and Sub-domains
• Domains and Sub-domains Template
• Student Learning Outcomes 101
• Diversity Survey
• Student Employee Learning Outcomes
• Top 10 SELO Things to Know

11
Domains and Sub-domains
12
(No Transcript)
13
SLO Template (p. 1)
14
SLOs 101
15
(No Transcript)
16
(No Transcript)
17
Diversity Assessment

• Project with Student Voice
• Students indicate all departments with which they
  have interacted
• As a result of those visits they indicate their
  level of learning on a variety of diversity
  issues
• Then they select the department which had the
  most significant impact and answer the same
  questions based on that one department
• Students and departments see what is learned
  about diversity

18
Student Employee Learning Outcomes (SELO) Process

• Learning Domains and Sub-domains had been created
• Committee wanted to help departments see how the
  model could be used
• Decided to develop outcomes and measures in areas
  that almost every department would/could be
  assessing
• Student Employees was a logical first choice
• Created common SLOs for student employees, along
  with rubrics for measuring these, for all
  departments to use
• Created 24 with the idea that each department
  could select 3-4 that were most appropriate
• Presented the SELOs and Rubrics to departments in
  a workshop, then asked those attending to pilot
  the instrument

19
Student Employee Learning Outcomes Assessment Tool
20
(No Transcript)
21
TOP 10 SELO Things to Know
22
SELO Pilot Response Not Stellar

• Extremely low response rate
• Created FAQ to help out
• Still spotty response
• Response that tool was scary, intimidating
• Viewed as a performance evaluation
• Too much detail, too many to evaluate
• Currently creating alternate options for SELO tool

23
SELO FAQs
24
(No Transcript)
25
Example Issue 1
Original
Issue Name of person being assessed implies
performance evaluation Possible Change
to Still not 100 satisfied about change
because supervisor implies power
Name of Student Employee_________________________
______ Name of Student Employee
Supervisor______________________ Person
completing form is Student or
Staff___________________
26
Example Issue 2

• Issue
• The numbers 4 to 1 seem evaluative
• Solve by
• Removie the numbers
• Arrange rubrics in list form

27
One possible alternative format for SELO
Instrument (Draft p. 1)
28
Second possible alternative format for SELO
Instrument (Draft p. 1)
29
Department SLOs

• Each department created their own SLOs
• We have a notebook with many department student
  learning outcomes included for your review.
• Two specific areas we will cover are Disability
  Services and Dining Services.

30
Disability Services Student Perceptions of
Learning Outcomes

• (total 88 respondents- responses are an
  aggregate, averaged score)
• 13. Please tell us about your learning as a
  result of your work with Disability Services for
  Students and its staff (On a scale of 1 to 5,
  where 1"almost never" and 5"almost always")
• DSS procedures help me learn to develop
  self-advocacy skills. (3.8)
• DSS staff help me learn to increase my
  self-acceptance of my personal characteristics
  and my disability. (3.7)
• Requesting accommodations helps me learn to
  improve my communication skills in an individual
  or group setting. (3.9)
• ? 2007 URI Disability Services for Students
  Campus Climate, Efficacy and Learning Outcome
  Instrument, Issued April/May 2007

31
Disability Services-Total 88 Student Perceptions
of Learning Outcomes
32
Disability Services Student Perceptions of
Learning Outcomes

• On the positive side
• approximately 70 of respondents rated their
  learning in the high range (4 or 5)
• Help DSS improve services
• On the negative side
• approximately 18 response rate (even with IPOD
  incentive)

33
Disability Services Student Perceptions of
Learning Outcomes

• Decisions about Changes
• Measurement
• Examine for differences among class level or
  disability
• Ensure software easily provides a distribution of
  responses as well as aggregate
• Individually assess learning over time in a one
  to one setting
• Department Functioning
• Look for ways to improve learning for lowest 30

34
Dining Services - Pilot SELO Instrument

• Implemented during the late fall of 2006 for the
  Rams Den Student Employee Manager Training
  Program.
• Evaluation was made by both the student and the
  manager using a pre/post design.
• The following four learning outcomes were
  selected
• Student employee demonstrates ability to work
  collaboratively with others.
• Student employee is able to balance studies and
  other responsibilities with work availability.
• Student employee uses criticism/feedback
  constructively.
• Student employee handles emotion in a manner
  appropriate to the workplace.
• Results of the initial evaluation
• Students liked doing the evaluation,
• Manager and student evaluations were the same
• Areas where the student needs to improve were
  identified
• Final evaluation showed
• Improvement in several areas
• Decline in only one area, for a graduating senior
  having difficulty balancing work and academics.
• Instrument was altered from the 4-1 scale to an
  A-D scale in the pilot.
• New learning outcomes have been specified for the
  coming academic year.
• New evaluation tool will contain only boxes to be
  checked, not numbers or letters.
• Assessment tool will be added to the student
  manager training manual for the Rams Den and
  HOPE Commons Retail Operations.

35
Dining Services - Diversity and Cultural
Awareness Assessment

• Identified as an area in which Dining Services
  could contribute to learning outcomes in the
  Campus Community.
• Form developed to evaluate the ways in which
  culture and diversity of foods could enhance an
  appreciation of other cultures.
• Taste testing of Indian Cuisine presented in the
  Rams Den this year.
• Results of this evaluation
• a. Ratings of reaction to food 1 did
  not like..5 liked it a lot
• 5. 49
• 4. 23
• 3. 19
• 2. .08
• 1. .04
• b. Tried food before 89
• Never had food before 12
• c. Willing to try again 78
• d. Increased interest in this culture
  74
• e. Willing to try foods from other
  cultures 67
• Students appear to have learned to appreciate
  foods from another culture.
• Dining Services is in the process of expanding
  the menus in the dining hall to incorporate more
  multicultural foods.

36
Keeping Committee Motivated

• We are all involved because we want to be there
• Maintain biweekly meetings to stay focused
• Established Book Review Group
• Board of Governors Mandate we have a 3 year
  deadline
• Vice Presidential Support
• Included assessment in certain job descriptions
• If we want to talk to Directors he finds the
  time
• If we want a retreat he gets people to come
• Need Food he finds it
• Need to send out emails he says ok
• Gave out copies of Learning Reconsidered to all
  Directors
• Created an Assessment Library
• We get involved in class presentations for CSP
  Students
• Person who keeps group pushing forward, keeps
  track of details and deadlines - helps committee
  succeed, feel effective.
• Group dynamics mutual respect, willingness to
  pitch-in lead to cohesion and energy for
  committees work.

37
Next Steps

• 1. Compliance with BOG mandate
• Continue with current projects
• Modifications to Student Employee Learning
  Outcomes instrument.
• Development of Diversity Learning Outcomes
  instrument.
• Assistance to departments
• Continue helping staff develop their
  understanding of the SLO assessment initiative
  (workshops, consultation, etc.)
• Feedback regarding their student learning outcome
  statements.
• Help with finding/devising assessments of the
  SLOs.
• Develop momentum in departments that have not
  made progress.
• Decide on policies and methods for compiling and
  reporting departments data within and beyond the
  Division.
• Use of computerized software packages
  (TrueOutcomes, StudentVoice, Surveymonkey, etc.)
• Who gets what results? Using what procedures?
  Should results from SA be integrated with results
  from Academic Affairs, and if so, how?

38
Next Steps (cont)

• 2. Steps beyond the mandate
• Assist SA staff who are teaching courses to
  develop learning outcome statements and measures
  for their classes.
• Educate/assist/reinforce departments in applying
  their assessment findings to improve their
  contributions to student learning.
• Work toward departments being self-motivating
  regarding learning assessment.
• Integrate learning assessment into more
  traditional kinds of assessment.
• Address question of how Student Affairs can
  contribute to students learning portfolios.
• Increase collaboration between Academic and
  Student Affairs to promote the importance of
  combining classroom and experiential learning.

39
Questions?

• Website www.uri.edu/assessment
• Email SAassess_at_etal.uri.edu
• Contact Information
• University of Rhode Island
• Kingston, RI 02881
• Grace Frenzel, Consultant to Student Affairs
• frenzel_at_uri.edu
• 401-874-9478
• Pamela Rohland, Director of Disability Services
• rohland_at_uri.edu
• 401-874-2098
• Leslie Williams, Director of Judicial and
  Commuter Affairs
• lesliew_at_uri.edu
• 401-874-5521