What I Wish I had Known when I started Assessing OutoftheClassroom Learning

1
"What I Wish I had Known when I started Assessing
Out-of-the-Classroom Learning"

• Jo Frederic, Services for Students with
  Disabilities
• Edie Blakley, Career Services
• Lisa Hoogesteger, Recreational Sports
• Eric Hansen, University Housing Dining Services

2
Overview

• Introductions
• Getting Started
• A Common Language
• The Assessment Plan
• Student Services in Action
• UHDS
• DRS
• Career Services
• Students with Disabilities
• What We Learned

3
Getting Started
4
9 Principles of Good Practice for Assessing
Student Learning American Association for Higher
Education

• The assessment of student learning begins with
  educational values.
• Assessment is most effective when it reflects an
  understanding of learning as multidimensional,
  integrated, and revealed in performance over
  time.
• Assessment works best when the programs it seeks
  to improve have clear, explicitly stated
  purposes.
• Assessment requires attention to outcomes but
  also and equally to the experiences that lead to
  those outcomes.
• Assessment works best when it is ongoing not
  episodic. Assessment is a process whose power is
  cumulative.

5
9 Principles Continued American Association for
Higher Education

• Assessment fosters wider improvement when
  representatives from across the educational
  community are involved.
• Assessment makes a difference when it begins with
  issues of use and illuminates questions that
  people really care about.
• Assessment is most likely to lead to improvement
  when it is part of a larger set of conditions
  that promote change.
• Through assessment, educators meet
  responsibilities to students and to the public.

6
Common Language

• Mission Describes the purpose of the
  organization and the constituents served.
• Goal (same as Objective or Initiatives) A broad
  general statement of what a program wants its
  constituents to know or do. Must be connected to
  the mission.
• Outcome (can be same as Learning Objective)
  More detailed and specific statements derived
  from the goals. These are specifically about what
  you want the end result of your efforts to be.
• For student learning outcomes, they are not what
  you are going to do to the student, but rather
  what you want the student to know or do.
• Methods The criteria, process, and tools used
  to collect evidence that indicates whether or not
  the outcomes have been met.
• Results Evidence by outcome
• Evaluation Analysis of the evidence
• Follow-up How the evidence informs future
  decisions and recommendations on program, as well
  as how evidence informs future assessment
  activity.

7
Assessment in Student Affairs

• Assessment Plan Sample
• Mission
• Goal
• Outcome
• Evaluation Method
• Implementation of Assessment
• Results
• Decisions

8
Student Services in Action
9
Mission and GoalsUniversity Housing Dining
Services

• Mission
• to provide housing and dining environments
  through facilities, programs, and services that
  support the academic and personal success of our
  residential students and other OSU community
  members.
• Goals
• Goal 1 To Promote Student Development and
  Academic Success
• Goal 2 To Foster Staff Development and Success

10
Developing Outcomes University Housing Dining
Services

• UHDS - Two Approaches
• The Hard Way Goal 1
• The Easier Way Goal 2

11
Measuring Outcomes University Housing Dining
Services

• UHDS - TBD

12
Mission and GoalsDept of Recreational Sports

• Mission
• We strengthen the University by providing quality
  recreational and educational opportunities that
  foster healthy living.
• Goals
• Provide quality student learning and leadership
  experiences
• Provide quality recreational programs, services
  and facilities that meet the needs of OSU
  students, faculty and staff.
• Promote awareness of and participation in
  recreational sports as part of a healthy
  lifestyle.

13
Developing OutcomesDept of Recreational Sports

• Goal DRS will provide quality student learning
  and leadership experiences.
• A. Process all student employees will
  participate in a dept. Orientation prior to
  beginning employment.
• B. Learning student employees will demonstrate
  skills in
• Knowledge of DRS and value of recreation
• Risk management and safety awareness
• Professionalism
• Customer service
• Diversity awareness and appreciation

14
Measuring OutcomesDept of Recreational Sports

• Safety awareness/risk management
• 100 of employees will demonstrate successful
  completion of First Aid and CPR courses. (success
  is determined with written and practical exams)
• 50 of employees will have a safety audit
  during the year. Audits measure analysis and
  application of safety skills and emergency action
  procedures.
• Audits are graded on a 4 point scale.

15
Developing Outcomes ContextDept of Recreational
Sports

• Blooms Taxonomy
• Six levels of classification for intellectual
  behavior
• Knowledge tell, list, describe, state
• Understanding explain, interpret, compare
• Application show, use, illustrate
• Analysis analyze, examine, explain
• Synthesis create, design, imagine
• Evaluation judge, justify, recommend

16
Developing Outcomes continuedDept of
Recreational Sports

• Knowledge of DRS and value of recreation
• List three resources to answer participant
  questions (Knowledge)
• Describe what new participants should know about
  DRS (Understanding)
• How do you use information from the DRS
  orientation to meet customer needs? (Application)
• Identify what activities contribute to
  participant sense of well-being (Analysis)
• Create a new activity to engage new students
  while touring DRS facilities (Synthesis)

17
Mission and GoalsCareer Services

• Mission
• As a gateway to the future we support the mission
  and goals of Oregon State University through
  centralized services that provide skills and
  knowledge to develop career life-planning
  processes. We stimulate a lifelong attitude of
  inquiry, openness, and social responsibility
  through partnerships with students, alumni, the
  academic community and employers.
• Goals
• Build student capacity for effective career and
  life development

18
Developing Outcomes Career Services

• Students will learn professional etiquette
• Students will learn effective interviewing
  skills
• Students will learn effective resume writing
  skills

19
Measuring Outcomes Career Services

• Goal Build student capacity for effective
  career and life development
• Outcome A Students will learn professional
  etiquette
• Networking and Dining Etiquette Event Survey
  Sample Questions
• I feel more confident in my ability to meet
  people and develop contacts at a social event
• I am better prepared to present myself in a
  professional manner and make a good first
  impression
• Revised Questions
• I feel more confident in my ability to approach
  people at a social event
• I am better prepared to introduce myself
• I am better prepared to present myself in a
  professional manner
• I am better prepared to make a good first
  impression

20
Measuring Outcomes Career Services

• Goal Build student capacity for effective
  career and life development
• Outcome B Students will learn effective
  interviewing skills
• Measurement methods
• Survey Students after workshop. Successful if
  80 of students report increased confidence and
  ability to interview effectively.
• Direct observation by advisors during mock
  interview video review.
• Direct observation by Making Outstanding
  Connections (MOC) program interviewers who will
  provide feedback at the end of the MOC program.
• Employers assess during on-campus interviews.
  Successful if at least 60 of students
  interviewed are rated above average or excellent
  in the following three areas appearance,
  preparation and communication skills.

21
Measuring Outcomes Career Services

• Direct observation
• Employers assess during on-campus interviews.

22
Measuring Outcomes Career Services

• Goal Build student capacity for effective
  career and life development
• Outcome C Students will learn effective resume
  writing skills
• Measurement methods
• Survey students after workshop. Successful if 80
  of students report increased confidence and
  ability to successfully develop an effective
  resume.
• Direct observation by advisors using a rubric
  during career week drop in appointments.

23
Measuring Outcomes Career Services
Resume Assessment Rubric -- Spring Career Week
2004
24
Mission and GoalsServices for Students with
Disabilities

• Mission
• Services for Students with Disabilities
  facilitates the continued success of students
  with disabilities and the Oregon State community
  by providing and promoting a supportive,
  accessible, and non-discriminatory learning and
  working environment.
• Goals
• Provide effective accommodations for disabilities
  based on law and/or current best practices.
• Educate the campus community about disability
  issues
• Positively influence decision-making and
  practices to include Universal Design principles.
• Promote self-determination for individuals with
  disabilities.

25
Developing Outcomes Services for Students with
Disabilities
26
Measuring Outcomes Services for Students with
Disabilities

• The Benefits of Delayed Gratification!
• Good Things Come to Those Who Wait

27
Using Information and Making Decisions Services
for Students with Disabilities

• Now weve got data! What did we learn?
• The Genie in the Bottle

28
Using Information and Making Decisions Services
for Students with Disabilities

• Results
• Use
• SSD Demographic Information
• Rapid, substantial increase in the number of
  students requesting/eligible to receive services.
• Satisfaction
• Students Voices
• SSD Alternative Testing Service Student
  Satisfaction Survey
• Testing rooms
• Proctors
• Process
• Accessibility
• Technology
• Communication with professors

29
Using Information and Making Decisions Services
for Students with Disabilities

• Overwhelmed by Opportunities
• Decisions Based on Satisfaction Survey
• SSD Alternative Testing Program
• Search for additional, accessible testing rooms
• Hire additional proctors
• Require proctor training
• Reconfigure space within SSD
• Collaborate with the Technology Access Program
• Create an on-line database

30
Using Information and Making Decisions Services
for Students with Disabilities

• Results Beyond Use and Satisfaction
• Learning Outcomes
• Goal 4 Promote and facilitate
  self-determination for individuals with
  disabilities
• Outcome (SSD Students)
• SSD students will manage their own accommodations
  by using the SSD Testing Database to request,
  monitor, and update their alternative testing
  requests.

31
Using Information and Making Decisions Services
for Students with Disabilities

• The End and The Beginning (again!)

32
What We Learned
33
Lessons Learned

• Dos
• Just do it
• Be open to ambiguity
• Involve leadership and staff and students
• Use a variety of tools surveys, rubrics, direct
  observation, interviews, etc.
• Check to make sure the answers provided are
  usable data Will we know what to do as a
  result?

34
Lessons Learned

• Dos
• Go slow - Allow time for discussion
• Determine an ongoing process to keep the momentum
• Understand that it is an iterative process we
  refine and revise as we go
• Practice, practice some more, and keep practicing

35
Lessons Learned

• Dos
• Begin with the end in mind
• What does it look like when
• Make it meaningful interesting for your day to
  day work. Be curious. Ask why.

36
Lessons Learned

• Donts
• Dont try to do too much
• Dont try to make the plan perfect before getting
  started
• Dont let nay-sayers drive your energy
• Dont rely on 100 agreement of everyone involved
• Dont reinvent the wheel - See if it is out there
  already
• Dont be afraid to change

37
Unintended Benefits

• Inter-departmental collaboration
• Corporate thinking and dialogue
• Narrative for Grants

38
Questions?