Engaging Students in Assessment

1
Engaging Students in Assessment

• National Conference on First-Year Assessment
• October 2008

Donald R. Whitaker, Ball State University
Sherry A. Woosley, Ball State University Amanda
R. Knerr, Pennsylvania State University
2
AGENDA

• Discussion of Current Practices
• Five Roles of Students in Assessment
• Definitions of Roles
• Examples of Roles
• Applications
• Discussion
• Questions
• Suggestions

3
Discussion of Current Practices
4
Small Group Discussion

• 1. What assessments are you currently doing at
  your institutions?
• 2. How are students involved in those
  assessments?

5
Five Roles of Students in Assessments

• Definitions and Examples

6
Role 1 Students as Subjects

• Definition and Examples

7
Role 1 Students as SUBJECTS

• DEFINITION
• Students are viewed largely as numbers.
• Results typically refer to global summaries or
  groups of students (not students as individuals).
• Students have limited, if any, contact with the
  assessment.

8
Role 1 Subject - Example

• BALL STATE UNIVERSITY
• Assessment Project
• Study of transfer students using available data
• Results
• Transfer students had a 66 retention rate
  compared to 77 retention rate for new matrics.
• Implications
• There is a retention issue for transfer students.
• We need further information about transfer
  students.
• Role of Students Subject
• Students were not involved in the project.
• Results were statistics and referred to groups.

9
Role 1 Subject - Example

• PENN STATE BEHREND
• Assessment Project
• Student Evaluation of the RA (SERA)
• Results
• Higher level of roommate conflicts where there
  were tight living/learning connections
• Implications
• Loosen the connections
• FIG/Living assignments based on school and no
  longer based on FIGs
• Role of Students Subject
• Students completed the survey.
• Results used numbers to describe groups of
  students/not individuals.

10
Role 1 Subject Other Examples

• Retention studies
• Student surveys
• Standardized testing
• Crime statistics
• Course evaluations
• Service numbers, enrollment counts, FTEs, etc.
• This is probably the most common role that
  students play in assessment.

11
Role 2 Students as Informants

• Definition and Examples

12
Role 2 Students as INFORMANTS

• DEFINITION
• Students are asked to share information about
  their experiences, expectations, perceptions,
  etc.
• Results may refer to summaries of groups of
  students but also often include stories,
  anecdotes, quotes, and individual perspectives
• Students must have contact with the assessment
  effort, but may not see the results

13
Role 2 Informant - Example

• University of Southern Indiana
• Assessment Project
• Reflexive Photography
• Results
• Importance of the physical environment on student
  perceptions
• Implications
• Paths, sculptures, etc. are important and are not
  easily assessed with traditional assessment
  methods.
• Student Role Information
• Students took pictures, described their
  significance, and talked about themes.
• Results included both individual elements and
  themes.

14
Role 2 Informant - Example

• When it rains or snows this is what we have to
  walk through to do our laundry or go to the
  convenient store. - Becky

15
Role 2 Informant - Example

• I like to sit here and study I am close enough
  to make it to class in 5 minutes. Yet I am still
  far enough that I feel all alone and can pay full
  attention to whatever I am studying - Abby

16
Role 2 Informant - Example

• University of Southern Indiana
• For more information about this project
• Harrington Schibik, (2003). Reflexive
  Photography as an Alternative Method for the
  Study of the First Year Experience, NASPA
  Journal, vol. 31, no.1
• Check the USI Institutional Research website
  (http//www.usi.edu/depart/instires/reflexphoto.as
  p)

17
Role 2 Informant - Example

• Ball State University
• Assessment Project
• Focus groups about a department website
• Results
• Need more information about internships
• Need for consistent, continual contact with
  advisor and/or faculty
• Implications
• Webpage should include internship info but should
  also emphasize personal contact with advisors.
• Student Role Informant
• Students participated in focus groups.
• Results included both individual quotes and
  themes.

18
Role 2 Informant - Example

• Penn State Behrend
• Assessment Project
• Survey for students who chose not to return to
  on-campus housing
• Included scaled questions open-ended questions
• Results
• Dissatisfaction with the damage charge policy
• Dissatisfaction that not all roommate/housing
  style preferences were honored
• Implications
• Damage charge structure may not be working.
• Importance of honoring roommate/housing style
  preferences.
• Student Role Informant
• Students participated in short survey.
• Results included themes and individual
  quotes/short answers.

19
Role 2 Informant Other Examples

• Qualitative Studies
• Interviews
• Focus Groups
• Reflexive Photography
• Feedback
• On-line feedback forms
• Student advisory boards
• Open Forums
• Online chats and blog postings

20
Role 3 Students as Users

• Definition and Examples

21
Role 3 Students as USERS

• DEFINITION
• Students are actual consumers of the information.
• Students are provided with the results of the
  study.
• Students may participate in the actual
  assessment, see the results, and have the
  opportunity to use the information from the
  results to inform their choices, activities, and
  experience.

22
Role 3 User Example

• Ball State University EBI
• Assessment Project
• Making Achievement Possible (MAP-Works)
• Results
• Students expectations ? their experiences
• Students plans will not produce their desired
  outcomes.
• Implications
• Students need information to realign their
  expectations and behaviors to produce the
  outcomes they want.
• Student Role User
• Students complete the MAP-Works survey.
• Students receive personalized feedback based on
  their survey responses. They can use the
  information to inform their behaviors.

23
Example of Feedback to Students
24
Example of Feedback to Students
25
Example of Feedback to Students
26
Role 3 User Other Examples

• Students use many sources to assess or evaluate
  us.
• Sources such as websites, ranking reports,
  internal and external publications, etc.
• Students use them to assess the quality, to
  evaluate the fit, or to determine the nature of
  an institution, department, or service.
• Students use many sources to assess or understand
  themselves. Tools such as alcoholedu, BASICs,
  etc.
• Students use them
• To understand their perceptions, attitudes, and
  activities related to alcohol and other substance
  use
• To compare that information to the normative data
  of their peers.
• These sources can include assessment data.

27
Role 4 Students as Collaborators

• Definition and Examples

28
Role 4 Students as COLLABORATORS

• DEFINITION
• Students are actively involved in the assessment
  project
• They may assist with
• Planning the assessment
• Collecting the information
• Making sense of the results
• Implementing the changes

29
Role 4 Collaborator - Example

• Ball State University
• Longitudinal Evaluation and Assessment Panel
  (LEAP)
• Students provide input about assessment
  activities.
• Students suggested issues that may need attention
  such as alcohol use, social support, and
  differentiating between major and non-major
  courses
• They suggested questions to be asked.
• Describe a time when you thought you might want
  to quit school and how did you overcome that?
• How do you deal with professors you dont
  particularly like and how do you survive the
  semester with them?
• They are helping us design new formats for new
  on-line MAP system, which will provide feedback
  to students.

30
Role 4 Collaborator Other Examples

• Collaborators give input, feedback, suggestions,
  etc. about assessment activities and topics.
• Assessment Advisory Boards
• Individual Students (undergraduate and graduate)
• Work Study Students

31
Role 5 Students as Authors

• Definition and Examples

32
Role 5 Students as AUTHORS

• DEFINITION
• Students are instrumental in the assessment
  project because they are responsible for it.
• They may be involved in every step or they may
  seek assistance in some areas.

33
Role 5 Author - Example

• Ball State University
• Student Government Association (SGA) Shuttle Bus
  Survey
• SGA wanted to survey students about Shuttle Bus
  Service and Needs. They developed a survey.
• The Assessment office provided assistance
• Survey revisions
• Methodology
• Survey administration
• Analysis of results
• SGA publicized the survey and the results. They
  also used the results to formulate policy
  recommendations.
• Student Role SGA instigated the study, sought
  assistance, and acted as the author of the work.

34
Role 5 Another Author Example

• Ball State University
• Amanda Knerr Masters Thesis
• Research Question How are hall directors (HDs)
  using the assessment reports from MAP?
• Knerr found
• HDs were using student reports to facilitate
  discussion in meetings and to initiate contact
  with at-risk students.
• HDs were not prepared to link summary statistics
  to practice.

35
Role 5 Another Author Example

• Knerr Masters Thesis (continued)
• Implications
• Individual student report formats are working.
• Need to change summary report format to focus on
  practical implications.
• May need further discussions with HDs to
  reinforce the links between assessment results
  and practice.
• Student Role
• Knerr was an author who evaluated the
  effectiveness of assessment reporting and made
  suggestions for how to improve it.

36
Role 5 Authors Other Examples

• Authors are responsible for the assessment
  project
• Graduate and undergraduate research projects
• Graduate assistantships
• Student initiatives

37
Small Group Discussion
38
Small Group Discussion

• What roles are students currently playing in your
  assessment?
• What other roles would be useful on your campus?

39
Additional Questions and Discussion
Donald R. Whitaker, Ph.D. Executive Director of
Institutional Effectiveness Ball State
University dwhitake_at_bsu.edu
Amanda R. Knerr Associate Director of Student
Affairs Pennsylvania State University ark14_at_sa.psu
.edu
Sherry A. Woosley, Ph.D. Associate Director of
Institutional Effectiveness Ball State
University sawoosley_at_bsu.edu