Who Are Our Students Assessment Forum

1
Who Are Our Students???Assessment Forum

• Monday, Feb. 9, 2004

Donald J. Lerew, Jr. Institutional
Researcher Ronald Burwell Professor of Sociology
2
Overview

• Welcome Review of Agenda
• IR Assessment Website Data Sets
• Assessment Calendar
• AICUP First-Year Student Survey 2003
• Senior Interviews Some Highlights
• NSSE Student Engagement Survey 2002
• CSS Senior Survey 2003

3
Talking Points

• How do seniors view their time at Messiah?
• How many hours do our students spend in academic
  pursuits in a given week?
• How do students views change regarding gender
  roles?
• Who stays and who leavesand why? Social vs.
  Academic Factors?
• How much effort do our students put into their
  college experience?

4
Research/Assessment Web Site

• General, college-wide statistics
• IR Publications Listing
• Assessment Calendar
• Assessment data
• CIRP, CSS, NSSE, CSEQ, SSI, FYSS
• Location Messiah Home gt Campus Offices gt
  Institutional Research

5
Survey Data Sets

• Most analyses have datasets available
• Datasets typically created for SPSS
• Contact Ron or I for a copy

6
Assessment Calendar

• Schedule of planned institutional assessment
  activities over next several years
• Presented and reviewed by Assessment Committee
• Goal is to not overburden students, while
  gathering sufficient, meaningful data
• Calendar is available on IR website

7
(No Transcript)
8
Assessment Calendar

• CIRP Cooperative Inst. Research Program
• Survey of New Freshmen at orientation
• Baseline picture of who students are before
  college
• AICUP First-Year Student Survey
• Survey of first-year students in spring semester
• Assesses academic and personal development of
  students during their first year of college

9
Assessment Calendar

• NSSE National Survey of Student Engagement
• Given to 2nd-semester freshmen and seniors
• Studies quality of UG learning on campus by way
  of student engagement
• CSS College Student Survey
• Surveys Messiah seniors
• Students self-reported academic and campus life
  experiences

10
Assessment Calendar

• CSEQ College Student Experiences Questionnaire
• Surveys seniors
• Assesses the quality of student effort in
  utilizing resources/opportunities at college
• SSI Student Satisfaction Inventory
• Surveys subset of each class level
• Measures student importance and satisfaction to
  various campus issues

11
Assessment CalendarOther Possible Instruments

• YFCY Your First College Year
• Supplement to CIRP
• FSSE Faculty Survey of Student Engagement
• Others???

12
AICUP First-Year Student SurveySpring 2003

• Survey of first-year students
• Surveyed during spring semester
• Surveyed through campus mail
• General and retention analyses conducted

13
AICUP First-Year Student Survey--Self-Assessmen
t--

• MC men are significantly less prepared are
  significantly more challenged in the area of
  study skills during the first year
• MC students indicate study and writing skills as
  their two biggest first-year skill challenges
• Both self-rating and first-year assessment of
  spirituality traits were significantly better
  than comparison institutions

14
AICUP First-Year Student Survey--Transition to
College--

• MC women experience significantly more
  homesickness than men
• MC students more likely to be homesick than at
  comparison institutions
• MC men report significantly less perceived help
  in areas of
• Dealing with roommate problems
• Dealing with personal problems
• Working with advisor

15
AICUP First-Year Student Survey--Academic
Advising--

• MC men relied significantly more on themselves
  for academic advice than did women (M-28.6,
  W-15.4)
• MC students placed more emphasis on soliciting
  academic advice from family (MC-25.9,
  Comp-15.1)
• MC students placed less reliance on the advisor
  (MC-21.4, Comp-29.0)

16
AICUP First-Year Student Survey--Academic
Advising--

• MC women met with advisor more often during first
  year (M-2.55, W-3.26 mtgs.)
• Comparison group averaged 3.80 meetings
• MC students rated advisors significantly better
  than comparison group students in
• Knowledge level, availability, overall
  satisfaction

17
AICUP First-Year Student Survey--Use of Time--

• MC students spend nearly 22 hrs/wk attending
  class or studying/doing homework
• MC men spend significantly more time than women
  watching TV and playing video/computer games
• MC women spend significantly more time in prayer
  and meditation

18
AICUP First-Year Student Survey--Use of Time--

• MC students spend significantly more time than
  comparison group students in
• Prayer/meditation
• Studying and completing homework
• Participating in student clubs/groups

19
AICUP First-Year Student Survey--Satisfaction--

• MC women significantly more satisfied in
• Quality of academic advising, quality of academic
  offerings, availability of academic offerings,
  Internet access, faculty attitude toward
  students, and personal counseling
• MC students indicated significantly higher
  satisfaction than comparison students in
• Library facilities/resources, food services

20
AICUP First-Year Student Survey--Satisfaction--

• 93.7 of MC students indicated that they would
  definitely or probably return for the second
  year (Comp. Group - 90.2)
• Prominent reasons for not returning
• Transferring
• Dissatisfaction with social atmosphere
• Lack of financial resources

21
AICUP First-Year Student Survey--Retention
Analysis General--

• Attrited students reported significantly
• Higher levels of homesickness and loneliness
• Lower levels of belongingness
• Attrited students indicated significantly less
  college help in the areas of
• Personal problems, student club involvement,
  organized social events, rec. athletic activities

22
AICUP First-Year Student Survey--Retention
Skills Assessment--

• Attrited students rated themselves significantly
  lower in the areas of
• Intellectual self-confidence, social
  self-confidence, ability to function
  independently, leadership ability, reading
  analytically, and spirituality
• While incoming skill levels differed, no
  significant differences in reported growth during
  first year

23
AICUP First-Year Student Survey--Retention
Other Items--

• Attrited students were significantly more likely
  to indicated the advisor as the primary source of
  academic advice equally less on self-reliance
• Retained students spent more time socializing
  with friends (7 vs. 4 hrs/wk)
• Both student groups indicated similar academic,
  residence hall, and social life expectations
• Attrited students indicated significantly lower
  levels of having those expectations met during
  the first year

24
Senior Interviews Spring 2002

• Approximately 27 completed interviews with
  seniors during the spring of 2002
• Transcriptions created for 24 interviews (sound
  quality created some difficulties in
  transcription)
• Topics Educational Experiences Religion Gender

25
Educational Experiences

• Based on questions from Richard Light
• Academic regrets
• Unforgettable experiences
• Critical moments
• Important books
• Interactions in the residence hall

26
Religion

• Awareness of and introduction to colleges
  religious heritage related to Lilly Project
• Important formative faith experiences
• Experiences with doubt
• Understanding of what is a mature Christian
  James Fowler

27
Gender

• Views on male / female roles with society and the
  family
• Views on ordination of women and role in the
  church
• Impact of Messiah College on ideas regarding
  gender

28
Senior Voices

• Academic regrets
• Unforgettable experiences
• Residence Hall interactions
• Views on gender roles
• Ordination of women

29
NSSE Student EngagementSpring 2002

• Survey of 1st-yr. and senior students
• Determine extent to which students engage in
  educational practices that are associated with
  learning outcomes
• Surveyed during spring semester
• Web administration
• General and school analyses

30
NSSE Student Engagement
31
NSSE Student Engagement--Acad/Intellectual
Experiences--

• Only 21 of MC seniors worked with other students
  on projects during class often or very often
  (Bac-Gen NSSE 43)
• MC first-year students worked significantly more
  often with classmates outside of class to
  complete class assignments
• MC students discussed class ideas with others
  outside of class significantly more often than
  comparison students

32
NSSE Student Engagement--Acad/Intellectual
Experiences--

• MC women were more likely than men to
• Use e-mail to communicate with instructor
• Work harder than they thought they could
• Discuss ideas with others outside of class
• Work with classmates outside of class
• Students w/ a B- or lower average were more
  likely to
• Come to class unprepared
• Work with others on projects during class

33
NSSE Student Engagement--Mental Activities--

• Five Levels (1) Memorize, (2) Analyze, (3)
  Synthesize, (4) Make Judgments, (5) Apply
• MC students had similar category breakdowns to
  comparison students
• Exception both first-year and senior MC students
  indicated lesser levels of memorization
• MC students w/ B- or lower indicated
  significantly higher usage of memorization

34
NSSE Student Engagement--Reading Writing--

• MC students indicated a significantly higher
  number of assigned textbooks/readings than
  comparison students
• MC students indicated writing significantly more
  5-19 and under-5 page papers

35
NSSE Student Engagement--Educational
Experiences--

• MC students indicated significantly higher
  participation/plan to participate rates in
• Practicums, internships, or field experiences
• Community service or volunteer work
• Foreign language coursework
• Study abroad
• Culminating senior experience (exam, capstone,
  project, etc.)
• MC women are more likely to attend

36
NSSE Student Engagement--Time Usage--

• MC students spend significantly more time
  preparing for class than do comparison students
  (FY-14.5, Sr.-14 hrs/wk)
• MC FY students spend 11 hrs/wk relaxing/socializin
  g 9 hrs/wk for seniors
• MC students work significantly more on-campus,
  but significantly less off-campus

37
NSSE Student Engagement--Time Usage--

• MC first-year women spend significantly more time
  preparing for class senior men, however, spend
  more time than women
• MC students with a B- or lower spend
• Less time preparing for class (9 vs. 14 hrs/wk)

38
NSSE Student Engagement--Educational/Personal
Growth--

• MC students indicated significantly more growth
  in
• Acquiring a broad general education
• Developing personal code of values/ethics
• Contributing to welfare of community
• Writing clearly and effectively
• Thinking critically and analytically
• MC first-year students reported significantly
  less growth in the use of computer/info.
  technology than did comparison students

39
NSSE Student Engagement--Institutional
Emphasis--

• MC students reported significantly more emphasis
  by the college on
• Spending significantly more time studying and on
  academic work (seniors only)
• Encouraging diversity contacts/interaction
• Providing support needed to thrive socially
• Attending campus events and activities
• Messiah women indicated higher perceived emphasis
  toward academic and social support

40
NSSE Student Engagement--Satisfaction--

• Over 90 of MC men and 95 of women rated the
  educational experience as good or excellent
• Significantly higher than comparison groups
• A high majority of students indicated they would
  probably or definitely return
• FY (W-94.7, M-90.6)
• Sr. (W-93.8, M-84.6)
• Students w/ B- or lower evaluated educational
  experience significantly lower than B or better

41
NSSE Student Engagement--Analysis by School--

• Summarizes mean responses by school
• Compares each school mean by the aggregate of all
  other schools
• Compares each school mean by class level

42
College Student Survey - 2003
43
Sample

• Sample size 267 graduating senior
• 32.2 male
• 67.8 female
• Slight oversample of females

44
Plans for fall 2003

• Approximately 1 in 5 seniors plan on graduate
  study in the fall
• Most of the remaining students plan on working
  either part time or full time

45
Academic Activities

• MC students report frequent peer interaction in
  connection with academic activities (e.g.,
  studying with other students and discussing
  courses with other students)
• MC students were more likely to be guests in
  professors homes compared to other private
  colleges

46
Other Activities

• Some 38.7 report drinking beer during the last
  year
• 56.8 report consuming wine or alcohol during the
  past year
• While these are lower than comparable rates at
  private colleges they are higher than in 2001
• About 40 of MC students begin to use alcohol
  while in college

47
Other experiences

• A majority of students reported that they
  occasionally or frequently were homesick (64.7)
  or depressed (63.9)
• About one third of the students reported
  frequently feeling overwhelmed (38.7)

48
Time Usage

• MC students report that they spend about 25 hours
  per week on academic pursuits (classes/studying)
• A comparison sample from private colleges report
  about 22 hours per week on the same activities
• In 2001 MC students reported slightly more time
  spent on academic pursuits (28 hours)

49
Interactions with Professors

• MC students report that professors frequently
  provided intellectual challenge (55.6) and
  emotional support (36.5)

50
Satisfaction with college

• Some 91 of MC students are satisfied with their
  overall college experience (private 86)
• Most MC students say they would choose the same
  college again (MC 89, private 80)

51
Objectives of importance

• The top four objectives were
• Integrating spirituality into my life
• Helping others in difficulty
• Raising a family
• Developing a meaningful philosophy of life

52
Political Identity

• By their senior year, MC students are more
  likely to say they are middle of the road
• First year 32.4
• Senior year 46.3
• Comparing 2001 with 2003, the 2003 seniors seem
  to be less conservative and more moderate (even
  upon entering college)

53
Questions regarding majors

• In both 2001 and 2003 we asked twenty
  supplementary questions regarding experiences in
  the major
• In general, MC students report very high levels
  of satisfaction with their experiences in the
  major the class of 2003 is even more positive
  than the class of 2001

54
Comparisons 2001 vs. 2003

• In a number of areas, the 2003 seniors are more
  positive than those of 2001
• Given the discussion regarding alcohol, it is
  interesting that the use of alcohol among seniors
  is higher in 2003
• Less time spent on academic pursuits
• Less conservative students

55
Questions??