Maximizing Your Assessment by Combining Student Survey Results: EBI, NSSE, and CIRP

1
Maximizing Your Assessment by Combining Student
Survey Results EBI, NSSE, and CIRP
Jillian Kinzie, Ph.D. Associate Director, Center
for Postsecondary Research and National Survey of
Student Engagement (NSSE) Institute John Pryor,
M.A. Director, Cooperative Institutional Research
Program, Higher EducationResearch
Institute Darlena Jones, Ph.D. Director of
Research and Development, Educational Benchmarking
2
Session Objectives

• Introduction
• Overview of CIRP, NSSE, and EBI
• Data Triangulation
• Using Studies to Measure
• Retention
• Integration to Campus
• First-Year Program Improvement
• Learning Outcomes
• Using Multiple Data Sources
• Other Data Considerations
• Small Group Discussion

3
Introduction
4
What is CIRP?
CIRP Cooperative Institutional Research Program
5
Introduction

• CIRP surveys have been used for decades to
  understand and illustrate the impact of college.
• Can use the surveys as cross-sectional surveys,
  but the key advantage of CIRP surveys is that
  they are longitudinal.

6
Astins I E O Model
Environments YFCY/CSS (e.g., place of residence
during college, interactions with peers and
faculty, curricular and co-curricular experiences)
Inputs CIRP Freshman Survey (e.g., academic
performance in high school, financial
concerns prior to college entry, expectations
for college, degree aspirations, self-concept in
high school)
Outcomes YFCY/CSS (e.g., satisfaction with
college, retention, gains in college,
post-college plans)
7
Higher Education Research Institute at UCLA
Cooperative Institutional
Research Program
Funded Research
Freshman Survey
YFCY
CSS

• Ford Foundation
• Templeton Foundation
• National Institutes
• of Health

Faculty Survey
8
CIRP Freshman Survey (TFS)

• Largest and longest-running national study of
  American college students2008 is 43nd
  administration
• Initiated in 1966 at the American Council on
  Education Housed at HERI (UCLA) since 1973

9
CIRP Freshman Survey (TFS)

• Three types of input items
• Pretests on possible outcomes measures
• Self-predictions about possible future outcomes
• Personal characteristics that might effect the
  propensity to attain certain outcomes

10

2007
1966
206,865 respondents 251 Colleges and
Universities
374,261 respondents 536 Colle
ges and Universities
Total over 42 years 13,408,942 students 1,708
colleges and universities
11

• Your First College Year (YFCY)
• Launched in 2000
• 208,157 students at 378 schools
• Designed to assess academic and personal
  development over the first year of college
• Developed in collaboration with the Policy Center
  on the First Year of College

12
Your First College Year Survey (YFCY)

• Mission Longitudinal assessment of student
  development and institutional/programmatic impact
  during the first year of college
• Study Launch Pilot Studies in Spring 2000
  2001 National Assessment launched in Fall 2002
• Data Collected Since its inception, surveyed
  nearly 150,000 first-year students at nearly 250
  institutions nationwide
• Administration Disseminated at end of the first
  year in an on-campus administration format paper
  and/or web versions overall response rates
  35-82.

13
YFCY

• Comprehensive in content to assess academic
  social experiences as well as cognitive
  affective development
• Designed as a follow-up instrument
• 40 of items are direct post-tests to questions
  on the CIRP Freshman Survey
• Survey Content
• Academic achievement engagement
• Learning strategies and pedagogical practices
• Residential employment experiences
• Interactions with family, peers, faculty staff
• Patterns of behavior
• Student values and life goals
• Satisfaction, self-concept, feelings of
  personal success
• Plans for the next academic year
• Space for 20 questions of local relevance

14
YFCY Deliverables and Data

• Standard Deliverables
• Institutional Profile
• Longitudinal Profile
• Electronic Data File
• Reports on Spreadsheet

• Additional Data Services
• Institutional Data in PowerPoint Executive
  Summary
• Peer Group Reports
• Data Merges

15
Using YFCY Data to Enhance Campus Assessment
Efforts

• Purposes
• Self-study reports strategic planning
• Retention studies
• Examining group differences among students
• Programmatic assessment
• Measuring student development institutional
  impact
• Feedback to campus constituents
• Creation of a student information system

• Methodologies
• Descriptive analyses with campus data
• Comparative analyses
• Institutional subgroups
• Institutional vs. national data
• Aspirant group or consortium
• Measures of association
• Factor analyses
• Trends analyses
• Multivariate analyses

16
Who is NSSE?
NSSE National Survey of Student Engagement
17
NSSE Stats

• Launched in 2000, supported by institutional
  participation fees
• Parent survey CSEQ (1979)
• National design team created survey
• Administered by Indiana University Center for
  Postsecondary Research
• More than 1,300 different baccalaureate
  colleges/universities from 50 states, Puerto
  Rico, and Canada (CCSSE is for 2-year
  institutions)
• Consortia can add up to 20 questions (e.g.,
  womens colleges, HBCUs, state systems)
• Spring administration to first-year senior
  students
• Third party, standardized administration average
  40 response rate, random sample, and select
  comparison groups.

18
NSSE Purposes

• Provide reliable indicators of good educational
  practices
• Support institutional improvement and
  accountability efforts
• Foster comparative consortium activity
• Refocus conversations about quality in
  undergraduate education
• Direct measure of frequency of participation in
  educational practices associated with learning
  and development.

19
Survey Instrument
College student survey that assesses the extent
to which students engage in educational practices
associated with high levels of learning and
development
20
Two Components of Student Engagement

• What students do time and energy devoted to
  educationally purposeful activities
• What institutions do using effective
  educational practices to induce students to do
  the right things

21
NSSE Reports

• Comparative data (National, Carnegie Class,
  Select Comparison Groups, and/or Consortium)
• Respondent Characteristics
• Frequency Distributions
• Mean Comparisons
• Benchmark Comparisons
• Multi-year Benchmark Comparisons
• Pocket Guide Report (for prospective students)
• Executive Snapshot Report
• Data File so you can link to other data!!

22
BCSSE Beginning College Survey of Student
Engagement

• Based on requests for related pre-college
  measures
• Measures selected high school experiences and
  students interest in and expectations for
  participating in educationally purposeful
  activities during college
• Designed to be combined with spring NSSE data for
  pre- and post- views of the first-year
  experience Use to examine gap between
  expectations and engagement and study effect of
  students background on NSSE scores
• Can also be used as stand alone assessment
• Pilots in 2005, 2006 Officially launched in 2007

23
NSSE Possibilities

• Merging NSSE data with school records
• Descriptive displays of engagement patterns by
  any number of student characteristics
• Prediction models for retention, degree
  attainment, grades, other outcomes
• Tracking student engagement year to year

• Comparisons against aspirational, regional, and
  mission-related institutions
• Program assessment
• Accreditation reporting
• Consortium and system data sharing
• Scholarly research

24
Who is EBI?
EBI Educational Benchmarking, Inc.
25
EBI Stats

• Founded in 1994 by Joseph Pica, Ed.D and Glenn
  Detrick (retired in 2002)
• Over 1500 Colleges and Universities (U.S.A. and
  15 other countries like Australia, Mexico, Spain,
  and Egypt) have participated in EBIs studies
• Surveyed over 12 million people
• Nearly 100 Assessments in 10 areas of higher
  education
• Over 500 custom assessments for schools like MIT,
  Ohio State, UCLA, University of Georgia, and
  University of Florida
• Produced over 17,000 customized reports
• Creation of WESS

26
EBIs Partners (alphabetical order)

• AACN (American Association of Collegiate Nursing)
• ACUHO-I (The Association of College and
  University Housing Officers International)
• ACUI (Association of College Unions
  International)
• AFA (Association of Fraternity Advisors)
• Ball State University
• MHLI (Military Housing and Lodging Institute)
• OTC (Outside the Classroom)
• Policy Center on the First Year of College

27
Relevant Assessments
EBI First-Year Initiative (FYI)

• Mission To assess the effectiveness of
  first-year courses / seminars.
• Survey Development Created in conjunction with
  the Policy Center for the First Year of College.
  Initial funding of project by the Pew Charitable
  Trusts. Currently funded by institution
  participation.
• Study Stats Pilot Study in Spring 2001 National
  Assessment launched in Fall 2001. 169 colleges
  and universities have submitted over 185,000
  responses.
• Why Participate Provides institution information
  on course effectiveness. Provides instructors
  their class results for individual improvement.

f
28
Relevant Assessments
ACUHO-I/EBI Resident Assessment

• Mission To assist college and universities in
  assessing resident satisfaction of residence hall
  functions and the effectiveness of residence hall
  life.
• Survey Development Created in conjunction with
  ACUHO-I. Housing professionals collaborated with
  EBI to design survey questions.
• Study Stats National Assessment began Spring
  1998. Since then, 567 colleges and universities
  worldwide have submitted over 2.6 million
  responses.
• Why Participate Provides evidence of student
  satisfaction of their on-campus living experience
  and student learning outcomes linked to CAS
  Standards.

29
Relevant Assessments
MAP-Works

• Mission An early warning indicator of retention
  and student success
• Survey Development Created in conjunction with
  Ball State University. Sherry Woosley from BSU
  collaborated with EBI to design survey questions.
• Study Stats Study piloted in 2006 full
  national study in 2007. Over 40 institutions are
  utilizing MAP-Works in 2008
• Why Participate Provides reporting directly to
  each first-year student regarding their behaviors
  benchmarked against their first-year cohort.
  Provides information directly to advisors, hall
  directors, and FYE instructors regarding their
  students so early intervention can occur.

30
Relevant Assessments
FoE/EBI Foundation of Excellence Student and
Faculty/Staff Assessments

• Mission Assist campus professionals in the
  self-study of their first-year program in support
  of the Foundations of Excellence
• Survey Development Created in conjunction with
  the Policy Center on the First Year of College.
  Randy Swing, John Gardner, and Betsy Barefoot
  collaborated with EBI to design survey questions.
• Study Stats Studies piloted Summer 2005.
  National Assessment launched Fall 2005. Since
  2005, 67 schools have participated in the 4 year
  study and 39 schools have participated in the 2
  year study.

31
Assessment Features

• General Features
• Professional Survey Development EBIs survey
  development team working in partnership with
  professional organizations
• Data Collection Paper surveys (where available)
  or online
• WESS State-of-the art data collection,
  reporting, and distribution system. Programmed
  and maintained by EBI software engineers.
• View Reports Online Provides visualizations of
  results and ability to download responses and/or
  calculated data for higher-level analysis
• Share Reports Online Ability to allow access to
  online reporting to others
• Written Analysis Analysis notebooks with full
  detailed descriptive analysis (factor and
  question means, frequency distributions, etc.)
  Analysis also available online in PDF format.

32
Assessment Features

• Comparisons
• Inter-Institutional Comparisons Provide
  comparative information for high level
  decision-making and resource allocation
• Intra-Institutional Comparisons The ability to
  code units (areas/hall/floors, chapters, or
  course sections) for internal comparison to
  support internal improvement
• Longitudinal Comparisons Provide trend data to
  assess the impact of initiatives
• Customization
• Statistical Analysis Provides statistical
  testing and information for resource allocation
• Institutional Specific Questions Institutions
  may add up to 10 questions (5 can be open-ended
  if using our online surveying system, WESS)

33
Data Triangulation
34

• What Is Data Triangulation?
• Application and combination of several data
  points or sources to overcome weaknesses of
  single-measure studies
• Why Use Data Triangulation?
• Increase confidence in findings through
  convergence of different perspectives
• The point at which perspectives converge is seen
  to represent reality
• Builds a rich data resource

35
Astins I-E-O Model
Environments CSS, YFCY, EBI, NSSE (e.g., place
of residence, interactions with peers and
faculty, engagement, programmatic features,
curricular co-curricular experiences)
Inputs CIRP Freshman, BCSSE, MAP-Works (e.g.,
academic performance in high school, financial
concerns prior to college entry, expectations
for college, degree aspirations, self-concept in
high school)
Outcomes YFCY, CSS,EBI, NSSE (e.g., post-college
aspirations, satisfaction with college, academic
and social adjustment, degree completion rates)
36
Similarities and Differences

• Comparing these instruments, well be answering
  the questions...
• What are the unique features of these
  instruments?
• What are the similarities between these
  instruments?
• What are the differences between these
  instruments?
• How do these instruments complement each other?
• Next, how these instruments address
• Retention
• Integration
• Learning Outcomes
• First Year Program Improvement

37
Retention
38
NSSE and Retention

• Links between NSSE Retention Goals
• NSSE is a stimulus for reflection on what the
  institution does well areas for improvement
• Focuses attention on ultimate goal of persistence
  improved educational experiences (not just
  retention)
• Identifies what distinguishes the institution,
  where
• Student experience matches espoused mission
• Student experience falls short
• NSSE results can be connected to institutional
  data to build persistence models

39
NSSE and Retention

• NSSE items and Tintos Model of Student
  Persistence
• Students engagement in effective educational
  practices as indication of level of academic
  social integration (1a-v 6a-f 9a-g etc.)
• Institutional environment items indicate the
  level of commitment to student success (10a-g)
• Student intentions to engage in educational
  activities before graduating (7a-h)

40
NSSE and Retention

• Retention Indicators
• Monitor satisfaction
• Examine intent to engage in enriching educational
  experiences
• Identify problematic student behaviors (e.g.,
  preparing for class preparing 2 or more
  drafts frequency of contact with faculty outside
  of class)
• Retention Studies
• Study relationship between engagement and
  first-year retention
• Compare persisters vs. non-persisters
• Study program impact (e.g., first-year learning
  communities course-based service experience)

41
NSSE and Retention at one institution
42
CIRP and Retention

• CIRP Freshman Survey asks students to predict the
  likelihood that they will transfer, stop-out, or
  leave
• YFCY asks if they have transferred from a
  different institution CIRP TFS pre-test data
  will be merged with the YFCY responses even if
  the pre-test was administered at a different
  institution
• YFCY asks students to share their plans for their
  sophomore year, i.e., intent to re-enroll
• Data can be merged with information from the
  registrar to compare predicted vs. actual
  retention rates

43
CIRP and Retention

• One of the biggest problems with studying
  retention is that it is inextricably linked with
  who you admit (pre-college characteristics and
  performance)
• Longitudinal assessment (e.g. CIRP Freshman
  Survey and YFCY) allows institutions to control
  potentially-biasing background characteristics
  when assessing first-to-second year retention
  rates
• Many of the so-called highly productive
  institutions turn out to be underproductive when
  the caliber of their entering students is taken
  into account. Alexander Astin, 2003

44
CIRP and Retention
Consideration of inputs and environments in
retention analyses identifies populations of
at-risk students and programs/experiences that
facilitate or inhibit persistence

• Ex. What entering characteristics positively
  predict first-to-second year enrollment?
  Emotional health is a positive predictor, but
  recent cohorts of students are exhibiting more
  stress greater levels of depression
• Ex. How do service-learning, first-year seminars,
  and learning communities affect the decision to
  re-enroll? Service learning is a positive
  predictor on its own First-year seminars and
  learning communities are effective when taken in
  combination

45
EBI and Retention
Enrollment Management/Retention
Student Affairs
Academic Affairs

• Who is responsible for student success on your
  campus?
• Enrollment Management/ Retention?
• Student Affairs?
• Academic Affairs?
• What information do you know about this
  first-year student?

Student ID YD252952HS GPA 3.93SAT Verbal
29Location In stateGender FemaleRace
African AmericanAge 18Major Undecided
Do you really know them?
46
Paradigm Shift
EBI and Retention
Im struggling in my math class
Im thinking about transferring
Student Affairs
Academic Affairs

• What would happen if
• ALL faculty/staff were responsible for student
  success?
• YOU knew student was struggling?
• Could you do something about it before it was too
  late?

Residence Hall Staff
Im really homesick
I dont think I can afford college
My roommate and I argue all the time
47
MAP-Works Process
48
Integration to Campus
49
NSSE and Engagement

• What is the level of student engagement on your
  campus?
• NSSE data demonstrates what students do and what
  students believe the institution emphasizes
• Results can be used to shape new student behavior
• Criterion reference benchmarking to compare
  against predetermined value (e.g., 20 FY
  students worked with peers on assignments outside
  of class should this be higher??)

50
CIRP and Student Involvement

• Astins Involvement Theory
• The amount of student learning and development
  associated with any educational program is
  directly proportional to the quantity and quality
  of student involvement in that program
• The effectiveness of any educational policy or
  practice is directly related to the capacity of
  that policy or practice to increase student
  involvement

51
CIRP and Academic Involvement
CIRP TFS CIRP YFCY
Studied gt 6 hours/week 49 68
Look up scientific research articles and resources 23 29
Revise your papers to improve your writing 54 53
Seek feedback on your academic work 49 42
Ask questions in class 59 34
Been bored in class 38 37
Asked professor (teacher) for advice after class 28 15
A average 38 21
52
CIRP and Co-Curricular Involvement
CIRP TFS CIRP YFCY
Performed volunteer work 89 63
Attended religious services 79 53
Socialized with Friends gt6 hrs/week 70 79
Student Clubs or Groups gt 3 hrs/week 40 31
Online Social Network Sites gt 3 hours/week 43 56
53
EBI and Interaction

• First-Year Initiative Assessment
• Course Improved Connections with Faculty
• Course Improved Connections with Peers
• Sense of Belonging and Acceptance
• AFA/EBI Fraternity/Sorority Assessment
• Fraternity/Sorority Enhanced Interpersonal
  Relationship Skills
• Top predictor of Overall Program Effectiveness

54
EBI and Interaction

• FoE/EBI Foundations of Excellence Student
  Assessment
• Top two predictors of Overall Evaluation of
  Program
• Overall Evaluation Transition Support
• All Students Campus Environment
• ACUHO-I/EBI Resident Assessment
• Interaction with Others in the Hall is a top
  predictor of Overall Evaluation of Program for
  90 of schools

55
EBI and Interaction

• MAP-Works all about Interaction
• Levels of Personal One-on-One Interaction
• Student to Faculty/Staff (like FYE Instructor /
  Advisor / Hall Director)
• Faculty/Staff interactions with other
  Faculty/Staff
• Upper Administration interaction with
  Faculty/Staff
• Upper Administration interaction with Student
• Aggregate Study of Student Interaction
• The factor, Social Integration is top predictor
  of a successful social transition to college

56
First-Year Program Improvement
57
NSSE and FYP Improvement

• Identify strengths and areas for growth
• Ex. FY students frequently engage in active
  learning () but are involved in enriching
  educational experiences at low levels (-)
• Assess effectiveness of FY interventions
• Ex. Low student faculty interaction scores at
  small college prompts institution to establish
  mentoring program, and involve students in
  undergraduate research 2 years later NSSE
  results rise on this benchmark

58
NSSE and FYP Improvement

• Sewanee University of the South Concern about
  FY students low active and collaborative
  learning results led Sewanee faculty to
  rejuvenate first-year seminar course, increase
  active pedagogies collaborate with student
  affairs to create rich co-curricular elements.
• Worcester Polytechnic Institute NSSE results
  showed FY students less engaged than seniors.
  Created new FY interdisciplinary, inquiry-based
  seminars better integration of disciplines
  engaging introductory courses. Associate Dean
  appointed to Office for the First Year.
  Assessment plan for the FY in development with
  NSSE indicators as key component.
• UW Green Bay Used NSSE items in Freshman Seminar
  Pilot Study to test impact of revised seminar.

59
CIRP and FYP Improvement

• Several common first-year programs are included
  on the survey
• First-year seminars
• Service learning
• Student-centered pedagogies
• Special Interest Housing
• Tutoring
• Remedial education
• Diversity/multicultural experiences
• Student-faculty research partnerships
• Measures satisfaction with numerous campus
  services and facilities

60
CIRP and FYP Improvement

• Using YFCY Data to Improve Programs
• Identify patterns of participation in specific
  programs
• Measure levels of satisfaction with
  services/programs
• Conduct sub-group analyses of involvement in
  programs to examine differential involvement
• Ex. First-year seminars and involvement with
  faculty
• Identify the impact of programs and services on
  key outcomes of the first year individually and
  in combination

61
EBI and FYP Improvement

• First-Year Initiative Assessment - Mission of
  study is the improvement of the first-year
  course/seminar. Pertinent factors include
  Course Improved
• Study Strategies
• Academic and Cognitive Skills
• Critical Thinking
• Connections with Faculty
• Connections with Peers
• Out-of-Class Engagement
• Knowledge of Campus Policies
• Knowledge of Academic Services
• Managing Time and Priorities
• Knowledge of Wellness

62
EBI and FYP Improvement

• ACUHO-I/EBI Resident Assessment
• AFA/EBI Fraternity/Sorority Assessment
• ACUI/EBI College Union Assessment and Student
  Activities Assessment
• All studies contain class standing as a
  categorical question. Institutions can filter
  results to first-year students to understand the
  impact of their programs
• All assessments mission is program improvement
  by understanding students perceptions of their
  program

63
EBI and FYP Improvement

• Foundations of Excellence Student and
  Faculty/Staff Assessments
• Both studies measure the institutions delivery
  of the first year of college from the first-year
  students perceptions and the faculty/staffs
  perceptions
• Assessments are integrated into a full self-study
  of the institutions first year of college
  overseen by the Policy Center.

64
EBI and FYP Improvement

• MAP-Works!
• Study measures first-year students expectations
  of their first-year and their perceptions of
  their environment during that initial transition
• Online reports are sent to each student and their
  advisor, hall director, and/or first-year course
  instructor.
• Students reports focus on the top areas of
  transition (i.e. study time, self-management,
  making friends, residence hall transition, etc.)
• Faculty/staffs reports focus on areas of risk
  self-reported by student (i.e. no support system,
  few expected study hours, expected graduation,
  etc.)
• Institutions reports focus on areas of low
  performance by large segments of population to
  support improvement of first-year programs.

65
Learning Outcomes
66
Self-Reported Outcomes

• Can be trusted so long as
• Questions are clear, with singular meanings and
  interpretations.
• The information requested is known to
  respondents.
• Respondents believe the questions merit a
  thoughtful response.
• Response options are complete and appropriate.
• Questions avoid risk to the respondent, and do
  not encourage the respondent to respond in
  socially desirable ways.

67
NSSE and Learning Outcomes

• Provides student learning process and outcomes
  indicators
• To what extent are FY students developing their
  writing skills?
• FY writing papers gt 5 pgs.
• FY students who prepare 2 or more drafts
• FY reporting substantial gains in writing
  effectively
• FY worked on paper requiring integrating ideas,
  sources

68
NSSE Educational and Personal Gains ( very
much or quite a bit)
NSSE and Learning Outcomes
Self-Reported Educational and Personal Gains from College First-Year Students Seniors
Thinking critically and analytically 81 87
Acquiring a broad general education 82 86
Working effectively with others 66 78
Writing clearly and effectively 72 77
Learning effectively on your own 70 77
Using computing and information technology 65 76
Acquiring job or work-related knowledge and skills 57 72
Speaking clearly and effectively 60 72
Understanding yourself 60 66
Analyzing quantitative problems 55 65
Solving complex real-world problems 49 58
Understanding people of other racial/ethnic backgrounds 50 52
Voting in local, state, or national elections 24 23
69
CIRP and Learning Outcomes

• Using TFS YFCY Academic Cognitive Indicators
• Grade point average
• Goes down by ½ point during first year
• Longitudinal change in self-ratings of skills
• Writing ability (8)
• Mathematical ability (-3)
• Academic ability (-5)
• Self-perceived changes
• General knowledge (21 report much stronger)
• Critical thinking skills (19 report much
  stronger)
• Analytical/problem-solving skills (17 report
  much stronger

70
CIRP and Learning Outcomes

• Using TFS YFCY Other Indicators
• Identity development
• 4 in personal rating of self-understanding
• Interpersonal skills
• 13 report their ability to work as part of a
  team
• Civic engagement
• 8 in goal to keep up to date with political
  affairs
• Awareness and acceptance of diversity and
  multiculturalism
• 3 in goal to improve understanding of other
  countries/cultures

71
EBI and Learning Outcomes

• Academic Assessments are linked to professional
  learning outcomes standards
• Undergraduate Business Exit Assessment (AACSB
  Standards)
• Nursing Exit Assessment (AACN Standards)
• Teacher Exit Assessment (NCATE Standards)
• Engineering Exit Assessment (ABET Standards)
• Student Affairs studies are linked to
  professional learning outcomes standards, for
  instance
• ACUHO-I/EBI Resident Assessment (ACUHO-I, CAS,
  and NASPA/ACPA Learning Reconsidered)
• AFA/EBI Fraternity/Sorority Assessment (AFA, CAS,
  NASPA/ACPA Learning and Leadership Reconsidered)
• ACUI/EBI College Union Assessment (ACUI and CAS
  Standards)
• ACUI/EBI Student Activities Assessment (Learning
  Reconsidered, Leadership Reconsidered, CAS
  Standards)

72
Using Multiple Data Sources
73
Linking Data

• In-house surveys
• National surveys (a growing list)
• CIRP
• TFS/ YFCY / CSS
• NSSE
• CSEQ / CSXQ
• EBI Benchmarking surveys
• Noel Levitz Student Satisfaction Inventory
• ETS Major Field Tests
• ACT Collegiate Assessment of Academic Proficiency
  
• Institutional data GPA, financial aid,
  transcripts, retention, certification tests, etc.

74
NSSE, EBI, and Engagement

• Washington State University study What is the
  relationship between residence hall environment
  and student engagement? Is there a difference by
  gender? (Robert Tattershall, Director, Housing
  and Conference Services, Washington State
  University)
• Used EBI data to characterize those residence
  hall floors with high and low interaction
• Identified correlations between NSSEs engagement
  measures and the residence hall interaction
  levels
• Found gender differences

75
NSSE, CIRP, and Retention

• Disaggregated NSSE results from seniors by those
  who started at the institution as first-year
  students and those who entered as transfer
  students
• Better understand the transfer student
  experience.
• Combines with CIRP to develop a fuller portrait
  of the transfer student experience.

76
CIRP, NSSE, and FYP Improvement at Elon

• Elon University
• High institutional ethos of assessment
• Building a student database
• Value-added assessment methodology guides studies
• Both CIRP/YFCY and NSSE
• Lead to additional research
• Are utilized for longitudinal study
  benchmarking
• Provide valuable measures of cognitive skills
• Assess student interaction with faculty
• Help to understand overall educational experience

77
CIRP, NSSE, and FYP Improvement at Elon

• Key uses of YFCY
• Time allocation
• Benchmark for issues of campus community
• Cognitive and skill changes
• Measuring use and satisfaction with campus
  programs and services
• Facilities evaluation (classrooms, labs, library)
• Social activities and behaviors
• Primarily used by Student Life , more recently,
  General Education Office

78
CIRP, NSSE, and FYP Improvement at Elon

• Key uses of NSSE
• Activities inside the classroom (e.g., speaking
  up, presentations, working with students)
• Activities outside the classroom (e.g., plays,
  museums, events)
• Evaluation of various functions (e.g.,
  administrators, academic advising)
• Academic rigor
• Primarily used by Student Life Academic Affairs

79
Survey Administration Cycles

• Some Examples
• Administer all surveys to all students and merge
  them together with campus data to create an
  extensive student information system
• NSSE every 3 years, alternate CIRP, EBI
  institutional surveys
• Administer NSSE and EBI two years in a row to
  establish baseline data and assess needs for
  first-year learning communities
• Administer NSSE to a random sample of first-year
  students and YFCY to those remaining
• Administer CIRP TFS and YFCY and/or BCSSE and
  NSSE for program participation patterns and
  outcomes supplement more detailed programmatic
  information from EBI

80
Q A
81
Contact Us

• Cooperative Institutional Research Program (CIRP)
• Higher Education Research Institute at UCLA
  www.heri.ucla.edu
• National Survey of Student Engagement (NSSE)
• Center for Postsecondary Research, Indiana
  University
• www.nsse.iub.edu
• Educational Benchmarking (EBI)
• EBI Service Center, Springfield, MO
• www.webebi.com