Making a Difference in Science Education for Underrepresented Students: The Impact of Undergraduate Research Programs

1
Making a Difference in Science Education for
Underrepresented Students The Impact of
Undergraduate Research Programs

• Kevin Eagan
• Gina Garcia
• Felisha Herrera
• Juan Garibay
• Sylvia Hurtado, Principal Investigator
• Mitchell Chang, Principal Investigator
• Higher Education Research Institute, UCLA
• 2010 AIR Annual Forum
• Chicago, Illinois

2
Introduction

• Graduate enrollment in science and engineering
  has been increasing
• However, URM enrollment continues to lag behind
• Proportion of URMs in graduate programs during
  2006-2007 academic year
• American Indian 0.4
• Black 4.9
• Latina/o 3.6
• STEM completion rates remain low (esp. for URMs)
• Huang, Taddese, Walter (2000)
• Higher Education Research Institute (2010)

3
Purpose

• To examine the effects of undergraduate research
  programs on students intentions to enroll in
  graduate school through the use of advance
  statistical techniques on multi-institutional
  data.
• Propensity score matching

4
Background

• Graduate School Enrollment
• Prior academic achievement
• Race/socioeconomic status
• Parent education
• Institutional selectivity
• Level of involvement
• Student faculty interaction

5
Background

• Benefits of undergraduate research programs

Retrospective Analyses Single Time Point Longitudinal
Hurtado, Cabrera, Lin, Arellano, Espinosa (2009) Barlow Villarejo (2004) Bauer Bennett (2003) Hathaway, Nagda, Gregerman (2002) MacLachlan (2006) Lopatto (2004) Maton Hrabowski (2004) Russell, Hancock, McCullough (2007) Hunter, Laursen, Seymour (2006) Seymour, Hunter, Laursen, Deantoni (2004)
6
Conceptual Framework

• Social and Cultural Capital
• Capital inherited through social position and
  family background
• Social capital acquired in college complements
  the capital that students bring with them
• Science Identity
• Fostering knowledge growth
• Opportunities to display scientific knowledge
  practices
• Acknowledgement of being a science person

7
Research Questions

• What pre-college experiences and characteristics
  of entering college students predict their
  likelihood of participating in a structured
  undergraduate research program during college?
• After accounting for students chances of
  participating in an undergraduate research
  program, what effect does participation in such a
  program have on students intention to enroll in
  graduate/professional school, particularly in a
  STEM field?

8
Methods Sample

• CIRP Longitudinal Sample (n4,212)
• 2004 Freshman Survey (TFS)
• 2008 College Senior Survey (CSS)
• Targeted institutions
• Strong reputations in STEM graduation rates
• Undergraduate research programs funded by NSF and
  NIH
• Minority-serving institutions

9
Methods Variables

• DV 3-part variable representing post-college
  intentions
• Enroll in graduate/professional STEM program
• Enroll in graduate/professional non-STEM program
• No intentions to pursue graduate/professional
  degree
• IVs
• Undergraduate research participation
• Science identity
• Career focus in 2008
• College GPA
• College experiences
• Pre-college preparation
• Demographics
• Institutional characteristics

10
Methods Analyses

• Missing data
• Propensity score matching
• Discussion of the counterfactual
• Estimation of the propensity score related to
  participation in an undergraduate research
  program
• Multinomial hierarchical generalized linear
  modeling

11
Methods Analyses

• Issues of selection bias/endogeneity
• Counterfactual framework
• a potential outcome, or the state of affairs
  that would have happened in the absence of the
  cause (Guo Fraser, 2010, p. 24)
• Comparing a treated individual with a
  non-treated individual
• Propensity score estimation

12
Methods Analyses

• Reweighting of the data with derivations of the
  propensity score
• Average treatment effect
• Average treatment of the untreated (ATU) effect
• Average treatment of the treated (ATT) effect
• Multinomial hierarchical generalized linear
  modeling

13
Limitations

• Secondary data analysis
• Limited DV intentions and combination of
  graduate and professional school
• Unobservable variables affecting undergraduate
  research participation
• Weighting adjustment using propensity score
  rather than matching by propensity score

14
Findings Predictors of Participating in
Undergraduate Research Programs

• Major Physical sciences (10.77), Life sciences
  (7.34), Health sciences (4.90)
• Race Black (5.71)
• Participated in pre-college research program
  (4.03)
• Degree aspiration in 2004 Ph.D. (3.54)
• Composite SAT score (100-point change) 2.27

15
Findings Effects of Undergraduate Research
Program Participation on Graduate/Professional
School Enrollment Intentions
  Intend to Enroll in a STEM Graduate/Professional Program Intend to Enroll in a STEM Graduate/Professional Program Intend to Enroll in a STEM Graduate/Professional Program Intend to Enroll in a STEM Graduate/Professional Program Intend to Enroll in a non-STEM Graduate/Professional Program Intend to Enroll in a non-STEM Graduate/Professional Program Intend to Enroll in a non-STEM Graduate/Professional Program Intend to Enroll in a non-STEM Graduate/Professional Program
  Delta-P Log odds S.E. Sig. Delta-P Log Odds S.E. Sig.
Average treatment effect (ATE) 7.84 0.39 0.16 4.96 0.23 0.17
Average treatment for the untreated (ATU) 7.95 0.40 0.16 5.98 0.28 0.18
Average treatment for the treated (ATT) 6.91 0.34 0.15 -0.45 -0.02 0.15
Unadjusted multinomial HGLM 8.38 0.42 0.14 1.77 0.08 0.15  
Simple mean comparison 13.50 1.80
16
Discussion

• Confirmation of results from prior studies
• Effect is much more modest than prior studies
  might suggest
• UG research programs attract students who already
  identify as scientists
• Average treatment of the untreated (ATU) effect
• Expand the reach of these programs
• Ensure programs not only harvest talent but
  develop it, too

17
Conclusion and Directions for Future Research

• Follow these students into graduate school and
  examine matriculation patterns
• Investigate via qualitative methods the quality
  of students research experiences
• UG research programs as wise investments

18
Contact Information
Faculty and Co-PIs Sylvia Hurtado Mitchell
Chang
Postdoctoral Scholars Kevin Eagan Josephine
Gasiewski
Administrative Staff Aaron Pearl
Graduate Research Assistants Christopher
Newman Minh Tran Jessica Sharkness

• Monica Lin
• Gina Garcia
• Felisha Herrera

• Cindy Mosqueda
• Juan Garibay

Papers and reports are available for
download from project website http//heri.ucla.e
du/nih Project e-mail herinih_at_ucla.edu
Acknowledgments This study was made possible by
the support of the National Institute of General
Medical Sciences, NIH Grant Numbers 1 R01
GMO71968-01 and R01 GMO71968-05 as well as the
National Science Foundation, NSF Grant Number
0757076. This independent research and the views
expressed here do not indicate endorsement by the
sponsors.