Dennis Sunal, Cynthia Sunal, Cheryl Sundberg, Glenda Ogletree,

1
Dennis Sunal, Cynthia Sunal, Cheryl Sundberg,
Glenda Ogletree, Erika SteeleThe University of
AlabamaCheryl L. Mason Corrine LardySan
Diego State UniversityDean Zollman Mojgan
Matloob-Haghanikar Kansas State
UniversityCollege Science Teachers (SCST)March
2730, 2008, Boston, MA
Impact of Undergraduate Science Course Reform on
Student Outcomes
2
NSEUS
National Study of Education in Undergraduate
Science

• Collaborative Partners
• University of Alabama
• San Diego State University
• Kansas State University

3
Overview

• 1. Background
• Summary of previous research problem
• 2. NSEUS Project Plan
• NSEUS goals
• What is a reformed course?

• Research questions
• Participants
• 3. NSEUS Results to date
• Data collection analyses
• Pilot Study
• National Survey
• Timeline
• Implications

4
This study addresses the overall problem

• How do undergraduate entry-level science courses,
  differing in level of reform, affect student
  learning outcomes?

5
Study components described

• 1) Literature review of research on previous
  undergraduate science reform,
• 2) National survey of faculty and courses at
  institutions involved in a professional
  development program aimed at undergraduate reform
  program,
• 3) Pilot study determining the feasibility or
  proof of concept, of the procedures and
  instruments for gathering data, and
• 4) Description of a national study of a
  population of institutions involved in reforming
  undergraduate courses with the intention of
  improving student outcomes.

6
Literature review of previous undergraduate
science reform Summary of Findings from the
Literature

• Reform courses use inquiry-based instruction to
  increase student achievement
• Process of reform positively impacts use of
  reform pedagogy in undergraduate science
• Graduates of reform courses more likely to use
  inquiry science teaching

7
Summary (cont.)

• Patterns emerging from the studies indicate
  reform coursework is possible, has positive
  effects on participants, and must overcome
  barriers.
• Patterns suggest strategies for planning and
  implementing reform coursework.
• The research base is limited but starting points
  are suggested in it.

8
What is a Reformed Course?

• Incorporates the following characteristics
• Based on national science standards
• Student-centered, i.e. active student roles
• Inquiry-based pedagogy
• Builds on students prior knowledge
• Incorporates interdisciplinary learning and
  collaborative approaches

9
Problem Developed from Literature

• Question
• What is the impact of engaging higher education
  faculty in implementing standards-based reform in
  undergraduate science courses on student learning
  outcomes?

• Study will investigate
• Short- long term effects on learning outcomes
• Comparisons between reform and non-reform courses
• Comparisons of courses demonstrating differing
  levels of reform

10
Study Design
11
NOVA Program

• NASA Opportunities for Visionary Academics
• Created to develop and disseminate a national
  framework for enhancing education of pre-service
  teachers in science, mathematics, and technology.
  (103 institutions in NOVA Network)
• Mission to improve the STEM literacy of future
  teachers by implementing standards- and
  research-based change nationally in higher
  education using NASAs unique content.
• NOVA Program Web Site
• http//www.nasa.gov/audience/foreducators/postseco
  ndary/grants/NOVA.html

12
Nova Professional Development Model
Professional Development
Collaboration among institutions
Interdisciplinary Collaboration
Standards-based Courses
Instructional Technology
Inquiry-based Instruction
NASA-based Content
Ongoing Research
13
2. NSEUS national survey to characterize study
population

• Conducted a survey of faculty and courses at
  institutions involved in a national professional
  development program

• Surveyed faculty and courses at institutions
  involved in undergraduate course reform

14
NSEUS Project Timeline

• Year 1
• Revise and complete literature review
• Survey characterize population
• Plan data collection protocols instruments,
  select study sample

• Conduct training on instruments and in research
  site protocols for all data collectors, and
• Begin pilot study,

15
NSEUS Study Population
16
NSEUS National Survey

• Description of Institutions (Study Population)
• 103 institutions
• 354 faculty in collaborative teams of 3
• 185 courses
• Diverse
• institutions

17
Institutions Offering Reform Courses Created
under the NOVA Program
Institutions Number of Courses Institutions Offering NOVA-Like Courses
Reform Courses still offered 72 146 41(104)
Reform courses stopped 31 39 8(14)
Total 103 185 49(118)
18
Instructional Methods Used in Reform Courses
Instructional Method Average of Time per Week
Lecture 14
Discussion/Interaction of student groups 15
Integrated inquiry lab 64
Integrated use of technology 07
19
Selected Reform Course Characteristics
Characteristic Average
Years course offered at institution 6
Times offered in a year 2
Course enrollment 35
Minority enrollment 25
Credit hours 4 (range 3 - 5)
Sections per semester 1.5 (range 1 - 6)
Students enrolling in reform courses per year 10,000
20
3. NSEUS case study to determine the feasibility
of the procedures and instruments

• Case (pilot) study to determine validity and
  reliability of

• Instrumentation
• Data collection protocols
• Site visit protocol Classroom science learning
  context (climate)

21
NSEUS Project Timeline

• Year 2
• Select study sample
• Conduct pilot (case) studies with on site data
  collection using 2 institutions, 54 undergraduate
  courses, and 12 elementary teachers
• Complete pilot studies

• Revise data collection protocols instruments,
• Conduct additional training in research protocols
  for all data collectors, and
• Select 30 study sample institutions from
  population

22
Instrumentation (partial list)

• Thinking About Science Survey Instrument (TSSI)
  (Cobern, 2000)
• Classroom Learning Environment Survey (CLES)
  (Taylor Fraser, 1991, 1997)
• Draw-A-Scientist Test (DASTT-C) (Thomas,
  Pedersen, Finson, 2001)
• Science Teaching Efficacy and Beliefs Instrument
  (STEBI A B) (Riggs Enochs, 1990),
• Reformed Teaching Observation Protocol (RTOP)
  (Sawada, Turley, Falconer, Benford Bloom, 2002)
  
• PCK Content Representation (CoRe) Pedagogical
  and Professional experience Repertoires (PaP-ers)
  (Loughran, Mulhall Berry, 2004).

23
DAST Results

• Short-term misconceptions of science, pre-post
  with undergraduate courses (lower score
  represents higher level)
• 2 institutions, 4 courses (2 reform/ 2
  comparison)
• Significant change higher posttest DAST scores

24
CLES Results

• Reform course students report more positive
  classroom learning environment (CLES)
• Environment related to science learning outcomes

25
Undergraduate Science Classroom Observations
Using the RTOP
Classroom Observations Made in the Reform Undergraduate Science Course Classrooms Common Classroom Observations Made in Both Types of Undergraduate Science Classrooms Classroom Observations Made in the Comparison Undergraduate Science Course Classrooms
Extensive student-student interaction during the class Extensive teacher-student interaction during the class Lectures were short and provided in a just in time manner coordinated with students inquiry activities Lecture and laboratory were integrated Teachers used technology smart boards, PowerPoint etc. Content presented in both courses was current, appropriate, and accurate. Little requested, or planned, student-student interaction Teacher lecture took up the majority of the time Students appeared bored and unengaged with the teacher and the content Lecture and laboratory were separated in time
26
Reform Undergraduate Course

• Pedagogy in Study Reform Courses Weekly Time
  Use

27
Pedagogy in Study Comparison Courses Weekly
Time Use
Comparison Undergraduate Course
28
4. NSEUS National Study

• RESEARCH QUESTIONS
• 1. What are the essential characteristics of an
  entry level reformed undergraduate science
  course?
• 2. How do reform science course characteristics
  differ from traditional courses?
• 3. How do course characteristics relate to
  undergraduate students short-term learning
  outcomes?
• 4. How do characteristics differ between courses
  with varying degrees of reform?
• 5. How do varying degrees of reform relate to
  undergraduate students short-term learning
  outcomes?
• 6. How do reform and traditional courses differ
  in their long-term impacts on K-6 teachers in
  their own science classrooms?

• National Study of a population of institutions
  involved in a long term professional development
  program, reforming undergraduate courses with the
  intention of improving student outcomes.

29
NSEUS Project Timeline

• Year 3
• Collect begin analysis of data from national
  sample in institutions and elementary classrooms
• Conduct data analyses

• Year 4
• Collect data from national sample of institutions
  and elementary classrooms
• Conduct analyses of data

30
Research Participants

• 30 institutions nationally (stratified random
  sampling of NOVA institutions), 2 courses from
  each institution (reformed comparison)
• 3 Regions coordinated by 3 Co-PIs East (UA),
  Central (KSU), West (SDSU)
• Faculty (60), students (4000), and in-service
  K-6 teachers (180) of reformed comparison
  undergraduate science courses at sample
  institutions

31
NSEUS National Study Sample

• Description of Institutions (Study Sample)
• 30 institutions
• 142 faculty in collaborative teams of 3
• 60 courses
• Diverse
• institutions

32
Data Collection

• Faculty, students, and in-service teachers
• Content analysis of course materials
• Multiple site visits with classroom observations
• Student perceptions of science, science
  achievement outcome product measures
• Interviews and focus groups

33
NSEUS Project Timeline

• Year 5
• Complete data collection
• Complete data analysis
• National dissemination of results and conclusion
  on undergraduate science reforms

• Conduct National Conference on Undergraduate
  Science Reform and its impact on faculty,
  undergraduate students, and inservice teachers

34
Contact Information

• http//nseus.org

35
National Study of Education in Undergraduate
Science