Issues Surrounding the Evaluation of Teacher Internship Programs

1
Issues Surrounding the Evaluation of Teacher
Internship Programs Donna Barrett Center for
Education Integrating Science, Mathematics
Computing Georgia Institute of Technology Atlanta,
Georgia
2

• . a collaborative effort of corporations,
  universities and school districts designed to
  enhance mathematics and science experiences of
  Georgia teachers and their students. Since 1991,
  teachers have been placed in over 1100 GIFT
  internships throughout the state.

3
Goals of

• Increasing the teachers personal knowledge base
  of science, mathematics and technology
• Enhancing the teachers enthusiasm for teaching
• Providing access to new techniques and procedures
  
• Encouraging partnerships with business and
  education

4
The GIFT Experience - Commonalities

• Common application
• Internship in STEM-based work setting
• Emphasis on Real World problem solving and
  reasoning
• Development of Inquiry Based Unit Action Plan

5
The GIFT Experience - Variables

• Location of placement
• University
• Corporate
• Informal science/public education
• Goal of placement
• Traditional
• Student research
• International
• Curriculum development
• Length of placement
• 4-8 weeks

6
The GIFT Experience - Variables

• Geography
• Placements across Georgia
• Teaching Level
• High school vs. middle school
• Teaching Field
• Science, math, technology, social science
• Type of school
• affluent vs. low income, stable vs. changing,
  minority vs. majority

7
GIFT Evaluation Plan

• An Advisory Board composed of 25 members from
  corporations, universities, and school districts
• Utilizes the strategy of subcommittee phone
  conferences to provide guidance on program issues
  such as evaluation
• Evaluation subcommittee has provided input on
  program evaluation
• Worked with an internal evaluator to develop a
  GIFT Logic Model and develop assessments

8
Assessing Impact on Teachers

• Participant Reactions
• Numbers
• Are participants satisfied?
• Teacher and Mentor Surveys
• Participant Learning
• Action Plan Analysis
• What have teachers learned about inquiry based
  learning?

GIFT Logic Model adapted from Guskeys 5 Level
Model for Assessing Professional Development
9
Assessing Impact on Teachers

• Participant Actions
• How have they applied inquiry based learning in
  their classroom?
• Surveys Pre-Summer, Post-Summer School Year
  Follow-up (Post-GIFT)
• Alumni Survey
• Classroom Observation

GIFT Logic Model adapted from Guskeys 5 Level
Model for Assessing Professional Development
10
Pre and Post Surveys
Based on surveys from www.retnetwork.org
11
Assessing Impact on Teachers

• Organizational Change
• Change in school infrastructure
• Teachers as agents of change
• Alumni Survey
• Focus Group Discussions
• Classroom Observation

GIFT Logic Model adapted from Guskeys 5 Level
Model for Assessing Professional Development
12
GIFT Alumni Survey

• Searching for 668 former participants
• GIFT database/Districts/Websites/Searches
• 123 Fellows No contact information
• 545 Contact Attempts 382 E-mail 163 mailed 36
  E-mail and/or mail returned
• 509 Total Sample Size
• 127 - Surveys returned
• Return Rate 25

13
To what extent, if any, do you feel that you
experienced each of the following types of
learning as a result of your participation in
GIFT?
Strongly Agree or Agree
I gained a greater understanding of the applications science, mathematics, and/or technology in every day life. 93.10
I gained greater understanding of fundamental concepts in science, mathematics or technology. 84.30
I increased my knowledge of current issues in scientific or mathematical research. 91.20
I increased my knowledge of careers that utilize science, mathematics, and/or technology. 95.10
I gained an appreciation of the difficulties some students encounter when learning new material. 78.50
It increased my comfort level with inquiry-based learning strategies. 76.00
It increased my ability to incorporate "real life" examples of the subjects I teach. 96.10
14
Leadership Roles

• 16 reported they became a department or grade
  level chair person AFTER their GIFT experience
• 21 reported they became a school district leader
  (Principal, AP or Coordinator) AFTER their GIFT
  experience

15
(No Transcript)
16
(No Transcript)
17
Limitations of Data

• The data is self reported, although anonymous
• Return rate of Alumni Survey 25
• Sample sizes often vary on the pre and post
  surveys
• Focus Groups have provided more specific
  information

18
Challenges in Correlating Student Achievement
with Teacher Professional Development

• Designing a scientifically valid study with
  appropriate controls
• Appropriate measures of student achievement
  across grade levels, subjects
• Access to student achievement data
• Teachers changing schools, subjects and grade
  levels

19
Assessing the Impact on Students

• Student Achievement
• Connecting the dots through a literature review
• Encouraging the use Research based best practices
  (Inquiry, Problem Based Learning, etc.)
• Impacting teacher content knowledge

GIFT Logic Model adapted from Guskeys 5 Level
Model for Assessing Professional Development
20
Future Evaluation Goals

• Collect student achievement data of GIFT
  participants proposed study with a local school
  district to collect preliminary data
• Criterion-Referenced Competency Tests (CRCT)
  Middle Grades
• End of Course Test (EOCT ) High School
• Biology, Physical Science, Algebra, Geometry
• Aggregate data from specific class and compare to
  state average
• Pre GIFT data, Post GIFT data, YR 1, YR 2 Post

21
Evaluation Tools/Resources

• Columbia Program http//scienceteacherprogram.org
  /
• Industry Initiatives for Science and Math
  Education (IISME) www.iisme.org
• RET Network Surveys www.retnetwork.org
• SWEPT study www.sweptstudy.org
• TRE Conference on Research Experiences.
  http//omp.gso.uri.edu/CTRE/
• Visiting a High School Inquiry Classroom How to
  Prepare and Observe http//cse.edc.org/pdfs/produ
  cts/observerguide.pdf

22
For more information

• Donna Barrett
• Program Director
• donna.barrett_at_ceismc.gatech.edu
• [phone]
• www.ceismc.gatech.edu/gift