Noyce Program Participatory Evaluation

1
Noyce Program Participatory Evaluation

• FRANCES LAWRENZ
• JIM APPLETON
• MARJORIE BULLITT BEQUETTE
• ANN OOMS
• DEENA WASSENBERG
• University of Minnesota

2
(No Transcript)
3
What Existing Research Shows

• Attention to recruitment and retention has varied
  over time
• Different types of incentives to affect
  recruitment and retention have been tried
• Alternative certification is one type of
  incentive and has a variety of definitions
• Salary total amount and comparatively is a clear
  incentive for STEM teachers
• Retention in a school is highly dependent on the
  context of the school (supportive colleagues and
  administration, optimism and focus on student
  success)

4
Theoretical Framework Noyce Logic Model

• The Noyce Program Ideal
• Depicted by the main path as well as bold
  headings preceded by addition signs (e.g., Plan
  to teach)
• Decision points en route to becoming a STEM
  teacher
• Indicated by diverging routes from the main path
  describing alternative options and the Noyce
  Ideal in bold headings
• Dashed boxes denote retention/recruitment by
  school or program
• Important STEM major decision factors
• Influenced by attributes of the candidate,
  pre-service program, and school/district
  (depicted as bulleted lists on the main path)
• Depicted as thought bubbles emerging from the
  decision point

5
Logic model
6
Evaluation Variable Data Summary

• Program itself many are gathering data on
  recruitment, demographics, and student
  performance fewer on program retention
• Post-program monitoring several to many are
  gathering data on teacher effectiveness,
  monitoring/fulfillment of scholarship
  requirements, school/district retention, teacher
  transition experiences/support, and
  inter-program/institution coordination fewer on
  school/district recruitment and characteristics
  (including teaching assignment characteristics)
• Ways of gathering data mostly formative and
  summative program effectiveness, and self-report
  (caution)

7
Use of Noyce Monitoring Data

• Whether the institution uses selected activities
  such as mentoring, early field experiences, etc.
• Information on the students in the program
• e.g., the numbers of students graduating, the
  numbers of career changers
• Information on the applicants for the Noyce
  awards
• e.g., numbers, GPA, support provided
• Information on the faculty involved in the
  program
• e.g., numbers, discipline areas and roles.
• Information about both current and prior
  recipients of the Noyce funding. This includes
• demographic information, GPA, prior background,
  intended teaching areas and levels, levels of
  funding, and completion dates.
• prior recipients also includes if and where they
  are teaching and if so, what.
• List of the districts involved with the Noyce
  program.

8
Ideas for an Evaluation of Noyce

• Precise definitions of variables/separate
  district and program recruitment and retention
• Numbers and demographics at each stage in the
  process of developing a teacher
• Description of the experience at each stage
• Assessment of the state of the candidate
  (performance, attitudes, plans, beliefs) at each
  stage
• Examination of interactions
• Longitudinal studies of persistence, activity and
  impact

9
Issues Raised at Conference Thank You!

• Recruitment both program and district
• Retention in district (program?)
• Training-practice alignment
• Sustainability
• Definitional concerns e.g.,LEA, consistent
• Type of evidence
• Community building
• Long term considerations

10
Existing Resources Some copies at Tables

• Evaluation instruments/processes presently used
  in Noyce
• CETP instruments
• LSC instruments
• CCSSO instruments
• Others??

11
Steps in Planning an Evaluation

• Brainstorm evaluation questions
• Focus through audiences, existing research and
  data, importance, and effort
• Determine information necessary to answer the
  questions
• Establish indicators for the information
  including data sources and collection processes
• Gather, analyze and share the information

12
Group Work15 Minutes recruitment, retention,
programs, and partnerships

• Brainstorm potential evaluation issues/questions
  about your topic
• Prioritize the list
• Speculate on information needs to answer the
  questions and possible indicators
• Consider possible data sources, processes, and
  instruments
• Prepare a 5 min summary for full group

13
Closure and Next Steps

• We will combine and post the suggestions on our
  web site
• We will structure continuing discussions with
  smaller groups of people on targeted issues via
  web or telephone (volunteers)
• We will host another virtual conference in early
  fall to discuss group progress and next steps

14
Contact Us!

• Frances Lawrenz
• lawrenz_at_umn.edu
• Marjorie Bullitt Bequette
• marjb_at_UMN.EDU
• R3 URL
• http//education.umn.edu/EdPsych/NOYCE/
• Additional input to the site is welcomed