Georgia SIG: Scaling Up and Evaluating Professional Development

1
Georgia SIG Scaling Up and Evaluating
Professional Development

• Marlene Bryar
• Director, Division for Exceptional Students
  (Georgia DOE)
• John OConnor
• SIG Director (Georgia DOE)
• Laura Brown
• Director, North Georgia Learning Resources System
• Lynn Holland
• Program Manager, (Georgia DOE)

2
Background Information
3

• Georgia Learning Resources System (GLRS)
• Network of 17 centers across Georgia that provide
  professional development to teachers, parents,
  other stakeholders.
• Approx. 30 professional staff and 17 clerical
  staff around state.
• Approx. 5 million annually.

• Georgia State Improvement Grant (SIG)
• Awarded SIG in 1999.
• Awarded a second SIG in 2004.

4
Georgia Learning Resources System
5
Georgia SIG and GLRSThey intertwine in delivery,
personnel and goals.
6
Our objectives for today

• Review a framework for designing effective
  professional development systems, including the
  evaluation systems.
• Share specific small professional development
  efforts that have grown into large-scale
  initiatives.
• Discuss the evaluation components used for each
  initiative.
• Review our List of Lessons Learned.

7
Two Layers of Lessons Learned

• Lessons from a State Education Agency
  perspective.
• Lessons from a GLRS (regional provider)
  perspective.

8
Table Activity

• Similar to the activity in the Panel Presentation
• At each of your tables, there is a challenge for
  you to solve as a small group. You have 5
  minutes.

9
Debrief Table Activity

• Each challenge consisted of your group designing
  a professional development system so that you
  could become an expert in a particular area.
• Golf
• Cake Decorating
• Tennis
• Making Stained Glass Windows
• Running a marathon

10
Name one element that was included in your plans
11
Did Everyones Plans Include the Following?

• Setting clear goals
• Determining summative evaluations
• Conducting initial needs assessment to determine
  your starting point
• Determining validated interventions
• Getting buy-in from 1) others you need on your
  team, or 2) others that will be impacted.
• Determining short-term objectives and formative
  assessments (including timelines)
• Surrounding yourself with others who are pursuing
  the same goals.
• Determining practice schedule with effective
  feedback from experts.

12
In a school context, it will look like
13
Building Systems Of Change
Adult Change Student Change
Effective Staff Development Practices



Validated Practice
Needs Assessment
Partner with Administrators Stakeholders to
Develop Plans For special education improvement
specialists, determine school-wide and
district-wide initiatives that need expansion or
modification to address needs of students with
disabilities. You must have Administrative
Diligence for any project to impact adult
practices and student performance.
Ongoing Support Coaching
14
Building Systems Of Change
1. Identify Problem
2. Identify Current Functioning Levels and
Develop Goal(s)
1. Identify Research- Based Solution to the
Problem
Implement Summative Evaluations
1. Determine Curriculum Series of Training
Sessions
2. Include modeling, practice, feedback
3. Develop Summative Evaluations
2. Compare School Practice With Best Practice to
Determine Performance Gaps
Adult Change Student Change
Effective Staff Development Practices



Validated Practice
Needs Assessment
Partner with Administrators Stakeholders to
Develop Plans For special education improvement
specialists, determine school-wide and
district-wide initiatives that need expansion or
modification to address needs of students with
disabilities. You must have Administrative
Diligence for any project to impact adult
practices and student performance.
Ongoing Support Coaching
Revise Replicate Change Efforts in Other
Schools
1. Create Service Provider Groups Coaching
Services That Support New Skill Implementation
2. Determine and Train Local Personnel
to Facilitate/Coach Service Provider Groups
3. Determine Amount of Time Schedule for the
Service Provider Groups Coaches to Work
Collaboratively
15
1 Lesson Learned

• We had to change our philosophy
• All professional development systems that are
  designed to impact student performance MUST
  contain these elements in order to be effective
• Partnering with stakeholders
• Conducting data-based needs assessments
• Using validated interventions
• Providing ongoing support and coaching
• Implementing formative and summative evaluations
• It is a waste of taxpayers funds and time to
  conduct one-shot workshops that do not include
  these elements.

16
Subcomponent of Lesson 1

• EVERYONE believes in these components regarding
  professional development
• until they want to develop a two-day conference.

17
Lesson Learned 2

• We evaluate the effectiveness of our professional
  development efforts by how much they impact
  student performance and adult practices.
• (You must design the evaluation components, both
  formative and summative, from the beginning.)

18

• Over the last several years, the Georgia SIG and
  GLRS network has conducted professional
  development systems that have grown in scale and
  have included effective evaluation components.

19
Example A New Teacher Mentoring Academies

• Background Information
• In Georgia, we have a high rate of special
  education attrition.

20
Background Information Percentage of fully
certified special education teachers in Georgia
(does not reflect highly qualified).
21
Special Education Teacher Retention
22
Pilot Project2001 2002 New EBD Teacher
Mentoring Academies

• The Georgia DOE distributed applications for
  participation.
• Novice teachers were paired with mentors.
• They each received simultaneous training
  regularly.
• Initial training was a full day.
• Met for 2 hours monthly via teleconferencing
  technology.
• Between training sessions, the mentor and protégé
  spent 2 hours/week together.
• LEAs had to pay the mentors 1,500 annually.
• The DOE provided the training.

23
2001 2002 Outcomes of the Pilot Project

• 24/25 mentor teachers planned on teaching special
  education the following year
• 23/25 novice teachers planned on teaching the
  following year.
• Pre and post data showed increases in the
  effectiveness of adult practices.
• Pre and post data showed increases in responsible
  behavior in their students.
• The pilot was replicated in 2002-2003 with
  similar results.

24
Lesson Learned 3

• Active participation and satisfaction (real
  ownership) increased when the LEA
• 1) Signed commitment statements up front.
• 2) Committed financial resources.

25
Mentoring Academies Scaling Up

• During the 2003-2004 school year, 10 GLRS
  conducted New Special Education Teacher Mentoring
  Academies (approx. 200 participants)
• 92.6 of the respondents reported their intention
  to teach special education the year following
  their participation in the academy.
• These academies were not limited to EBD teachers,
  but included other special education teachers.

26
Scaling Up Even Further New Georgia SIG

• During the 05 06 06-07 school year, each of
  the 17 GLRS centers will support New Special
  Education Teacher Academies
• 680 Special Education Teachers (including mentors
  and new teachers) will participate in the
  academies.

27
New Teacher Academy Enhancing Evaluation

• In addition to conducting intent
  questionnaires, each GLRS will collect follow-up
  data to determine the number of teachers who
  actually taught special education the following
  year.
• Each GLRS will also collect data on changes in
  adult practices and student performance. These
  goals/evaluation methods will be determined
  locally.

28

• Lesson 4
• The SIG alone can not change statewide systems.
  We had to rely on GLRS and set the same
  expectations for ALL professional development
  initiatives being funded by the Georgia
  DOE/Division for Exceptional Students

29
History of GLRS

• Prior to the 1999 SIG, GLRS had done a good job
  of providing lots of training activities to their
  respective LEAs.
• In 1999, we included GLRS in the shift regarding
  professional development.
• In 2001, during very tough financial times, each
  GLRS was afforded the opportunity to apply for
  funding for Capacity Building Professional
  Development Projects in which they would provide
  professional development to their LEAs.

30

• In 2001, each GLRS submitted a grant for a
  Capacity Building Project(s). They had to
  include data based needs assessment,
  partnerships with stakeholders, formative and
  summative evaluations, validated interventions,
  ongoing coaching, etc.
• Approximately 1 million total was awarded to the
  17 centers.
• This process has been continued annually.
• Funds are not approved until a comprehensive plan
  is provided (in partnership with their
  stakeholders.)

31
Same Requirements Different People

• The same requirements are in place for
  GDOE/Division for Exceptional Students personnel
  and SIG activities.
• For example, if the Visual Impairments specialist
  wants to provide professional development
  opportunities, she must meet the same standard
  before she is granted funding.

32
Lesson Learned 5

• Money talks If you control funding, people
  respond.

33

• One example of a GLRS Capacity Building Project
  is the North GLRS Math Initiative

34
Example B North GLRS Math InitiativeSee
handout
35
Example C Expanded Reading First

• 2001 2002 2002-2003 School Years
• The first Georgia SIG supported courses that were
  taught around the state, Effective Strategies
  for Teaching Reading.
• GLRS taught the course in 20 different cohorts
  across Georgia over multiple years.

36
Georgia Reading First

• Two years ago, the Georgia DOE was awarded a
  Reading First Grant.
• In the grant submission, the Georgia DOE stated
  that K-12 special education teachers across the
  state would be provided an opportunity to
  participate in professional development.

37

• The Division for Exceptional Students is
  collaborating with Georgias Reading First
  project to implement three strands of
  professional development.

38
Georgias Reading First

• 1st Strand GLRS provides reading courses to
  K-12 special education teachers in Reading First
  LEAs but who are not at Reading First schools.
  This is paid with Reading First funds.
• Teaching Struggling Elementary Readers was
  delivered 27 times throughout the state with 531
  teachers participating in the course.
• Teaching Struggling Secondary Readers was taught
  to 33 cohorts that consisted of 660 teachers.

39
Expanded Reading First

• 2nd Strand GLRS provides reading courses to
  K-12 special education teachers who are not in
  Reading First LEAs. (Discretionary Funds.)
• 251 teachers participated in Teaching Struggling
  Elementary Readers
• 251 teachers participated in Teaching Struggling
  Secondary Readers

40
Evaluation

• The fluency measure on the DIBELS was used for
  all students, but the data was difficult to
  aggregate because of the differences in course
  delivery and ages of the students.
• For the 2005-2006 school year, teachers will use
  the same informal measure to report changes in
  their practices and the performance of their
  students.

41

• Lesson Learned 6
• Reading First taught us that Special education
  professional development is more powerful as a
  supplemental component to a larger initiative.

42

• Lesson 7
• The communication demands of the initiatives grow
  as the initiative is scaled up.
• Communication about the logistical components of
  the initiatives (e.g. funding streams and
  Continuing Education Units) is as important as
  communication about the content of the initiative.

43
Expanded Reading First (cont.)

• 3rd Strand IHE subcontractors provide academies
  for teachers of low incidence disabilities (D/HH,
  VI, OI, High Risk Language Delayed.) (SIG funds
  and discretionary funds).

44
Expanded Reading First Orthopedic Impairments
Institutes with Follow-Up

• Led by Dr. Kathy Heller, Georgia State University
• Conducted three institutes throughout the state
  for teachers of students with significant
  physical and speech impairments.
• Provided on-site consultation with teachers who
  participated in the institutes.

45
Outcomes for the OI Institute

• Adult Knowledge Change On a 7-point Likert
  scale, teachers self-reported increases
• Knowledge of Decoding Strategies
• Pre-test Mean 3.67 Post-test mean 5.00
• Knowledge of Comprehension Strategies
• Pre-Test Mean 4.75 Post-test mean 7.00

46
OI Outcomes Continued

• Adult Practice Change 233 recommendations were
  made during the 46 on-site visits with teachers.
• 207 of those recommendations (89) were either
  implemented or in progress.

47

• Student Change (Based on individual students)
• Student A Pretest 79 accuracy (word
  identification) with deficits in comprehension
  and fluency Post-test 92 accuracy, 100
  comprehension, and a reading speed of 55 words
  per minute.
• Student B Pretest 52 correct (word
  identification) Post-test 92 correct.
• Student C Pretest read 17 cwpm (correct words
  per minute) Post-test 32 cwpm.

48
Lesson 8

• Some evaluation measures are less formal than
  others.
• The evaluation for some students with low
  incidence disabilities will be more
  student-specific than high incidence disabilities.

49
Lesson 9
You have to think about scaling up before you
start. Often times, pilot projects dont grow to
new sites.
50
Video of the OI Expanded Reading First Efforts