PROFESSIONAL DEVELOPMENT FOR EVALUATORS

1
Leveraging Performance Management to Support
School Priorities

• PROFESSIONAL DEVELOPMENT FOR EVALUATORS
• Wednesday, July 31

2
Getting Started Revising Goals

• Please consider the goals are there any
  revisions you would suggest?
• Characteristics of high-quality goals
• SMART
• Aligned to school priorities
• High leverage
• Includes mid-year benchmark

3
Todays Agenda

• Collecting Evidence
• A Closer Look at Observations
• A Closer Look at Artifacts
• Lunch
• Evaluations
• Breakouts
• Planning for the work at your school
• Managing Directed Growth Improvement plans
• Closing

4
What progress have we made toward the goals of
the training?

• Goal 1
• Evaluators will know how to implement the new
  system (technical 5-step implementation) and how
  to get it done and done well (evaluation best
  practices).

5
What progress have we made toward the goals of
the training?

• Goal 2
• Evaluators will leave with concrete, specific
  plans for implementation in their school,
  including how to
• Use the evaluation system to further their school
  priorities
• Talk with teachers about the evaluation system

6
What progress have we made toward the goals of
the training?

• Goal 3
• Evaluators will leave knowing the specific
  responsibilities of evaluators and teachers under
  the new evaluation system.

7

• Evidence
• Observation Artifacts

8
Evidence is collected throughout the cycle

• Every educator is an active participant in an
  evaluation
• Process promotes collaboration and continuous
  learning

Continuous Learning
9
All Evidence is entered in EDFS

• Both educators and evaluators can upload
  artifacts
• Evaluators upload both observation evidence
  feedback

10
Evidence should be collected on

• Progress on Ratings on OVERALL
• (2) Goals (4) Standards RATING

11
Evidence Collection

• Evidence should be explicitly related to specific
  standards goals.
• When appropriate, evaluator provides feedback to
  educator using language from the rubric.
• Evaluator does not rate educator practice after
  each piece of evidence, but during the formative
  or summative evaluation.

11
12
Evidence vs. Feedback

Evidence Feedback
Facts Aligned to Rubric Ex 1 Lesson plan submitted as artifact Ex 2 Ms. Y said, "Yes, I know. You are working hard. You can do this. Claim Interpretation Aligned to Rubric Ex 1 The lesson plan does not include a closing activity. How is summarizing incorporated into the lesson? Ex 2 Ms. Y's response show that she continues to foster a safe intellectual environment where students take academic risks.  
13

• Evidence from Observations

14
Observations

• Turn Talk
• What have your experiences with observation been
  like?

15
What the Research Says . . .

• Evaluators need multiple opportunities and
  settings to observe and assess educator practice
• Multiple observations paired with timely feedback
  are a key part of a strong evaluation system

16
Observing Lessons CAUTION!

• We observe all the time, which means that it is
  easy to do, but hard to do objectively.

• Remember
• We tend to see what we want to see
• We have to be aware of and avoid bias,
  particularly personal idiosyncrasies
• We have to avoid preconceptions
• Observation should be based on agreed upon
  criteria

17
Observation Bias
What is bias? What are some experiences you have
had with bias? What are some examples of bias
you have about teaching? What can we do to
diminish the influence of bias in our work?
18
Principles of Brief Observations

• Frequent
• Focused
• Varied
• Useful and Timely Feedback

19
Strategies for Collecting Evidence

• Identify a focus ahead of time
• Goals, specific Elements
• Record evidence, not judgment
• Quotations, observed actions or movements by
  teacher and students, times, numbers, literal
  descriptors, etc
• Be an efficient note-taker
• Establish abbreviations, paraphrase
• When possible, meet with the teacher
• Check in briefly to ask questions/give feedback

20
Examples

• Observations Tools from Sara
• Observations Tools from Mary

21
Other forms of Observations Team Feedback
22
Saras tracking tool
23
CHS approach to scheduling
24
Systems for Tracking Observations
25
Observations in the EDFS

• BPS developed an observation tool in EDFS
• Evidence and feedback should be entered in EDFS,
  and tagged to an appropriate area of the rubric
• Observations may only be used in formative or
  summative evaluations if the educator received
  feedback within 5 days of the observation

26
Observations EVALUATOR View
27
Observations EVALUATOR View
28
Observations EVALUATOR View
29
Next version of the observation tool
30
Next version of the observation tool
31
Observations EVALUATOR View
32
Observations Educator Evaluator View
33
Observing Practice

• Chris McCloud
• 7th Grade Math Teacher

34
C. McClouds self-assessment against the
rubric
Strengths Needs
Improving student performance in math for students who enter my class performing below grade level Indicators/Elements Teaching classrooms with diverse needs (especially meeting the needs of student with an IEP) Indicators/Elements Family outreach and communication Indicators/Elements Additional support for implementing the revised MA Curriculum framework Indicators/Elements Improving communication with families for whom English is a second language Indicators/Elements Strengthen leadership skills Indicators/Elements
34
35
Chris McClouds Goals

• SL Based on the fact that my 7th grade ELL
  students averaged 30 on my pre-assessment, my
  goal is that they will average at least 80 on
  each of the six unit assessments that they will
  take before May 15.
• PP In order to support my ELL students in
  averaging 80 on unit assessments, I will
  consistently identify and teach symbols, key
  terms and other math vocabulary, and use daily
  exit tickets that measure both vocabulary and
  conceptual understanding. I will measure my
  progress through student vocabulary notebooks and
  tracking exit ticket performance data.  

36
Observation of Chris McCloud Selective Scripting

1. Read over the two excerpted elements from the
   rubric (I-A-4. Well-Structured Lessons, II-A-2.
   Student Engagement)
2. Watch the video.
3. While you are watching the video, collect
   evidence that will allow you to rate Mr. McCloud
   on the two elements. Use the worksheet to
   selectively script evidence related to each of
   the elements.
4. When you are finished, complete the worksheet and
   rate Mr. McCloud.

37

• How did it go?

38
Value of Feedback

• Targeted, Ongoing Feedback
• Improved Educator Practice
• Improved Student Achievement

39
Characteristics of Effective Feedback

• FOCUSED feedback should focus on what was
  observed
• EVIDENCE-BASED feedback should be grounded in
  evidence of practice
• ACTIONABLE feedback should reinforce effective
  practice and identify areas for continued growth
• TIMELY feedback should be provided shortly after
  the observation

40
Constructing Feedback

• Chris McCloud
• Using your observation notes and the rubric
• Draft two points of feedback for Chris
• Focus on feedback that is
• Focused
• Evidence-based, and
• Actionable

41
Feedback conversations

• What has been your experience with feedback
  conversations?
• Volunteers for a roleplay?

42
Feedback conversations Resources

• Beyond the Scoreboard, from Educational
  Leadership
• Talk About Teaching, by Charlotte Danielson
• Leverage Leadership, by Paul Bambrick-Santoyo
• Difficult Conversations, by Stone, Patton Heen

43
Reflect

• What are the implications of what we just
  discussed for your practice? For your school
  community?

44

• Evidence from Artifacts

45
Collecting Evidence What can you see?
In the classroom Outside of the classroom In artifacts/ conversations
Well-structured lessons Student engagement Classroom management Differentiated instruction A safe learning environment Professional collaboration Family and community engagement Reflection Results of data analysis Adjustments to practice
46
Collecting and Assessing Evidence - Artifacts

• Artifacts may include
• Tracking analysis of student assessment data
• Student work
• Lesson plans
• Identify chart possible sources of evidence in
  the standard you are assigned. What kind of
  documents would be useful for teachers upload?

47
Artifacts Clarifying Expectations

• How many would you like people to submit?
• What kind of documents?
• How detailed of a rational would be useful?

48
Artifacts Guidance

• A suggested format for rationales
• Step 1 Identify the standard, indicator,
  element, or goal that the artifact addresses.
• Step 2 Describe the artifact and identify the
  section that directly connects to the standard,
  indicator, element, or goal.
• Step 3 Highlight the artifacts impact on
  student learning.
• Step 4 Specify the evidence of professional
  growth or proficiency the artifact provides.

49
Artifacts Guidance

• Look over the examples on p. 4-6
• What is strong?
• Anything more you would add?

50
Zils SLG and Action Steps
51
Zils SLG Artifact
52
Mid Year Check Point
53
Artifacts Examples

• Looking at these artifacts, what evidence do you
  have of the teachers practice?

54
Trio Activity

• Thinking about a specific team or educator, think
  about which artifacts you would suggest they
  upload.
• Consider what kinds of artifacts would show
• elements that cannot be observed?
• evidence of growth in areas noted for
  improvement?
• progress on goals?
• performance on each standard?

55
Artifacts on EDFS Educator Evaluator View
56
Artifacts Educator Evaluator View
57
Artifacts Educator Evaluator View
58
Artifacts Educator Evaluator View
59
Artifacts Educator Evaluator View
60
Artifacts Educator Evaluator View
61
Artifact Tips

• Start an electronic folder for artifacts on your
  desktop
• Keep file names short (fewer than 32 characters)
• Upload videos somewhere else, and include a link
• Genius Scan app for turning pictures into PDFs

62
Artifact Tips Contd

• All artifacts have to be uploaded to EDFS even
  if the hard copy is not scanned, it has to be
  entered as evidence and a hard copy kept in a
  binder.
• You can comment on artifacts through EDFS, but
  the online system is not a substitute for
  face-to-face conversation.

63
Reflect

• What are the implications of what we just
  discussed for your practice? For your school
  community?

64

• Evaluations

65
Evaluations

• The basics
• Evaluations on EDFS
• Examples

66
What is a formative assessment?

• A mid-plan check-in
• rating on progress towards each goal
• rating on each standard
• A document that considers all evidence
• observations to date
• artifacts to date
• It may be used to change a plan
• If there is a significant change in practice,
  however, this is not mandatory. The plan may
  continue until the summative.

67
What is a summative evaluation?

• A summary at the end of the plan
• rating on progress towards each goal
• rating on each standard
• A document that considers all evidence
• the formative assessment
• observations artifacts since the formative
• consistency or change in practice since the
  formative
• The overall rating determines the next plan

68
Not the evaluation of the past

• Evaluations are NOT write-ups of individual,
  formal observations.

69
Formative Assessment Summative Evaluation

• Meetings
• Upon request of educator or evaluator
• Required for Summative if the overall rating is
  NI or Unsatisfactory
• Recommended for all educators

70
Evaluations in EDFS
71
Formative Assessment
Step 1
Step 2
72
Evaluations in EDFS
73
Evaluations in EDFS Rating Goals
74
Evaluations in EDFS Rating Standards
75
Evaluations in EDFS Overall Rating
76
Evaluations in EDFS Prescriptions
Prescriptions for any standard in which an
educator is rated NI or UNSAT.
77
Evaluations in EDFS Comments
Comments are optional for both the evaluator
the educator
78
Evaluations in EDFS Release
79
Evaluations in EDFS Sign-Off
80
Evaluation Examples

• Read one of the examples you have.
• What do you notice about how evidence is used in
  the write-up?

81
Rationales for Ratings
Standard   I. Curriculum, Planning, Assessment  
Rating   Exemplary, Proficient, Needs Improvement, Unsatisfactory
Rationale Claim/Connection to rubric  
Rationale Evidence 1
Rationale Evidence 2
Rationale Evidence 3
Rationale What claim can you make? What is the
evidence to support it (artifacts, observations)?
82
Reflect

• What are the implications of what we just
  discussed for your practice? For your school
  community?

83
Breakouts

• Group 1
• Planning for evaluation work at your school
• Group 2
• Working with teachers on improvement or
• directed growth plans

84
Planning the work at your school

• Marys document

85
Planning the work at your school

• Guiding Questions
• What has your school done to support staff with
  the evaluation process?
• What additional supports do you think would be
  helpful?
• What are ways of structuring this support at the
  school?
• What timelines should guide the work?
• How can you and your team find the time to do
  this well?

86
Working with teachers rated NI or U

• You can change the plan at the time of the
  formative assessment (though a formative
  assessment does not HAVE to change the plan).

87
Timelines and Requirements
88
Directed Growth and Improvement plans

• Considerations
• Choosing length
• Using a calendar to calculate dates of formative
  assessment and summative evaluation
• Writing goals from prescriptions
• Developing the Action Plan, including supports
  and timelines
• Collection of evidence
• Supports
• Did I sheets and timelines

89
http//educatoreffectiveness.weebly.com

• For more information, visit
• EDFS http//eval.mybps.org/
• http//educatoreffectiveness.weebly.com
• Email questions, comments and feedback to
• bpsevaluation_at_boston.k12.ma.us
• MA Department of Elementary and Secondary
  Education (DESE) Evaluation Site
• http//www.doe.mass.edu/edeval/

90
Office of Educator Effectiveness

• Ross Wilson, Assistant Superintendent for
  Educator Effectiveness
• Chason Ishino, Implementation Specialist
• Jared Joiner, Implementation Specialist
• Emily Kalejs Qazilbash, Implementation Specialist
• Angela Rubenstein, Implementation Specialist
• Kris Taylor, Implementation Specialist
• Jenna Costin, EDFS On-line System Coordinator
• Jen Kozin, Data Analyst
• Evaluator Training Facilitators
• Mary Driscoll, Edison K-8
• Sara Zrike, Hurley K-8

91
Contact us if you have questions

• Ross Wilson, Asst. Superintendent for Educator
  Effectiveness

Network Implementation Specialists BPS email
A Emily Qazilbash eqazilbash
B Nicole Ireland nireland
C TBD eval_at_mybps.org
D Angela Rubenstein arubenstein
E Jared Joiner jjoiner
F Kris Taylor ktaylor
High Schools Chason Ishino cishino
EDFS tech support Jenna Costin jcostin
Data analysis Jen Kozin jkozin
92
What progress have we made toward the goals of
the training?

• Goal 1
• Evaluators will know how to implement the new
  system (technical 5-step implementation) and how
  to get it done and done well (evaluation best
  practices).

93
What progress have we made toward the goals of
the training?

• Goal 2
• Evaluators will leave with concrete, specific
  plans for implementation in their school,
  including how to
• Use the evaluation system to further their school
  priorities
• Talk with teachers about the evaluation system

94
What progress have we made toward the goals of
the training?

• Goal 3
• Evaluators will leave knowing the specific
  responsibilities of evaluators and teachers under
  the new evaluation system.

95
Closing

• Please complete the exit ticket.
• Thank you so much for participating, and please
  be in touch!