SchoolBased Oversight Team Training VGLA

1
School-Based OversightTeam TrainingVGLA VAAP

• Richmond Public Schools
• October 7, 2008

2
School-Based Oversight Teams

• Presenters
• Michelle Boyd
• Juanita Nicholson
• Exceptional Education and Student Services

3
Objectives

• SBOT will be able to
• provide technical assistance to teachers
  collecting evidence for the VGLA and VAAP.
• monitor the development of VGLA and VAAP
  collections of evidence.

4
Virginia Grade Level Alternative Assessment
(Review)

• Presenter
• Michelle Boyd
• Exceptional Education and Student Services

5
Review
evidenced-based alternative assessment for the
SOL test
method of demonstrating knowledge of SOL content
through non-traditional means
student work sample and products are collected
throughout the instructional process and
submitted as a COE for local scoring
VGLA
available to students in grades 3-8 who meet the
participation criteria
participation is made on a test-by-test basis per
content area
participation is determined by an IEP, 504, or
LEP team/committee
6
Participation Criteria for Students with
Disabilities (VGLA Manual p. 23)
1. The student must have a current
IEP/504-plan 2. As a result of a disability,
the student demonstrates his/her individual
achievement of SOL content through means other
than multiple-choice formats 3. As a result of
a disability, the student is unable to
demonstrate his/her individual achievement on the
SOL test for the assigned course or grade level
using available accommodations and/or formats.
7
Participation Criteria for LEP Students (VGLA
Manual p. 24)
1. The student is at Level I or Level II
of English Proficiency.
Maximum participation in reading for students
with LEP is three consecutive years
8
Rule 1 Evidence must be student generated.

• The following cannot be considered for scoring
• Work copied from the board, textbooks, computer,
  etc
• Take-home test, open-book test, or homework
• Work that includes examples that clearly provide
  answers for students
• Group work (unless student contribution is
  clearly identified)

9
Rule 2 Evidence must include an SEI Tag.

• A properly completed SEI tag must be on each
  piece of evidence.
• Evidence without an SEI tag, cannot be considered
  for scoring.

10
Student Evidence Identification (SEI) Tag
Content Area submitted Standard of Learning
defended Bullet of SOL defended (if
applicable) Check one or both
STUDENT EVIDENCE IDENTIFICATION (SEI) TAG
VGLA VSEP Content Area
Mathematics SOL 8.14 Bullet b Inferred
Demonstrated
Beginning with the 2008- 2009 school year, all
items submitted in the COE must include a Student
Evidence Identification (SEI) tag.
11
SEI Tags Inferred vs. Demonstrated Evidence
(p.14)

• Inferred and Demonstrated Evidence may be used.
• DEMONSTRATED knowledge or skill is clearly
  shown in the evidence
• INFERRED Evidence permits a positive conclusion
  to be drawn regarding the knowledge or skill.

12
Rule 3 Evidence must clearly address
accommodations, captions, and grading as
appropriate
13
Accommodations Rule 3 (part I)

• If an accommodation has been used in the
  creation of the evidence, the accommodation must
  be documented in the evidence.
• Note Students must meet specific criteria to
  use the read aloud accommodation on reading
  assessments.
• Additional information in packet

14
Captions Rule 3 (part II)

• If the evidence includes photographs, the
  captions that address the students individual
  achievement must be included.
• Each photograph must have a caption
• Photographs without captions cannot be considered
  for scoring.

15
Grading Rule 3 (part III)

• Evidence must be graded
• Work samples (graded and correct and incorrect
  answers must be clearly identified)
• Audio, video, interview, or anecdotal record must
  include statement(s) of accuracy so that the
  students level of individual achievement is
  clear to the scorer.
• Grades may be a letter grade, numerical grade, or
  evaluative comment.

16
VGLA COE Required Forms

• Student Demographic Form
• Participation Criteria
• Affidavit of Student Performance
• Photo Consent Form (as appropriate)
• COE Cover Sheet
• Administrator Support Documentation
• COE Organizer

17
Required Components
Properly Labeled Evidence
Affidavit
18
Affidavit (VGLA Manual p.25)

• Revised Affidavit of Student
• Performance includes multiple
• signature lines for
• Special Education Teacher
• Course Content Teacher
• General Education Teacher
• Building Administrator/ Designee
• Other

19
RPS Required Documentation

• Student Demographic Form (available in the
  spring)
• VGLA Participation Criteria (VGLA Manual pg. 23
  or pg. 24)
• Photo Consent Form (as appropriate RPS Ex. Ed.
  Website)
• VGLA Coversheet (available in the spring)
• Administrator Support Document (available in the
  spring)
• VGLA COE Organizer in front of evidence

20
VGLA Review Questions
21
Virginia Alternate Assessment Program (Review)

• Presenter
• Juanita Nicholson
• Exceptional Education and Student Services

BB
22
Review
alternate assessment designed for a small group
of students with disabilities
method of demonstrating knowledge of ASOL
content through non-traditional means
a collection of work samples to demonstrate
performance on the ASOLs for which students have
received instruction
VAAP
available to students in grades 3-8 and grade 11
who meet the VAAP participation criteria
participation is available in reading, math,
science, and history/social science (no writing)
participation is determined by an IEP team/
committee and documented in the IEP minutes
23
Virginias Accountability Assessment System
24
What are Aligned Standards of Learning?

• Virginias approach to providing access to and
  assessing students with significant cognitive
  disabilities on the general curriculum (SOL).
• Content standards based on SOL skills that have
  been reduced in depth and complexity.

25
Depth and Complexity

• Reduced in complexity grade level designation
  removed, student/teacher may select any standard
  from the required reporting categories. Skills
  often reflect prerequisites to actual grade level
  in which the student is enrolled.
• Reduced in depth students are required to
  defend only one aligned standard or one bullet
  from each required reporting category.

26
How many ASOL does that mean per content area?

• VAAP students are required to defend one ASOL for
  each reporting category within each content area.
  
• Reading 2
• Mathematics 5
• Science 4
• History 4

27
ASOL and Reporting Categories
28
Rule 1- Evidence Must Show Individual Student
Achievement

• Does the evidence show any level of achievement
  with the ASOL being defended?
• Are all of the ASOL addressed (stem and a
  bullet)?
• Has hand-over-hand (full physical assistance)
  been used?

29
Rule 2- Evidence Must Be Student-Generated

• Has evidence such as homework, open-book tests or
  group work been included?
• If the evidence is from a group project, is the
  work of the VAAP participant clearly identified?
• Has the evidence been copied from the blackboard,
  textbook or computer?
• If the evidence is a work sheet, are there
  examples or directions that provide answers?

30
Rule 3 - Evidence Must Include a Correctly
Labeled SEI Tag

• Does each piece of evidence have a completed SEI
  tag with the content area, ASOL and the bullet
  (if appropriate) listed?

STUDENT EVIDENCE IDENTIFICATION (SEI)
TAG Content Area ASOL Bullet
31
Rule 4- Evidence Must Include a Completed VAAP
Content Area Cover Sheet

• Is there a completed VAAP Content Area Cover
  Sheet for each Content Area included in the COE?

32
Rule 5- Evidence Must Include a Signed
Affidavit Or It Will Not Be Scored

• Is there a signed affidavit that includes all
  teachers and other school staff who have made
  contributions to the COE?
• Has the building administrator (or designee)
  signed-off on the COE?

33
Rule 6- Evidence Must Clearly Address
Accommodations, Photograph Captions and Grading

• If accommodations have been used, are they
  documented on the evidence or on the VAAP Content
  Area Cover Sheet?
• Does each picture have a caption that clearly
  address the students level of achievement?
• Has the work been graded accurately with right
  and wrong answers clearly identified?

34
Why VAAP COEs Fail ?

• Omitted Reporting Category
• Duplicated Reporting Category
• Evidence with hand-over-hand instruction

35
VAAP COE Forms

• Required Forms
• Student Demographic Sheet (RPS Exceptional
  Education Website)
• Participation Criteria (pg. 4)
• Content Area Cover Sheet (pg. 13)
• Affidavit of Student Performance (pg. 13)
• Photo Consent (RPS Exceptional Education Website)
• Optional Forms
• Teacher Checklist for Collections of Evidence
• Instructional Priorities Planning Sheet

36
VAAP Review Questions
37
Collection of Evidence (COE)
Evidence
38
Types of Evidence

• Various forms of evidence can be demonstrated
  under the direct supervision of a teacher or
  other school personnel.
• Evidence may consist of, but is not limited to
• Work Samples
• Audiotape
• Videotape
• Anecdotal Records
• Interviews
• Charts/Graphs
• Captioned Photographs

39
Work Sample

• Must be related to a specific grade level SOL or
  ASOL.
• Should evidence the students level of individual
  achievement and knowledge on the referenced SOL
  or ASOL.

40
Work Samples

• Examples
• Worksheets
• Tests/Quizzes
• Writing Samples
• Book Reports
• Drawings
• Diagrams

41
Audiotape

• Audiotapes can only contain information or work
  generated by the student however, it is
  acceptable for school personnel to read questions
  to the student. The student is not permitted to
  use cue card or notes.
• Scripts are highly recommended.

42
Audiotape

• Examples
• Student answers questions on a specific topic.
• Student reads a selection aloud.
• Student describes a procedure.
• Student demonstrates understanding of vocabulary
  words and concepts.
• Student answers true/false questions.

43
Videotape

• Video tapes can only contain information or work
  generated by the student however, it is
  acceptable for school personnel to read questions
  or interview the student.
• NOTE A signed photo permission form is needed.

44
Videotape

• Examples
• Student completes a math computation at the
  blackboard.
• Student explains directionality using a wall map.
• Student conducts a scientific experiment and
  explains the steps.
• Student is interviewed on the parts of a short
  story.

45
Interview

• Can be administered by the teacher or other
  school personnel.
• Should follow a question answer format.
• Should contain verbatim exchanges between the
  student and the interviewer.
• Questions should be prepared in advance so that
  they adequately cover the SOL or ASOL.
• Interviews may be audio-taped or video-taped.
• The SOL or ASOL should be stated and/or written
  at the beginning of the interview.

46
Anecdotal Record

• May be completed by teacher or school personnel.
• Should contain description of an observed SOL or
  ASOL skill or/knowledge.
• Should contain matter of fact language and not
  a judgment of a skill level. For example Sue
  worked hard on her report.
• Should include the date of performance, a
  description of the observed skill or procedure,
  and the students level of achievement.

47
Example of Anecdotal Record

• SOL 513 Measure and draw right, acute and
  obtuse angles and triangles using appropriate
  tools.
• March 1, 2008 During math lesson, Maria was
  asked to create three types of angles using the
  appropriate tools. Maria picked up the
  protractor and created an acute angle of 35
  degrees, right angle at 90 degrees and a 125
  obtuse angle. Each of these was correctly done.
• March 8, 2008 Maria was asked to measure 5
  different types of triangles. Maria identified 4
  out of 5 correctly.

48
Charts/Graphs

• Should not focus on student progress on a skill.
• Should reflect student skills and/or knowledge
  and may be generated by the teacher and/or
  student.
• Should indicate clear individual achievement of
  the SOL or ASOL referenced.

49
Charts/Graphs

• Examples
• Student generated Venn Diagram.
• Student plots coordinate points.
• Student creates a historical timeline.

50
Captioned Photographs

• Captioned statements should explain the knowledge
  and/or skill demonstrated in the photograph.
• Clearly indicate the student addressing the SOL
  or ASOL.

51
Multiple Choice Items (VGLA Only)

• Multiple choice tests, quizzes and worksheets may
  not be used in their entirety
• Individual correct items should be identified and
  labeled.
• Other evidence should be used to support.
• Multiple choice items should not exceed 10 of
  the collection of evidence.

52
COE Scoring
Scoring
53
Holistic Scoring

• Scored at the SOL level the stem and the bullet
  level.
• Scoring teams look at all evidence submitted and
  assigns a score based on the evidence of student
  knowledge of the standard, including all bullets.

54
VGLA Scoring Rubric
55
VAAP Scoring Rubric
56
School-Based Oversight Teams
57
What is a School-Based Oversight Team?

• A group of 2-5 building level staff persons.
• Administrators, Special Educators, General
  Educators, Title One Staff, and/or ESL Educators

58
Why were School-Based Oversight Teams Established?

• To manage the large number of portfolio-based
  assessments.
• To ensure division-level consistency.
• To support the achievement of students with
  disabilities.
• To support school accreditation and AYP

59
Responsibilities of School-Based Oversight Teams
for VGLA

• To oversee and regularly monitor the development
  of collections of evidence for the VGLA and VAAP.
• To complete and submit monitoring documents to
  principals and Division Oversight Team.
• To ensure follow-through on recommendations for
  improvement.

60
Monitoring Process
61
Monitoring

• Periodic and systematic review of evidence-based
  assessments in the development process with
  opportunities for feedback and intervention.

62
What Monitoring is Not

• Checking with the teacher
• Flipping through an evidence based assessment
• Review without feedback

63
Why Monitor?

• To create better COEs.
• To increase the alignment between the students
  level of achievement and the evidence presented
  in the COE.
• To catch potential problems early.
• To prevent the mad dash before the COEs are due.

64
Why COEs Fail

• Missing evidence to support the SOL or ASOL
• Missing SOL or ASOL
• Use of unacceptable evidence
• Use of evidence not aligned with the SOL or ASOL
• Ungraded or incorrectly graded evidence
• Missing or incomplete SEI tags

65
Monitoring Decisions

• Who will monitor?
• When will monitoring occur?
• Who will receive feedback?
• How will feedback be given?
• What are your intervention options?

66
Components of Monitoring

• Trained Reviewers
• Feedback Loops
• Intervention Options

67
Trained Reviewers

• Persons with Content Knowledge and Testing
  Program Knowledge
• School Test Coordinator
• Assigned or Hired Staff
• Building Level Teams
• Central Office Level Teams

68
Feedback Loops

• Teachers preparing collections
• Principals
• VGLA or VAAP Leaders
• Trainers/Technical Assistance
• Providers

69
Intervention Options

• Individualized Consultation and Support
• Peer or small group sessions
• Attendance at Local Training
• Access to state and local SOL or ASOL and
  assessment resources

70
Monitoring Plan

• School-based Review Teams created and trained
• Teachers turn in evidence-based assessments at
  least every nine-weeks to the SBOT
• All evidence-based assessments reviewed by team
  according to division pacing chart
• Team report shared with principal and central
  office

71
Monitoring Plan
Select And Train School-Based Monitoring Teams
Monitoring Teams review COEs at least every 9
weeks.
Monitoring Teams report to the principal and
School Test Coordinator.
Interventions are scheduled as needed.
Feedback is provided to teachers.
72
Monitoring Tools

• VGLA Tools
• Content / grade specific tracking forms
• VAAP Tools
• VAAP Monitoring Sheet

73

Pre-scoring
74
Pre-scoring

• A detailed review of evidence-based assessments
  prior to submission
• Compared to scoring rules
• Prior to scoring
• Opportunity for feedback and intervention

75
What Pre-scoring is Not

• A mad dash to create a collection of evidence

76
Pre-scoring Decisions

• Who will pre-score?
• How will you train them?
• When will pre-scoring occur in relation to the
  submission date?
• Who will receive feedback?
• How will feedback be given?
• What are your intervention options?

77
Why Pre-score?

• To provide a last check to ensure that all the
  scoring rules have been followed
• To ensure that no last-minute detail has been
  overlooked
• To provide an opportunity to make changes before
  the COE is submitted

78
Rule 1- Evidence Must Show Individual Student
Achievement

• Does the evidence show any level of achievement
  according to the SOL/ ASOL being defended?
• Is the evidence on grade level?
• Are all of the SOL addressed (stems and bullets)?

79
Rule 2- Evidence Must Be Student-Generated

• Has the evidence been copied from the blackboard,
  textbook, or computer?
• If the evidence is from a group project, is the
  achievement of the particular student clearly
  identified?
• If the evidence is a worksheet, are there
  examples or directions that provide answers?

80
Rule 3 - Evidence Must Be Labeled with SEI Tag

• Does each piece of evidence have a completed SEI
  tag with
• Content area
• SOL
• Bullet (s)
• Demonstrated or Inferred checked

81
Rule 4- Evidence Must Address Accommodations,
Captions and Grading

• If accommodations have been used, are they
  documented in the evidence?
• Does each picture have a caption that clearly
  address the students level of achievement?
• Has the work been graded accurately with correct
  and incorrect answers clearly identified?

82
Media Considerations

• Do videotapes and audiotapes have transcripts?
• With an SEI Tag
• Are videotapes and audiotapes clearly labeled
  with SEI tags?
• On the case, not on the DVD
• Is equipment/software needed to review the media
  clearly identified on the case?

83
Organizational Considerations

• Is the evidence organized according to the
  scoring worksheet?
• Are all required state and local forms completed?

84
Pre-scoring Plan

• PHASE 1 - Building Level Review
• Submitting teacher review
• School-based oversight teams review
• Building Administrator review and sign-off

• PHASE 2 - Central Office Review
• Submitted from schools
• Reviewed by central office teams
• Return to schools to correct identified errors or
  omissions

85
Tools for Pre-scoring

• VGLA Implementation Manual
• Teacher Checklist for Collection of Evidence..26
• Administrator Checklist for Collections of
  Evidence..27
• VAAP Implementation Manual
• Teacher Checklist for Collection of Evidence..95
• Administrator Checklist for Collections of
  Evidence..97

86
Next Steps

• Obtain and review the SBOT schedule for your
  building
• Consult with the building administrator to
  determine the feedback process in your building
• Determine who will facilitate the SBOT meetings

87
Next Steps

• Review the VGLA and VAAP timelines
• Submit nine week reports by the due dates
• Be a TEAM player
• Review all VGLA portfolios according to RPS
  Pacing Chart.

88
Upcoming Professional Development
Content Staff Development October 21 22, 2008
89
Resources

• Rapid Response Teams
• RPS VGLA Website
• RPS VAAP Website
• VDOE Website www.pen.k12.va.us
• TTAC Online www.ttaconline.org
• VGLA Implementation Manual
• VAAP Implementation Manual

90
District Level Support

• Office of Exceptional Education and Student
  Services
• [phone]
• Director
• Elementary and Secondary Coordinators
• Instructional Specialists

91
District Level Support

• Department of Instruction
• [phone]
• Director
• Instructional Specialists

92
Questions Answers