Virginia Grade Level Alternative Assessment

1
Virginia Grade Level Alternative Assessment

• 2007 Scoring
• Train-the-Trainer
• Regional Workshops

2
Welcome Introduction of Presenters

• Sharon Siler, VDOE
• Lia Mason, VDOE
• Jeff Phenicie, VDOE
• Matthew Lumb, PEM
• John Osteen, PEM
• Terri McClure, PEM

3
Overview of Training Topics

• Critical Pre-Scoring Activities
• Planning and Conducting a Local Scoring Event
• Post-Scoring Activities and Recommendations
• Training Scorers to use Scoring Tools

4
Participants will be able to

• plan and implement pre-scoring activities to
  ensure proper technical and instructional quality
  of collections
• plan and conduct successful scoring events
• train local scorers to effectively use scoring
  tools to ensure scoring is consistent across the
  state and
• collect data from the scoring event that can be
  used to improve instruction and assessment
  practices.

5
Method of Addressing Questions

• Listen to the entire presentation.
• Use index cards to write down questions.
• Submit your questions at the end of the
  presentation.
• Q A periods will be used to provide answers.
• A Parking Lot may be used for some questions.

6
Pre-Scoring Activities

• Jeff Phenicie
• jeff.phenicie_at_doe.virginia.gov
• Virginia
• Department of Education

7
Purpose of Pre-Scoring Activities

• To ensure Technical Quality
• DIS labels
• Captions
• Required forms
• To ensure Instructional Quality
• Alignment of Evidence with Standards of Learning
• Focus on Individual Student Achievement
• Grading
• Appropriate use of Videotapes, Audiotapes
• Organization of Student Work

8
Technical Quality

• Review Collections of Evidence for
• DIS Labels
• Affidavits
• Local Required Forms

9
Data Identification System (DIS) Sticker
Content Area being submitted Standard of
Learning being defended Bullet of SOL being
defended (if applicable) Check one or both
VGLA/VSEP DIS
VGLA/VSEP DIS
Example
?
?
10
Affidavit

• Work samples submitted must be
• solely that of the student.
• In the presence of a teacher and/or
  paraprofessional.
• Affidavit states that I did not
• fabricate, alter, or modify student work samples,
  products, or data
• describe student behaviors that provide a
  negative image of the student or
• provide any accommodation/assistive device that
  is not a regular part of the students daily
  instruction.

11
Instructional Quality

• Review Collections of Evidence for
• Alignment to Standards
• Graded Work
• Appropriate Videotapes and Audiotapes
• Photographs with Captions
• Organization of Evidence

12
Instructional Quality

• A collection may consist of a combination of the
  following types of evidence
• Work Samples - tests, quizzes, worksheets, and
  projects
• Audiotape/Videotape oral presentation, role
  play
• Anecdotal Record - observation using a protocol
• Interview
• Charts/Graphs
• However, ONLY work completed by the student
  under the direct supervision of a teacher or
  paraprofessional is allowable.

13
Instructional Quality

• Evidence must demonstrate knowledge and/or skills
  in the SOL addressed.
• Evidence should be submitted for all of the
  assessed SOLs based on the test blueprints.
• Evidence may prove mastery of more than one SOL.
• All evidence must be solely the students work
  completed in the presence of a teacher or
  authorized school personnel.

14
Evidence submitted must include everything in the
stem as well as all of the bullets.

• USI.4 The student will demonstrate knowledge
  of European exploration in North America and West
  Africa by
• a) describing the motivations, obstacles, and
  accomplishments of the Spanish, French,
  Portuguese, and English explorations
• b) describing cultural interactions between
  Europeans and American Indians (First Americans)
  that led to cooperation and conflict
• c) identifying the location and describing the
  characteristics of West African societies
  (Ghana, Mali, and Songhai) and their interactions
  with traders.

15
Instructional Quality

• Student evidence must be graded
• Letter Grades
• Teacher Comments
• Check Marks
• Numerical Grades

16
Instructional Quality

• Photographs
• Captions describing student performance
• Videotapes and Audiotapes
• Short video and audio clips not entire lessons
• Media is labeled correctly
• Scripts completed

17
VGLA COE Organizer
18
Recommendations

• Recommended for Pre-Scoring Review Collections of
  Evidence
• Special Education Teacher Review
• Content Teacher Review
• Building Administrator Review

19
VGLA Collection of Evidence Coversheet
20
SOL Instructional Tracking Form
21
Pre-Scoring Division Level

• Develop a team approach
• Central office staff
• Specialists
• Principals
• Assistant principals
• Department chairpersons
• Expert teachers

22
Pre-Scoring Division Level

• Develop a team approach
• Assign reviews by criteria schools, grades,
  etc.
• Address both technical and instructional
  quality.

23
Conducting A Scoring Event

• Sharon E. Siler
• sharon.siler_at_doe.virginia.gov
• Virginia
• Department of Education

24
VGLA Due Dates

• VGLA scores may
• be entered in 2 phases
• 50 by June 8, 2007
• 50 by June 29,2007
• Testing Memo 593

25
Determine Important Local VGLA Dates

• Date of Scorer Training
• Date of Final Master Lists from Schools
• Date of Pre-Scoring Review
• Date of Scoring and Score Entry
• Date of Audit Shipment

26
Determine Important Local Submission Procedures

• Develop and communicate expectations across all
  schools
• Order of materials in the Collection of Evidence
• Boxing of Collections of Evidence (i.e., by
  content area, by teacher, etc.)
• Location of submission
• Date/Time of submission

27
Coordination ResponsibilitiesPrior to Scoring

• Plan and Implementation of Pre-Scoring Activities
• Contact with Students in Private School
  Placements
• Develop VGLA Master Participant Lists
• Select and Train Scorers
• Select and Train Data Entry Staff
• Select Facility for Scoring Event and secure
  storage of Collections of Evidence
• Gather Scoring Materials

28
Scorers

• Manpower Issues
• Who will score? (Teachers, Retirees, Central
  Office Staff)
• How long will it take? (45 minutes per
  collection 8 collections per scorer/team per
  day)
• Note Individuals who have provided technical
  assistance or have conducted a review of the
  collections of evidence cannot act as scorers.

29
Scoring Facility

• Space Requirements
• Scoring Area
• Distribution Area
• Coordinator Area
• Refreshment Area
• Area for Video and Audio
• Data Entry Area with Computers and Printers
• Secure Area for Collections of Evidence

30
COEs
Coordination Table
C O E s
Scorer 1
Scorer 2
Scorer 3
Scorer 4
Scorer 7
Scorer 8
Scorer 5
Scorer 6
Scorer 12
Scorer 9
Scorer 10
Scorer 11
Food Table
Food Table
To Data Entry Video/Audio Review Re-Boxing Area
31
Scoring Materials

• Rubric
• Rubric Addendum
• Scoring Rules
• SOL Curriculum Framework (Reference Only)
• VGLA Procedural Manual
• VGLA Worksheets
• Office Supplies

32
Coordination ResponsibilitiesDuring Scoring

• Review Scoring Protocols
• Provide Scoring Materials
• Maintain Work Flow
• Establish Goals
• Report Progress
• Address Concerns
• Check-off w/ Participant Lists
• Oversee Online Data Entry
• Pull Collections for Audit

33
VGLA Scoring Event
VGLA Scoring
34
Check-In Procedures

• Match Collections of Evidence to Master
• Participant List
• Assign label code to each Collection of
• Evidence
• Re-box Collections of Evidence by Content
• Area

35
Check Out Procedures

• Check-off scored Collections of Evidence
• against Participant Lists
• Re-box Collections of Evidence for retrieval
• for audits

36
Sample Scoring Schedule

• 700 - 800 Room Set-Up and Distribution
  of Materials
• 800 - 830 Scorer Sign-In and
  Continental Breakfast
• 830 - 845 Announcements, Review of
  Protocol, Daily Goal Setting
• 845 - 1200 SCORING and DATA ENTRY and
  CHECK-OUT
• 1200 -100 Lunch
• 100 - 330 SCORING and DATA ENTRY AND
  CHECK-OUT

37
Common Scoring Issues

• School Fails to Submit Collection of Evidence for
  Participating Student
• Incomplete Collections of Evidence
• Ungraded Work
• Incorrect DIS labels
• Wrong Students Work in Collection of Evidence

38
Scoring Pitfalls

• Poor match of facility to Scoring Event
• Poor organization of work flow
• Insufficient manpower
• Insufficient training
• Scorer/Team drift
• Poor implementation of pre-scoring
• activities

39
Coordination ResponsibilitiesPost-Scoring

• Shipping Collections of Evidence for Audit
• Storing Collections of Evidence Securely
• Post-Scoring Activities to Improve Instruction

40
VGLA Audit

• VDOE will audit 10-15 of VGLA COEs.
• Audit lists will be sent to school divisions
  weekly.
• Shipping materials to school divisions by
• June 15, 2007
• Ship Date June 25, 2007
• Audit Dates July 2007
• Dates are Tentative

41
Post-Scoring Activities

• Ask Scorers to Identify
• Exemplary organizational practices
• Exemplary instructional practices
• General tips for teachers
• Teachers with outstanding Collections of
• Evidence
• Specific areas needing improvement
• Scoring Event observations

42
Retention of Collections

• Collections of Evidence are considered part of
  students educational records.
• Score reports are maintained in education record.
• Collections of Evidence may be maintained in a
  separate secure location however, location must
  be indicated.
• Teachers and parents must be given access under
  supervision.