Comprehensive English Language Learning Assessment CELLA

1
Comprehensive English Language Learning
Assessment(CELLA)
TRAIN THE TRAINER TRAINING SESSIONS -- January
2009
2
Welcome Introductions

• Lori Rodriguez, Chief, Bureau of Academic
  Achievement through Language Acquisition
• Wendy Nucci, ETS CELLA Program Manager
• Paul Rybinski, ETS Assessment Development Lead,
  English Language Learning
• Zulma Torres, ETS CELLA Program Manager
• Lorenzo Murolo, Inbound Operations Manager

3
Morning Agenda

• Welcome Introductions
• Review Content of Tests
• Who Should be Tested
• Functional Level Testing
• What is it?
• How is it administered?
• Administering Level A
• Administering Levels B-D

4
Afternoon Agenda

• Whats New for 2009
• 2009 Comprehensive Schedule
• Training Materials
• 2009 Supplemental Order Process
• Responsibilities
• Test Administrator
• School Coordinator
• District Coordinator
• 2009 DFAs and Scoring Guides
• Answer Docs and Pre-ID Labels
• Question and Answers Session

5
What is CELLA?

• CELLA provides
• evidence of program accountability in accordance
  to Title III of NCLB
• data on students over time
• information about the language proficiency of
  students across the State
• diagnostic information about students strength
  and weakness in English
• fair, reliable, and valid information about ELLs
  in grades K through 12

6
How was CELLA developed?

• CELLA was funded by a grant from the U.S.
  Department of Education.
• A collaboration with AccountabilityWorks and a
  consortium of five states Florida, Maryland,
  Michigan, Pennsylvania, Tennessee
• As the grantee, AccountabilityWorks managed the
  work of the consortium
• ETS was awarded the subcontract to develop the
  test, ancillary materials, conduct the field test
  and develop the scoring programs
• The State provided substantial input and the
  majority of field tested students

7
CELLATest Content
8
Overview of CELLA

• Test Sections
• Listening
• Speaking
• Reading
• Writing

• Test Levels
• Level A (Grades K-2)
• Level B (Grades 3-5)
• Level C (Grades 6-8)
• Level D (Grades 9-12)

Test Format Listening All Multiple Choice
Speaking All Constructed Response Reading
All Multiple Choice Writing Multiple Choice
Constructed Response
9
CELLA Item Types

• Reading Vocabulary
• Synonym
• Antonym
• Idiom
• Root Affix

• Listening
• Listen Match
• Picture Description
• Short Talks
• Extended Listening

• Writing Multiple Choice
• Grammar, structure written expression
• Paragraph choices
• Recognizing errors

• Speaking
• Oral Vocabulary
• Speech Functions
• Personal Opinion
• Story Retelling
• Graph Interpretation

• Reading Comprehension
• Main Idea
• Detail
• Inference/Prediction
• Reference
• Rhetorical Elements
• Vocabulary in Context

• Constructed Response
• Writing sentences
• Writing paragraphs

10
Test Document Configuration(On Grade Level ALL
Sections)
Level A Grade K - 2
Level B Grades 3 - 5
Level C Grades 6 - 8
Level D Grades 9 - 12
11
Testing Decision Tree (Whom Do You Test?)
12
Which Students Should be Tested?

• All students in grades K-12 classified as ELLs,
  with codes of LY or LP as of May 31, 2008 and
  in membership in the district at the time of
  testing
• Former ELLs (code of LF) that were exited from
  the ESOL program on or after May 31, 2008 and
  before April 14, 2009
• Any student that enrolls after January 16, 2009
  and before April 3, 2009, and is classified as an
  ELL must be assessed

13
Who is NOT Tested?

• Students enrolling on or after April 6, 2009,
  with affirmative responses on the Home Language
  Survey are NOT included in the 2009
  administration of CELLA

14
Functional Level Testing
15
What is Functional Level Testing?

• Testing students language ability according to
  what they can actually do and NOT according to
  what is expected at their grade level
• Used only for literacy skills (Reading
  Writing), which must be learned, and NOT for oral
  skills (Listening Speaking), which are more
  naturally acquired
• Students may take the Reading and Writing
  sections at a level lower than grade level

16
Why administer Functional Level Testing?

• There is a often wide disparity in skill levels
  of ELLs (especially in the higher grades)
• To get reliable and accurate test results for
  students at all proficiency levels within each
  grade-span would require more test items
• Test administration time is kept short by giving
  students only those items most appropriate for
  their skill level
• Without functional level testing, measures of
  growth are inaccurate for very low performing
  students

17
Benefits of Functional Level Testing to the
Student

• Frustration during testing is minimized.
• Abilities are more accurately reflected by the
  scores.
• Interventions are more effectively and
  appropriately.
• Results can be compared across districts more
  accurately.

18
CELLA Results

• Administering the CELLA Reading Writing
  sections according to students' functional level
  does NOT result in an "off-grade" version of the
  assessment
• Off-grade" testing typically indicates that
  student results are scored and reported against a
  different, lower standard. This is NOT the case
  with CELLA
• Every student in a given grade span (e.g., K-2,
  3-5, 6-8, 9-12), regardless of which test level
  is administered, will be reported as "English
  Proficient" against the same standard (or
  "cut-score") on the assessment's vertical scale

19
How To Implement CELLA Functional Level Testing

• Teachers and CELLA Test Administrators should use
  the following tools to determine the correct
  Reading Writing level to administer to students
  classified as ELLs at the time of testing
• 2009 CELLA Testing Decision Flowchart, AND
• 2009 CELLA Functional Level Testing
  Decision- Making Rubric,
• -OR-
• CELLA Online Locator Test

20
Which students should NOT be tested with
Functional Level Testing?

• Former ELLs (LFs) who exited the ESOL program on
  or after May 31, 2008 must be tested in Reading
  Writing based on their grade level

21
2009 CELLA Functional Level Testing
Decision-Making Rubric

• Developed based on the CELLA test items and the
  content demands in Reading Writing required of
  students to reliably determine their level of
  English proficiency in these two domains
• Teachers use a check mark next to the level of
  the students attainment in each of the four
  English skill areas
• Students with 3 or 4 check marks in one column
  should take the test indicated in that column

22
2009 CELLA FUNCTIONAL LEVEL TESTING DECISION-MAKIN
G RUBRIC
23
How to Administer Functional Level Testing
24

• Test Material for Functional
• Level Testing

25
FUNCTIONAL LEVEL TESTING Example 1

• Example A student in grade 9 will take Level D,
  Listening and Speaking

The student is eligible to take the Level C,
Level B, or Level A Reading Writing test
OR
OR
ALL student responses are recorded onto the Level
B, C, D Answer Document
ALL Student responses are recorded on the
Functional Level A Test Book
26
FUNCTIONAL LEVEL TESTING Example 2

• Example A student in grade 5 will take Level B
  Listening Speaking and is eligible to take the
  Level A Reading Writing test

AND
ALL student responses are recorded onto the
Functional Level A Test Book
27
Level ATraining
28
Level A Sections
29
Administering Level A

• The test should be individually administered to
  students in kindergarten.
• Grades 1 and 2 may take Listening, Reading and
  Writing in small groups.
• The One-on-One section must be individually
  administered
• Grade 2 will take additional extension items in
  Reading Writing

30
Administering Level A Core and Extension

• The Core Section should be taken by all students
  in Grades K-2
• Listening Speaking sections are identical for
  all students in grades K-2
• The Extension contains more challenging Reading
  Writing items and is administered only to
  students in Grade 2
• Students in grades K and 1 receive fewer Reading
  Writing items than students in Grade 2

31
Administering Level A

• STOPPING POINTS are available for individually
  administered sections of CELLA
• Test administrators should stop the
  administration of that section only IF the
  student cannot respond to five questions in a row
  AND it is clear that the questions are above the
  students proficiency level

32
Level A Student Responses

• Please demonstrate how students should mark their
  responses in their test book

Correct Incorrect
33
Test Administrators One-on-One Scoring Section
Page 38 of the Level A Student Test Book
34
CELLA Speaking/One-on-One Administration
Knowledge Skills Needed to be an Effective
Evaluator of Language Proficiency

• Proficiency in English
• Thorough understanding of how rubrics work
• Knowledge of and experience with the specific
  skills being measured
• Practice with rubrics
• Objectivity

35
Administering Level A
One-On-One SectionSpeech Functions

• Measures a students oral response to a specific
  prompt
• Criteria include
• Appropriateness of information
• Grammatical accuracy

36
Administering Level A
One-On-One Section Speech Functions Rubric
Review Activity

• There are a total of 22 training tracks for
  Speech Functions
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 49
• Training Items are on page 56
• Answer Key is on page 63
• Activity
• Listen to training CD Tracks 1-2
• Discuss why the response scored a 2, referring to
  the rubric
• Listen to CD Tracks 3-4
• Discuss why the response scored a 1 and 0
• Make sure you LISTEN to student responses, rather
  than read the transcript

37
Administering Level A

• One-On-One SectionPersonal Opinion
• Measures students ability to orally state and
  defend an opinion
• Criteria include
• Clarity of response
• Adequate support
• Good control of grammar adequate vocabulary

38
Administering Level A
One-On-One Section Personal Opinion Rubric
Review Activity

• There are a total of 9 training tracks for
  Personal Opinion
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 50
• Training Items are on page 57
• Answer Key is on page 64
• Activity
• Listen to training CD Tracks 23-24
• Discuss why the response scored a 2, referring to
  the rubric
• Listen to CD Track 27
• Discuss why the response scored a 1
• Make sure you LISTEN to student responses, rather
  than read the transcript

39
Administering Level A

• One-On-One SectionStory Retelling
• Measures a students ability to hear a story
  (while looking at sequential picture cues) and to
  then retell it with detail
• Criteria include
• Comprehensive response
• Vocabulary
• Grammar
• Fluency

40
Administering Level A
One-On-One Section Story Retelling Rubric Review
Activity

• There are a total of 16 training tracks for Story
  Retelling
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 51
• Training Items are on page 58
• Answer Key is on page 64
• Activity
• Listen to training CD Tracks 32-33
• Discuss why the response scored a 4, referring to
  the rubric
• Listen to CD Track 35
• Discuss why the response scored a 2
• Make sure you LISTEN to student responses, rather
  than read the transcript

41
Administering Level A

• One-On-One SectionReading Aloud for Fluency
• Measures reading fluency
• Criteria include
• Rate
• Accuracy

42
Administering Level A

• One-On-One SectionReading Aloud for Fluency
• Administration - Types of Errors
• Substitution
• e.g., bird instead of bear
• Mispronunciation
• e.g., fell instead of fall
• Words pronounced with an accent are counted as
  correct if they cannot be confused with other
  English words
• Omissions
• i.e., skipped words
• If the student stops or struggles with a word for
  3 seconds, you may tell the student the word and
  count it as an error

43
Administering Level A
One-On-One SectionReading Aloud for Fluency

• There are a total of 16 training tracks for
  Reading Aloud for Fluency
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 53
• Training Items are on page 62
• Answer Key is on page 66
• Activity
• Listen to training CD Tracks 48-49
• Discuss why the response scored a 4, referring to
  the rubric
• Listen to CD Track 53
• Discuss why the response scored a 0
• Make sure you LISTEN to student responses, rather
  than read the transcript
• Note
• If the student makes repeated errors on the same
  word, count the error only once
• Repetitions and self-corrections are not counted
  as errors

44
Level B-D Listening/Speakingfor Grades 3-12
45
Administering Levels B-D Listening Speaking

• Grades
• Level B (3-5)
• Level C (6-8)
• Level D (9-12)
• Test design is identical across levels for
  Listening Speaking
• Listening all multiple-choice Speaking all
  constructed-response
• Listening is group administered testing time is
  about 25 minutes
• Level B is the only exception with one
  additional item

46
CELLA B-D Listening Speaking Sections

• Listening
• Listen and Match
• Picture Description
• Short Talks
• Extended Listening
• Materials Needed
• Levels B, C D Answer Sheet
• Directions for Administration Scoring
  Guides
• Listening Speaking test book
• Listening CD

• Speaking
• Oral Vocabulary
• Speech Functions
• Personal Opinion
• Story Retelling
• Graph Interpretation
• Materials Needed
• Levels B, C D Answer Sheet
• Directions for Administration Scoring Guides
• Listening Speaking Test Book
• Training CD for Speaking

47
Test Level Selection Area
48
Level C Training
49
Administering Level C
Speech Functions

• Measures a students oral response to a specific
  prompt
• Criteria include
• Appropriateness of information
• Grammatical accuracy

50
Administering Level C
Speech Functions Rubric Review Activity

• There are a total of 45 training tracks for
  Speech Functions
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 186
• Training Items are on page 194
• Answer Key is on page 204
• Activity
• Listen to training CD Tracks 1-2
• Discuss why the response scored a 2, referring to
  the rubric
• Listen to CD Tracks 3-4
• Discuss why the response scored a 1 and 0
• Make sure you LISTEN to student responses, rather
  than read the transcript

51
Administering Level C

• Personal Opinion
• Measures students ability to orally state and
  defend an opinion
• Criteria include
• Clarity of response
• Adequate support
• Good control of grammar adequate vocabulary

52
Administering Level C
Personal Opinion Rubric Review Activity

• There are a total of 13 training tracks for
  Personal Opinion
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 188
• Training Items are on page 197
• Answer Key is on page 206
• Activity
• Listen to training CD Tracks 46-47
• Discuss why the response scored a 2, referring to
  the rubric
• Listen to CD Track 48
• Discuss why the response scored a 1
• Make sure you LISTEN to student responses, rather
  than read the transcript

53
Administering Level C

• Story Retelling
• Measures a students ability to hear a story
  (while looking at sequential picture cues) and to
  then retell it with detail
• Criteria include
• Comprehensive response
• Vocabulary
• Grammar
• Fluency

54
Administering Level C
Story Retelling Rubric Review Activity

• There are a total of 16 training tracks for Story
  Retelling
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 198
• Training Items are on page 200
• Answer Key is on page 206
• Activity
• Listen to training CD Tracks 59-60
• Discuss why the response scored a 4, referring to
  the rubric
• Listen to CD Track 62
• Discuss why the response scored a 2
• Make sure you LISTEN to student responses, rather
  than read the transcript

55
Administering Level C

• Graph Interpretation
• Measures students ability to orally summarize and
  interpret a graph
• Criteria include
• Summary response
• Comparison response
• Vocabulary
• Grammar
• Fluency

56
Administering Level C
Graph Interpretation

• There are a total of 12 training tracks for Graph
  Interpretation
• Refer to the Directions for Administration
  Scoring Guides
• Rubric is on page 201
• Training Items are on page 203
• Answer Key is on page 208
• Activity
• Listen to training CD Tracks 75-76
• Discuss why the response scored a 4, referring to
  the rubric
• Listen to CD Track 80
• Discuss why the response scored a 0
• Make sure you LISTEN to student responses, rather
  than read the transcript

57
New for 2009
58
Addition of Binder Markings on ALL2009 Test Books
Binder Markings (Numbers will be located on the
top of the binding)
59
Security Numberson Test Books Clearly Identified
Security Numbers
60
Going Green

• Were reducing the number of printed materials
• The Informational Flyer is available exclusively
  online at http//www.fldoe.org/aala/cELLa.asp
• The Interpretive Guide will be printed in English
  and Spanish only. All languages will be posted
  on the CELLA website
• School, District and State Score Reports will be
  On CD No paper copies
• Student Reports Will Continue to be provided in
  paper version
• Student Score Report has been revised to include
  a cut score table.

61
2009 Schedule
62
2009 CELLA ComprehensiveSchedule

• Event
• 2009 Train the Trainer for NEW CELLA
  Administrators
• 2009 CELLA Update Webinars for the experienced
  Administrator
• Begin Online Locator Test Administration
  www.awschooltest.com

• Date
• January 13, 2009
• January 15, 2009
• Tuesday January 20
• Wednesday January 21
• February 16, 2009

63
2009 CELLA ComprehensiveSchedule continued

• Event
• Receipt of 2009 Training Materials
• Districts Begin Training
• Receipt of 2009 CELLA Testing Materials
• 2009 CELLA Test Administration Window

• Date
• By January 31, 2009
• February April 2009
• March 30 April 10, 2009
• April 14 May 15, 2009

64
Test Security
65
CELLA Test Security

• All CELLA testing materials are to be secured
• Test administrators must be able to account for
  ALL materials assigned to them
• Test administrators will be asked to sign a
  security form at each school site
• Test administrators will be asked to sign
  materials in and out each day

66
Districts Can Keep Non-secure Materials

• Non-secure materials include
• Test Administrators Manual
• Speaking Training CDs (ONLY)
• Unused paper bands
• Unused document count forms
• Unused return labels
• Unused non-secure large print materials
• Note ALL Secure Material Must Be Returned to
  ETS.

67
Training Materials
68
2009 Training Materials
Each district will receive 2 complete sets of the
materials below and their orders for additional
kits and individual items

• 2009 Test Administration Manual
• 2009 Directions for Administration Scoring
  Guides
• Set of 6 CDs
• 4 Speaking Training CDs, one per level
• 2 Listening CDs (AB and CD are combined)

• Test Book Kit
• Level A Student Test Book
• Level A Functional Level Test Book
• Level A One-on-One Prompt Book
• Level B, C, D Listening Speaking Test Books
• Level B, C, D Reading Writing Test Books
• Levels B, C, D Answer Sheet

2009 Training Manual and CD handouts will be
available online
69
Packaging, Delivery, Return of Training
Materials

• Training materials
• Will be sent to districts only
• They will NOT be packaged by school
• Delivered by January 31, 2009
• Training Materials must be returned with the test
  administration (brown boxes)
• Reminders
• Return labels will be included in the test
  administration shipments and NOT in the training
  materials
• Training test booklets can be used if needed in
  the administration as overage if used and free of
  any marks or notes
• NO SUPPLEMENTAL ORDERS will be processed for
  training materials.

70
2009 Supplemental Order Process
71
2009 Supplemental Order Process

• Only District Coordinators can place supplemental
  orders
• Orders can only be placed after the initial
  shipment is received and checked
• Orders will be placed using the On-line Form
  Once completed, print and press submit
• Orders will be shipped within 48 hours of
  receipt. Orders received after 2pm are processed
  on the next business day

72
Packaging, Delivery Return of Test Materials
73
Materials Available for Test Administration

• All test books packaged in 10s
• 2009 Test Administration Manual
• Directions for Administration Scoring Guides
• Levels B, C D Answer Sheets packaged in 10s
• One-on-One Prompt Book (for Level A only)
• Listening CDs
• Pre-ID Labels
• Return Material Kit
• White boxes pre-addressed with a return shipping
  label to ETS

74
Packaging Delivery of Test Materials

• Orders are produced based on the Pre-ID file
  received from the DOE, supplied by the districts
  in Survey 7
• Orders are packed by school (and General Service
  Route if applicable) and shipped to the district
  for distribution
• Delivery of materials will be from March 30th
  through April 10th
• A packing list of all schools will be provided to
  the district
• A comprehensive packing list of all material sent
  to a school will be in Box 1 of the school boxes

75
Receipt of Test Materials

• Receipt of Material
• District box(es) should be opened and checked
  against the box packing lists
• Each school should count and verify that all
  boxes are received using the box count printed on
  the outside of each box
• Each school should open the box(es) upon receipt
  and verify all items listed on the shipping
  notice are enclosed in the shipment

76
Accommodated Format MaterialsMaterial will
arrive in an oversized brown box

• Return all Large-Print and Braille Materials to
  ETS
• All completed answer documents should be sorted
  by grade
• Complete a header sheet for each grade
• Reuse the oversized brown boxes
• Apply the pre-addressed shipping labels that is
  provided
• When packing the box, place the unused material
  on the bottom and any used material on the top

• Large-Print Version
• Large-Print Test Booklet
• Large-Print Answer Document for recording
  responses
• Directions for Administration Scoring Guides
• Braille Version
• Braille Test Booklet
• Standard Answer Document to record test takers
  responses
• A Standard Test Booklet that matches the Braille
  level being administered to assist the test
  administrator
• Directions for Administration Scoring Guides

77
Damaged Answer Documents

• If an answer document is damaged either from the
  initial packaging of the documents or through
  handling during test administration

• For example torn, folded, marked by a pen, etc.
• The School Coordinator needs to transcribe all
  the information from the damaged answer document
  onto a new document
• The damaged answer document must be returned to
  ETS and returned in the brown boxes with the Not
  To Be Scored materials
• ALL DNS must be bubbled in on the damaged
  document (Item 26)
• The new transcribed answer document should be
  bundled with the To Be Scored for that grade

78
Damaged Answer Documents (continued)

• If due to a students illness -- the School
  Coordinator should attempt to transcribe the
  students responses onto a new answer document

• If unable to transcribe, make-up date within the
  testing window should be arranged
• District Coordinator must call ETS at
  1-866-881-2802 and provide the security number
• The original document should be destroyed in a
  secure manner

79
CELLATest Material Returns

• Lorenzo Murolo
• Inbound Operations Manager

80
Functional Areas

• Test Administrator Responsibilities
• Required Administration Information
• Security Log
• Test Material Sorting
• School Coordinator Responsibilities
• Prepare Materials for Return
• Large Print and/or Braille Materials
• Packing District Coordinator ONLY Boxes
• District Coordinator Responsibilities
• Materials Return List
• District Coordinator ONLY Boxes
• Return Material to Contractor

81
Test Administrator Responsibilities

• What/How to prepare for the School Coordinator

82
Test Administrator Responsibilities

• Required Administration Information
• All fields must be accurately filled out
• Ensure all students have a Test Code Disposition
• Notify the School Coordinator if secure material
  is missing

Test Codes
83
Test Administrator Responsibilities

• Security Log
• Complete this log when entering and exiting the
  room

84
Test Administrator Responsibilities

• Test Material Sorting

• To Be Scored
• Verify the DNS has NOT been gridded
• Group your TO BE SCORED material by GRADE
• DO NOT insert Answer Sheets INSIDE of the Test
  Books
• Notate the TO BE SCORED answer document count BY
  GRADE for the School Coordinator

85
Test Administrator Responsibilities

• Test Material Sorting

• Not To Be Scored
• Unused Scannable Answer Sheets
• Used DNS Scannable Answer Sheets
• NOTE Any Used Scannable Answer Sheets mixed with
  this material WILL NOT BE SCORED
• All Used and Unused Test Books
• NOTE Used Grade K-2 SCANNABLE Test Books should
  NOT be included with the Not To Be Scored, UNLESS
  the DNS is notated
• Secure Listening CDs
• Secure 2009 Directions for Administration
  Scoring Guides
• Test Administration Manual
• Security Log
• Required Administration Information

86
School Coordinator Responsibilities

• What/How to prepare for the District Coordinator

87
School Coordinator Responsibilities
TO BE SCORED

• ?CELLA Document Count Form for To Be Scored
  Answer Sheets
• ?CELLA Answer Document Return Summary Form
• ?Paper Bands
• ?Packaging

88
School Coordinator Responsibilities
Prepare Materials for Return To Be Scored

• Fill out the Document Count Form
• NOTE You must create one PER GRADE and place it
  ON TOP OF EACH GRADE STACK. Do NOT include Blank
  Answer Sheet Counts

Fields to Populate
89
School Coordinator Responsibilities
Prepare Materials for Return To Be Scored

• Fill out the Answer Document Return Form
• NOTE Grid ALL GRADES TESTED (Box 4) and the
  TOTAL number of TO BE SCORED Document Counts (Box
  5).
• The document should be placed ON TOP of the
  first Grade Stack
• Photocopy the Answer Document Return Summary for
  your records

Fields to Populate
Some schools may received blank documents.
These fields must be complete
90
School Coordinator Responsibilities
Prepare Materials for Return To Be Scored

• Paper Band EACH grade SEPARATELY
• NOTE The Document Count Summary (RED) must be
  UNDER the paper band on top of the first grade
  level

GRADE 3
G R A D E 4
G R A D E 5
G R A D E 6
91
School Coordinator Responsibilities
Prepare Materials for Return To Be Scored

• ?Package all TO BE SCORED material in the WHITE
  boxes
• ?If you have MORE THAN ONE To Be Scored box,
  place the stack containing the Answer Sheet
  Return Summary in the FIRST WHITE BOX
• ?Securely tape your box on the top and bottom
• ?The Orange TO BE SCORED return label will
• already be pre-applied to the WHITE BOX
• ?Fill out ALL steps on the return label
• Box counts should be labeled 1 of X, 2 of X
• DO NOT count your BROWN boxes here
• DO NOT PHOTOCOPY your return label
• All boxes MUST have a return label
• ?DO NOT place a NOT TO BE SCORED
• DOCUMENT COUNT FORM (Green) in this box
• ?Return all boxes to the District Coordinator

92
School Coordinator Responsibilities
NOT TO BE SCORED

• ?CELLA Not To Be Scored Document Count Form
• ?Packaging

93
School Coordinator Responsibilities
Prepare Materials for Return Not To Be Scored

• ?Fill out the CELLA Not To Be Scored Document
  Count Form
• Unused Test Books
• Used Test Books (Excluding the USED Level A
  Scannable Test Books)
• Defective Documents
• ALL DNS gridded documents
• Secure Listening CDs
• Secure 2009 Directions for Administration
  Scoring Guide
• ?Ensure NO ANSWER SHEETS
• were placed INSIDE of the
• TEST BOOKS

Fields to Populate
94
School Coordinator Responsibilities
Prepare Materials for Return Not To Be Scored

• ?Package all NOT TO BE SCORED material in the
  BROWN boxes
• Unused Test Books
• Used Test Books (Excluding the USED Level A
  Scannable Test Books)
• Defective Documents
• ALL DNS gridded documents
• Secure Listening CDs
• Secure 2009 Directions for Administration
• Scoring Guide
• DO NOT place any TO BE SCORED MATERIAL
• in the BROWN boxes
• ?Securely tape your box on the top and bottom
• ?Affix the Green return label on the
• BROWN box
• ?Fill out ALL steps on the return label
• Box counts should be labeled 1 of X, 2 of X
• DO NOT count your WHITE boxes here

95
School Coordinator Responsibilities
Large Print and/or Braille Materials

• ?CELLA Document Count Form for To Be Scored
  Answer Sheets
• ?CELLA Answer Document Return Summary Form
• ?Packaging

96
School Coordinator Responsibilities
Prepare Materials for Return Large Print and/or
Braille Materials

• ?Fill out the Document Count Form
• NOTE You must create one PER GRADE and place it
  ON TOP OF EACH GRADE STACK. Do NOT include Blank
  Answer Sheet Counts

Fields to Populate
97
School Coordinator Responsibilities
Prepare Materials for Return Large Print and/or
Braille Materials

• ?Fill out the Answer Document Return Form
• NOTE Grid ALL GRADES TESTED (Box 4) and the
  TOTAL number of TO BE SCORED Document Counts (Box
  5) for Large Print and Braille ONLY.
• The document should be placed ON TOP of the
  first Grade Stack
• Photocopy the Answer Document Return Summary for
  your records

Fields to Populate
Some schools may received blank documents.
These fields must be complete
98
School Coordinator Responsibilities
Prepare Materials for Return Large Print and/or
Braille Materials

• ?Package all TO BE SCORED and NOT TO BE SCORED
  material in the ORIGINAL BROWN boxes it was
  shipped in
• ?Ensure the DO NOT SCORE HEADER is CORRECTLY
  placed on the DNS Stack, and placed on the bottom
  of the box
• ?DO NOT RETURN the Braille and Large Print
  material with the
• Regular Shipments
• ?Securely tape your box on the top and bottom
• ?Affix the White return label on the
• ORIGINAL BROWN box it was shipped in
• ?Fill out ALL step on the return label
• Box counts should be labeled 1 of X, 2 of X
• DO NOT count your WHITE or REGULAR BROWN
• DO NOT SCORE boxes here
• DO NOT PHOTOCOPY your return label
• All boxes MUST have a return label

99
School Coordinator Responsibilities
?Packing District Coordinator ONLY Boxes

• ?2009 CELLA District Only Check List
• ?Packaging

100
School Coordinator Responsibilities
District Coordinator ONLY Boxes

• ?2009 CELLA District Only Check List
• 2009 Required Administration Information
• Security Log(s)
• Seating Chart (IF Required by your district
• Packing Slips (IF Required by your districts
• 2009 Test Administration Manuals
• Trainings Speaking CDs
• Unused Level B, C, D Answer Sheets
• Remaining
• Unused TO BE SCORED Document Count Form(s)
• Unused NOT TO BE SCORED Document Count Form(s)

101
School Coordinator Responsibilities
District Coordinator ONLY Boxes

• Packaging
• ?Securely tape your box on the top and bottom
• ?Write DISTRICT COORDINATOR ONLY on the side of
  EACH box
• ?DO NOT place any return labels on these boxes
• ?Return all boxes to the District Coordinator

102
District Coordinator Responsibilities

• What/How to prepare for returning material for
  scoring

103
District Coordinator Responsibilities
Prepare Boxes for Return

• ?Materials Return List
• ?District Coordinator ONLY Boxes
• ?Return Material to Contractor

104
District Coordinator Responsibilities
Materials Return List

• ?Verify the school coordinator correctly labeled
  and numbered the boxes from the school
• No photocopied return labels
• Label counts reflect the CORRECT BOX TYPE

Box Types
105
District Coordinator Responsibilities
Materials Return List

• ?Log the quantity of box types you are returning
  on the Materials Returns List
• ?Make sufficient copies of the Materials Return
  List

106
District Coordinator Responsibilities
District Coordinator ONLY Boxes

• ?Open ALL boxes, review the Records of Required
  Administration, and notify the FDOE if ANY secure
  CELLA materials are missing
• ?Verify the Security Logs are complete
• ?DO NOT DESTROY any non-secure material, UNTIL
  SCORES HAVE BEEN REPORTED
• ?Ensure NO SCORABLE or SECURE material are in the
  boxes

107
District Coordinator Responsibilities
Return Material to Contractor

• ?Contact K2 Logistics to schedule a pickup, and
  provide the pick up location and box counts
  (888-886-0780)
• ?Designate a person to be AVAILABLE at the pickup
  site on the scheduled date
• ?At the time of pickup, FAX the completed CELLA
  Materials Return List to ETS Customer Service
  (866-387-2598)

108
2009 Test Administration Manual
109
Review of the 2009 Test Administration Manual

• Whats New and Reminders on pages 1-4
• Functional Level Testing on page 15
• Test Invalidation and Do Not Score on page 32
• Package TO BE SCORED Materials for return page 44
• Required Administration Information Document,
  Security Log, Materials Return List, and Testing
  Signs

110
2009 Directions for Administration Scoring
Guides
111
Test Accommodations

• Section in DFA to addresses ALL allowable
  accommodations (begins on page 288)
• Students with Disabilities with Current IEPs
• Students with Section 504 Plans
• Not Permitted Accommodations
• List of not permitted accommodations is included
• Guidance on accommodations for Deaf or
  Hard-of-Hearing

112
Updates to the DFA/Scoring Guides

• Electronic Use Policy added to the first section
  of each DFA/Scoring Guide Level
• Clearer instructions and demonstration graphic on
  how to fill in the responses in the Level A test
  book included (graphic on page 10 of DFA).

113
Pre-ID Labels StudentDemographics
114
Pre-Identification (Pre-ID) Labels

• DOE will provide information about each student
  taking CELLA using data from the Florida
  Automated Student Information Database (Survey 7,
  January 2009)
• Two (2) labels will be provided for each student
• The first label is for the answer sheet for
  grades 3-12 OR for the Level A Student Test Book
  for grades K-2
• The second label is provided as an extra if
  needed

or
or
115
Pre-ID Label Sample

• The following information MUST be correct for the
  student label to be used
• Students Last Name
• Student ID Number
• District Name
• School Number (code)
• Grade Level

116
Required Demographic Information for students
that HAVE a Pre-ID Label

• Even if the student has a preidentified student
  label, the following information MUST be provided
  and gridded on the Level A Student Test Book,
  Level A Functional Level Test Book, or on the
  Level B, C, and D Answer Sheet
• Grid 1 Student Name
• Grid 2 Test Administrator Name
• Grid 3 School Name
• Grid 4 School Number
• Grid 5 District Name and District Number
• Grid 14 Test Date
• Grid 17 Listening/Speaking Test Level
• Grid 18 Reading/Writing Test Level
• Grid 25 Accommodations (if applicable)
• Grid 26 Do Not Score or UNDO Do Not Score (if
  applicable)

117
Demographic Information ONLY for students that
do NOT have a Pre-ID Label

• Personal and demographic information will be
  collected for each student
• Level A, last two pages of the test book
  completed by the Test Administrator
• Level B-D, first two pages of the answer sheet
  assist the student if needed
• Must Grid 1 23 (refer to Test Administration
  Manual)

118
Receipt of Pre-ID Labels

• Pre-ID labels will be provided to each district
• Labels are packaged by grade within each school
• The labels will arrive separate from test
  materials
• REMINDERS
• Do NOT use Pre-ID labels from previous test
  administrations
• Do NOT apply a label over another label

119
2009 Answer Sheet Do Not Score
120
Answer Sheet Levels B-D Reading/Writing Item 18
Test Form and Test Level Selection
Form 2 is pregridded Test Administrator Enters
the Level for Reading/Writing
121
2009 Answer Sheet

• Item 14, Test Date pregridded 2009, fill in
  April or May bubbles. Shaded so the Test
  Administrator knows its a required field
• Item 15, Date of Home Language Survey
• Item 16, ELL Status

122
2009 Answer Sheet Continued

• Item 17, Listening/Speaking Test pregridded
  Form 2 Test Administrator bubbles in Level
• Item 18, Reading/Writing Test pregridded Form
  2 Test Administrator bubbles in Level
• 23, Exceptional Student Education (Grid Only
  One)
• Arranged in Alpha Order
• Updated Exceptionalities to match FCAT

123
2009 Answer Sheet Level A Test Booklets

• Item 26, Do not score contains DNS and UNDO
  bubbles for each content area
• If all or part of a student answer sheet for any
  one of the tests is not to be scored, grid the
  appropriate DNS bubble(s) for each of the tests.
  
• If a test has been invalidated by mistake, erase
  the DNS bubble(s) and grid the corresponding UNDO
  bubble(s).
• If the DNS bubble is gridded for any of the tests
  (Listening, Speaking, Reading or Writing), that
  section will not be scored.
• If all DNS bubbles are gridded, none of the tests
  will be scored.
• If the DNS bubble(s) for a test has been erased
  and the corresponding UNDO bubble(s) has been
  gridded, that test will be scored unless it is
  packaged with the NOT TO BE SCORED materials

124
Questions and Answers