DAC Academy

1
DAC Academy August 27, 2009 Colorado
Department of Education Unit of Student Assessment
2
DAC Academy 2009
Todays Agenda

• Assessment 101
• Essential Information for 2009-2010
• DAC Panel/Q A Session (working lunch)
• Accommodations
• Break-out Sessions
• Colorado Growth Model
• S.B. 09-163
• Navigator Demo and CTB Logistics

3
DAC Academy 2009

• The it factor!

4
DAC Academy 2009
DAC Roles Responsibilities

• In regards to the administration of the state
  assessments, DACs ensure
• Quality
• Accuracy
• Security
• Documentation
• Communication
• Training

5
DAC Academy 2009
DAC Roles Responsibilities

• Appointed by the district superintendent
• - Responsibilities and powers may vary
• Need to have a specific and specialized skill
  set/level of expertise
• - Educational or psychometric background

6
DAC Academy 2009
DAC Roles Responsibilities

• You Have a Vital Leadership Role!
• Official point of contact
• - One DAC (One official district e-mail address)
• - Account for any and all information to USA/CDE
  and CTB
• ONLY district staff member officially designated
  to communicate with the school district, USA, and
  the assessment vendor
• The first question we ask those who call us
  Have you talked to your DAC?

7
DAC Academy 2009
DAC Roles Responsibilities

• Oversee Administration
• - Scheduling (Section 5.0)
• - Ethical practices (Section 4.0)
• - Misadministrations (Section 7.0)
• Personnel issues
• - Distribution, collection, packing and shipping
• - Accommodations are provided and documented

8
DAC Academy 2009
DAC Roles Responsibilities
Training/Communication (Sections 7.7 9.3)
- Train SACs and Test Proctors (documentation) -
Consistent and reliable distribution of critical
information to superintendents, teachers, and
core assessment team members Manage
Materials - Maintain security of all materials
(Section 8.0) - Ensure accurate score reports
(Section 14.0) Pre-coded label use and student
data grid
9
DAC Academy 2009
Colorado Student Assessment System
Unit of Student Assessment
Achievement(Colorado Model Content Standards)
College Entrance
National and State Trends
EnglishLanguage Acquisition(Colorado ELD
Standards)
Colorado Student Assessment Program
Colorado ACT
National Assessment of Educational Progress
Colorado English Language Acquisition Program
CSAP
COACT
NAEP
CELApro
CELAplace LCE
CSAPA
On December 18, 2006 Colorado received FULL
Approval from the USDoE for our Standards and
Assessment System
10
DAC Academy 2009
Colorado Student Assessment Program CSAP

• PURPOSE To measure student achievement relative
  to the Colorado Model Content Standards.
• The assessment has been in place since 1997
  (pre-NCLB).
• 98 percent participation rate across grades 3-10
• 31 tests
• Science (Grades 5, 8, 10)
• Reading(Grades 3-10)
• Writing (Grades 3-10)
• Mathematics (Grades 3-10)
• Lectura/Escritura (Grades 3 4)

11
DAC Academy 2009
Colorado Student Assessment Program CSAP
12
DAC Academy 2009
Colorado Student Assessment Program Alternate
CSAPA

• PURPOSE Standards based achievement test for
  students with a significant cognitive disability.
  
• Eligibility
• Determined through a students IEP team.
• Less than 1 percent of all students.
• 27 tests (all CSAP grades and content areas)

13
DAC Academy 2009
Colorado English Language Acquisition Assessment
CELApro

• PURPOSE English Language Acquisition
• NOT to be confused with an achievement test
• Measures four domains of language development
• Speaking, Listening, Reading, Writing
• Academic Language Social, Basic Interpersonal
  Communication
• Given to all NEP and LEP K-12 students
• As defined per LCE Unit definitions

14
DAC Academy 2009
Colorado ACT

• College Entrance Exam
• Accepted by U.S. Colleges, Universities, Military
  Academies and NCAA
• Colorado ACT (COACT)
• State test date 4/28/10
• Make-up test date 5/12/10
• All 11th grade students by law
• Vouchers
• Only ONLINE schools- ACT National test date
  4/10/10
• 11th Grade Alternate for students eligible to
  take CSAPA
• Managed by Exceptional Student Leadership Unit
• 2010 dates TBD

15
Colorado ACTCOACT
DAC Academy 2009
Test Supervisor (TS)
DACs may need to develop a communication process
with Test Supervisors for Accountability.
CONTRACT
DAC
Test Supervisors must develop a communication
plan with Back-up Test Supervisors and Test
Accommodations Coordinators.
16
ACT UPDATES
DAC Academy 2009

• ACT Plastic Polymailer Bags
• Schools will continue to use red envelopes
  green envelopes
• Envelopes placed in separate polymailer bags
  instead of in return boxes
• Standard time answer documents administrative
  forms only

17
ACT Blended Supervisors Manual
DAC Academy 2009

• Includes instructions for both the ACT Plus
  Writing and the ACT No Writing
• Both the standard time and special testing
  manuals have been revised with this combined
  format
• Follows the successful model of our ACT National
  Supervisors Manual

18
ACT Online Services for State Testing
DAC Academy 2009

• School Information forms, Appointment of Testing
  Staff forms, Testing Staff Profile forms,
  Confirmation Forms, Makeup Order Forms handled
  online this year
• Minimize use of paper forms increase
  convenience for school staff
• Online tutorial offered to the Principal and
  Testing Staff
• Tutorial posted online for interested parties
• Online schools will not be subject to this change
  for the 2009-2010 testing cycle

19
DAC Academy 2009
National Assessment of Educational Progress NAEP

• 2010 National (nearly 93,000 students) and
  Sampling
• of grades 4, 8 and 12
• -CO has shortened window Feb 1- 26, 2010

20
National Assessment of Educational Progress NAEP
DAC Academy 2009

• Participating schools selected by National
• NAEP Statisticians
• All schools identified last May 25 selected G4
  6, G8 12, G12 7
• All participating schools have received initial
  notification from NAEP State Coordinator
• Will receive national results for grades 4, 8
  12 for the social studies subjects by variables
  and background survey responses. Do not receive
  disaggregated results for districts or individual
  schools- not designed for this.

21
National Assessment of Educational Progress NAEP
Roles
National NAEP Office
NAEP State Coordinator NSC
School Community
Contracted Assessment Team
22
National Assessment of Educational ProgressNAEP-
Relationship between NAEP School Community
NAEP State Coordinator

• Confirms the assessment date
• Provides schools with info for parental
  notification
• Responds to questions
• Works with district/school personnel to ensure a
  smooth process

District Contact

• Assists NSC in School
• communications
• Works with NSC and the Supervisor to
• Oversee the process

NAEP School Coordinator
School Community
23
NAEP-The Nations Report Card public Web site
DAC Academy 2009

• http//nces.ed.gov/nationsreportcard/

24
NAEP-The Questions Tool http//nces.ed.gov/nation
sreportcard/itmrls/
DAC Academy 2009
25
NAEP-Thousands of Released Items
DAC Academy 2009
Sample Item Math Grade 8 MC, medium Difficulty
  What is the intersection of rays PQ and QP in
the figure above?    A) Segment PQ B) Line
PQ C) Point P D) Point Q E) The empty set 
26
Assessment Purposes
Program Evaluation
USA Unit of Student Assessment
Achievement Tests
CSAP CSAPA
Body of Evidence (Program / Placement)
Language Acquisition Test
CELAplace CELApro
27
Unit of Student Assessment Management,
Administration and Data Reporting
Where the data go
AYP (Title I)
SchoolView
Colorado Longitudinal Growth Model
Accreditation
AMAOs (Title III)
28

School List (August 31st)
CTB CSAP/CSAPA/CELApro
Student (October)
Online Enrollment (November)
Updated Pre-coded Labels (January)
CO ACT (Sept)
Final Label File
Other Test Companies CAP4K
USDOE
CDEs Data Warehouse
SBD
AYP AMAO CEDAR EDEN
Schools/Parents/Students
Growth/ Accreditation
Press Releases
29
Data Operations - Top 5!

• The lifecycle of assessment data
• School Master, Data grids, SBD, Reporting,
  Accountability
• ADE collections Pre-Coded Labels
• SBD CELA, CSAP/CSAPA, COACT
• ADE respondent for each collection
• Logistics Training in November online
• Online Enrollments on Navigator in the fall
• CELApro
• CSAPA and CSAP special populations
• New for 2010-11 Race/Ethnicity codes

30
DAC Academy 2009
Assessment Life Cycle CSAP Example

• Reporting

District Validates their own data prior to final
reports
August/July
Online Enrollments
May/June
Student Biographical Data Review
After
Oct/Nov
Nov
During
Dec
DAC Training
Precoded Labels
DACs Train SACs
31
CTB.com and Navigator

• Secured Site for DACs
• Online training
• Online enrollments and special populations
• Access oral scripts, translated oral scripts,
  teacher read directions for CSAP
• Archived Emails
• Data NCounts, GRT files, official data retrieval

32
CTB Navigator (CSAP, CSAPA, and CELApro)
33
DAC Academy 2009
Core Assessment Team

• Who is it?
• District Assessment Coordinator
• ELA Director/Coordinator
• SPED Director/Coordinator
• CBLA Contact
• Others

34
DAC Academy 2009
Core Assessment Team

• Their Role
• Are students receiving the services they need?
• Instructional
• Programmatic
• Assessment
• Collaboration and Communication are key!
• This ensures alignment between instruction and
  assessment.
• Required trainings

As the population of Colorado changes, so do the
needs of our students.
35
DAC Academy 2009
Core Assessment Team

• Critical functions include, but are not limited
  to
• Online Enrollments/Early Test Windows
• On CTB Navigator
• Automated Data Exchange Collections
• Through CDE (Student October, Student
  Biographical Data, etc.)
• Accommodations in place THINK EARLY!
• Attendance at November trainings
• CSAP, CSAPA, CELApro (face to face)
• CSAP, CELApro, CSAPA logistics (online only)
• Online administration trainings for CSAP, CSAPA,
  and CELApro will still be available.

36
DAC Academy 2009
Core Assessment Team District Assessment
Coordinator (DAC)

• Oversees and manages all aspects of state
  assessment administration
• Ensures standardized ethical test administrations
• Training
• Manage data collection processes
• What else???

37
DAC Academy 2009
Core Assessment Team ELA Director

• Appropriate accommodations
• Colorado Accommodations Manual for ELLs
• Accommodations monitoring
• Assessing language proficiency
• Designation and re-designations of ELLs (NEP, LEP
  and FEP) for assessment purposes
• Meeting the needs of ELLs with disabilities
  (dually identified)
• Distinguishing between language difficulty and
  learning disability
• Language Culture and Equity Unit
• Joanna Bruno (303) 866 - 6870

38
DAC Academy 2009
Core Assessment Team Special Education Director

• Meeting the assessment needs of students with
  disabilities
• Colorado Accommodations Manual
• Matching a specific need by including specific
  accommodations in plans
• IEP (Includes eligibility for CSAPA)
• 504
• RtI
• Accommodations monitoring
• Work with ELL team to meet needs of dually
  identified students
• Non-standard Accommodations Requests
• Exceptional Student Leadership Unit
• Dena Coggins (303) 866 - 6863

39
DAC Academy 2009
Core Assessment Team CBLA Contact

• Body of evidence from assessments
• Programmatic interventions/ILPs
• Accommodations aligned
• Literacy Grants Initiatives
• Dena Coggins (303) 406 - 8653

40
DAC Academy 2009
Core Assessment Team District Trainers

• Faculty and staff must be trained
• on a yearly basis!
• This is true for EVERYONE
• Core Assessment Team ? Teacher administering the
  test
• (November training will be discussed in future
  slides.)

41
DAC Academy 2009
Core Assessment Team Expansion

• Districts are unique
• Other staff may need to be included in the work
• ADE respondents
• Superintendents/other District Leadership
• SASID/RITS respondents
• Lead Teachers, etc.
• Communication is key

42
DAC Academy 2009
Core Assessment Team Automated Data Exchange
(ADE) Respondent

• Who are they?
• Responsible for submitting data from the various
  data collections throughout the year.
• These include
• Student October (November)
• Precoded Labels (January)
• CELApro Student Biographical Data (March)
• CSAP/CSAPA Student Biographical Data (June)
• COACT Student Biographical Data (July)

43
DAC Academy 2009
Reporting Results

• Requirements
• CRS 22-7-409-1.9
• The results of the assessmentsshall be part of
  the students permanent academic record.
• CSAP labels on high school transcripts
• Parents receive reports in a timely manner
• All schools in each district receive reports in
  at timely manner

44
DAC Academy 2009
CDE/USA Support and Service

• We recognize that you have other duties as
  assigned.
• How we support districts
• Resources
• Comprehensive manuals and guides
• Trainings
• November
• Face-to-face (January)
• Web ex (CTB Navigator)
• -24/7 availability
• E-mail and Phone calls

45
DAC Academy 2009
CDE/USA Support and Service

• Unit of Student Assessment e-mails to DACs
• Try to limit these
• E-mails archived on CTB Navigator
• Scoop
• Weekly newsletter sent out to districts
• Subscribe by contacting the CDE Communications
  Unit at CDE_Communications_Office_at_cde.state.co.us
  
• Includes important deadlines, announcements and
  reminders
• Compiles multi-unit info from CDE for easy access
  to
• Superintendents, DACs, etc. (anyone who
  subscribes)

46
DAC Academy 2009
Assessment Resources

• 2009-2010 Procedures Manual
• 2009-2010 Accommodations Manuals
• 2010 SAC/DAC Manuals
• Test Proctors/Test Examiners Manuals
• SBD Manual
• Technical Reports
• Data Interpretation Guidelines
• Guides to Test Interpretation

47
DAC Academy 2009
Questions about Scores

• If school/district personnel or parents have a
  question about a score
• First, the DAC should review
• GRT file
• Past results
• District assessment results
• Irresolvable questions are forwarded to CDE by
  the DAC
• USA works with CTB to examine the data
• May possibly call you in to review material with
  us
• Parents cannot view assessments

48
DAC Academy 2009
Questions about Items

• Proctors should not be studying items
• Items only accessed during test administration
• If a test proctor or examiner has a question
  about an item
• Contact the SAC, who will contact the DAC
• USA works with the DAC
• We are glad to address important questions about
  the items
• Include in proctor training instructions
• Do Not copy the item
• Do Not discuss the item (with other teachers,
  etc.)
• Do Not email the content of the item

49
DAC Academy 2009
How to Handle Advice from Sources other than USA

• When you have questions
• Contact Unit of Student Assessment (Section 1.0)
• Record the name of the staff member you speak
  with
• We cannot verify the accuracy of information
  given by other districts or outside sources
• Know your lifelines

50
DAC Academy 2009

• BREAK
• (15 minutes)

51
DAC Academy 2009
Whats New for 2009-2010?

• Colorado Growth Model
• http//www.schoolview.org/
• Documentation (one signed document)
• all district personnel have been trained
• DACs hold on to individual signed forms

52
DAC Academy 2009
Whats New for 2009-2010?

• Procedures Manual - main updates
• Scheduling pickups of materials (Section 2.4)
• Location of materials (Section 6)
• Punch out tools (Section 7.3)
• Handling misadministrations (Section 7.6)
• Chewing gum (Section 9)
• Eligibility flowchart for CSAP vs. CSAPA (Section
  10)
• Home schooled students (Section 12.1)
• Facilities (Section 12.18)
• When a student response requires action (Section
  12.21)
• Index

53
DAC Academy 2009
Whats New for 2009-2010?

• Accommodations Manual updates
• 5 step process for selection accommodations
• Eligibility flowchart for CSAP vs. CSAPA
• More defined rules around scribing
• Abacus was added as a restricted accommodation
• Nonstandard accommodations
• Same form
• Copy of an IEP or 504 highly recommended
• Not allowed for grievously ill students

54
DAC Academy 2009
Peer Review Requirements

• How do you know?
• Training
• Housecleaning
• Chain of Custody

55
DAC Academy 2009
Peer Review Requirements

• Training (Section 9.3)
• Inadvertent misadministrations do not occur
• Comprehensive and ongoing training must include
• Test Security
• Test Proctor Requirements
• Teacher vs. Test Proctor
• Ethics
• Required documentation
• Individual documentation (DACs keep)
• One form submitted to Unit of Student Assessment
• School administrations must be involved

56
DAC Academy 2009
Peer Review Requirements

• Housekeeping (Section 7.4)
• USA recommends signed statements on file in each
  district
• Every principal signs ensuring that there are no
  assessments or proprietary materials related to
  the state assessments stored anywhere on the
  school premises (e.g. test booklets from previous
  years)
• Ensure that any teacher-created materials
  (whether distributed or not) are free of actual
  assessment items
• (Does not apply to released items) DACs need a
  methodology to document that this has been taken
  care of at every school.

57
DAC Academy 2009
Peer Review Requirements

• Chain of Custody (Section 8.2)
• Documented before, during, and after testing
• Large print, Braille, Teacher Read Directions,
  Oral Scripts, Translated Oral Scripts
• Materials should not be delivered to schools more
  than 1 week in advance
• Distribute only the content area being assessed
• Returned to a designated secure location
• Not stored in classrooms

58
DAC Academy 2009
Assessment Timeline 2009-2010 (Section 2)

• CSAP, CSAPA, CELApro and CELAplace
• Training
• September ELA, Accommodations, and Procedures
  Manuals
• November Required training for CSAP, CSAPA,
  CELApro, and logistics (i.e. SAC/DAC manuals)
• Ordering and Delivery of Materials
• October Online Enrollments for CELApro
• October Online Enrollments for CSAPA
• November Online Enrollments for CSAP special
  populations
• November Finalize shipping information for CTB
• May Online enrollments for CELAplace 2009-10
• Test Administration
• October-December Submit Nonstandard
  Accommodation forms for approval
• November Early testing window request completed
• January CELApro Administration window
• February-April CSAP and CSAPA Administration
  window
• Data
• September Student October collection opens
  (CELApro Precoded labels)
• December Precoded Labels collection opens (CSAP,
  CSAPA, and COACT
• March Grade 3 CSAP Reading Ncount and CELApro
  Ncount and SBD

59
DAC Academy 2009
Assessment Timeline 2009-2010

• COACT
• Test Supervisor Checklist of Dates
• Late September 2009
• ACT mails initial letter from CDE and School
  renew form to principals with Checklist of Dates
  and Standard Testing Requirements.
• October 2009
• Receipt Deadline for principals to return School
  Renewal Forms.
• Receipt Deadline for TS and TACs to return
  renewal forms to ACT
• November 2009
• TACs receive mailing which includes procedures
  for requesting accommodations, quantities of
  individual request forms, and an order form for
  practice test.
• Online schools receive a special mailing which
  includes National Test Date Voucher Eligibility
  Request and Test arrangements to be made for May
  makeup.
• Mid November 2009
• Schools are reminded to submit off-site testing
  requests, if appropriate.
• Late November 2009
• Announcement of training workshop locations and
  dates mailed to TS, BTS, TACs, and DACs.
• December 2009
• Receipt deadline for schools to submit completed
  proposals for off-site testing arrangements to
  ACT
• Early January 2010
• Receipt deadline for CDE Assessment to receive
  National Test Date Voucher Eligibility Request
  for ONLINE students

60
DAC Academy 2009
November Training Schedule

• Nov. 3 Pueblo
• Nov. 4 Colorado Springs
• Nov. 10 Durango
• Nov. 11 Grand Junction
• Nov. 13 Denver
• Nov. 16 Ft. Collins (tentative)
• Nov. 17 Ft. Morgan (tentative)
• Nov. 18 Boulder (tentative)

61
DAC Academy 2009
Standards Review Update

• Phase I Mathematics Music Science Reading,
  Writing Communicating
• New drafts available by August 31st
• Phase II Civics Economics Geography History
  and expectations for personal
  financial literacy
• Recently completed the 8-city public feedback
  tour on Aug 26th
• Phase III Dance Health Physical Education
  Theatre Visual Arts World
  Languages
• In progress
• Public feedback tour set for October 5th 21st
• This work also includes the Expanded Evidence
  Outcomes and the ELL Standards.

62
DAC Academy 2009
Year of Assessment

• Stakeholders group being formed now
• Assessment survey to be posted in September
• Current plan is to hold meetings across the state
  in order to compile public feedback
• Similar to standards revision process
• RFP due out in June 2010

63
DAC Academy 2009
Office of Standards and Assessments (303)
866-6929

• Standards
• Jo OBrien Assistant Commissioner
• Angela Norlander Standards
• Anna Huffman Standards
• Student Assessment
• Jim McIntosh Director
• Andrina Aragon CSAP
• Pam A. Sandoval NAEP
• Christine Deines COACT
• Margaret Lake Data Operations
• TBA CSAPA
• TBA CELApro
• TBA Data Operations
• Research and Evaluation
• Dianne Lefly Director
• Bill Bonk Longitudinal Growth
• Marie Huchton Longitudinal Growth