Florida Association of Science Supervisors Presentation

1
Florida Associationof Science SupervisorsPresent
ation

• November 3, 2005
• Orlando, Florida

2
Presentation Topics

• FCAT Update Science
• Setting FCAT Science Achievement Levels
• Grade 11 Student Motivation Discussion
• Interpretive Products
• Release of FCAT Tests
• 2006 Testing Schedule
• FCAT Update Other Content Areas
• Cognitive Complexity

3
FCAT Science Update

• FCAT Science Standard Setting meeting for
  educators representing Grades 5, 8, and 11 held
  in January and September 2005
• DOE plans to report student scores and
  achievement levels beginning with the 2006 test
• Science to be included in school grade
  calculations beginning with the 2007 test

4
Modifications to Appendix B

• Grade 5
• B.1.2.2 now also assesses B.1.2.3, B.1.2.5,
  and
• B.1.2.6
• C.2.2.4 now also assess C.2.2.3
• D.2.2.1 now a CS also assessing B.2.2.2
  and B.2.2.3 (was with G.2.2.3)

5
Modifications to Appendix B

• Grade 8
• B.1.3.1 now also assesses B.1.3.4
• F.2.3.3.- now an Annually Assessed also assessing
  F.2.3.4 and G.1.3.2
• G.2.3.2 now an Annually Assessed assessing
  D.2.3.2, G.2.3.3, and G.2.3.4 (previously the AA
  parent was G.2.3.4

6
Modifications to Appendix B

• Grade 11
• A.1.4.3 now Annually Assessed also assessing
  B.1.4.3
• C.1.4.1 now Annually Assessed also assessing
  C.1.4.2 and C.2.4.6
• H.1.4.2 now also assesses H.1.4.5

7
Upcoming FCAT Science Meetings

• Item Review 3 weeks, 5 different groups
• October 31, November 7, and January 9
• (review of item development for field testing in
  2007)
• Rangefinder Review
• November 30 through December 9
• (review of scoring criteria for 2006 operational
  performance task items)
• Rangefinding
• Wave 1 April and May
• Wave 2 September
• (setting of scoring criteria for performance
  tasks field tested in 2006)

8
Process for Nominating Science Educators

• Nomination forms
• Sent to Superintendents
• Given to District Science Supervisors and
  educators serving on committees
• More diversity needed in Science database
• Gender
• Ethnicity
• Science content areas

9
Next Version of FCAT Science Item Specifications

• On hold pending the revision to the Sunshine
  State Standards
• 2002 Draft of Item Specifications on website
• PDF Appendix B (dated October 2004)sent to
  District Assessment Coordinators also available
  via email request to me
• Sample Test Materials have current version of the
  reference sheets and scoring rubrics

10
Setting Achievement Levels for FCAT Science

• Summary of Process and
• Timeline

11
What is Standard Setting?

• Process of defining the frame of reference for
  the interpretation of test scores in
  criterion-referenced assessments, or
• The matching of achievement levels to test
  scores
• Involves both quantitative and qualitative
  judgment of test items and expected student
  performance

12
Standard Setting Process

• Performance review meeting - January 2004
• Grades 5 and 8 FCAT Science Standard Setting
  meeting held with educators - January 2005
• Grade 11 FCAT Science Standard Setting meeting
  held with educators September 2005
• Educator committees represented schools and
  districts from across the state

13
Standard Setting Process

• Table and Group Discussions focused on
• Desired student content knowledge and skills of
  the SSS benchmarks
• How much content knowledge and skills are
  necessary for each of the achievement levels
• Materials used during the meeting included
• Performance descriptions from previous meeting
• FCAT Achievement Level descriptions
• Test items ordered by difficulty (based on
  student performance)

14
Setting of Achievement LevelsWhat Happens Next

• FCAT Science Achievement Level Timeline
• Business Panel meeting November 2
• Staff review of input and recommendations
• Draft of State Board of Education Rule
• Regional workshops November
• Rule development workshop
• Workshop with State Board of Education
• Advertisement of Rule
• State Board of Education Rule Adoption - February
• Reporting of student scores by Achievement Level
  - May 2006

15
Student Motivation

• Previous discussions on Grade 11 student
  motivation included
• Requiring a specific score on FCAT Science for
  graduation
• Requiring a specific score on FCAT Science OR
  four science courses for graduation
• Science endorsement for students receiving a
  specific score on FCAT Science

16

• Interpretive
• Products

17
FCAT Interpretive Products

• FCAT Handbook
• A Resource for Educators
• Program-level information about the FCAT
• Posted to DOE website
• Copies are now in districts

18
FCAT Resources

• Department of Education FCAT Web Addresses
• http//www.firn.edu/doe/sas/fcat.htm
• or http//www.fldoe.org
• FCAT Publications (General Public)
• FCAT Publications (Educator)
• FCAT Scores and Reports
• Content Focus Reports for 2004 and 2005 tests
• (2006 Report to be posted in May)

19
Some FCAT Publications Available

• Test Design Summary
• Details about the structure of the test,
  including number of items, time limits, and
  session breakdowns
• See handout provided or check the DOE website
• Content Focus Reports
• For each content or reporting cluster
• More specific description of benchmark content
  assessed
• in each reporting cluster, each year

20
FCAT Interpretive ProductsSpring Deliveries

• Mid May 2006
• Florida Inquires! (Grades 5, 8, 11)
• Florida Reads! (Grades 4, 8, 10)
• Florida Solves! (Grades 5, 8, 10)
• Florida Writes! (Grades 4, 8, 10)
• FCAT Results Folder for parents to keep track of
  various reports
• See 2005 2006 FCAT Interpretive Products
  Shipment and Delivery Schedule handout for more
  information

21
Released FCAT Tests

• What is available now?
• What are the plans for future release?
• Where to find more information?

22
Tests Released in 2005

• Reading and Mathematics
• Grade 4 and 8
• 2005 Operational Tests
• Grade 10 Reading
• March 2004 Retake version
• Grade 10 Mathematics
• 2004 Operational version

23
Future plans (if resources are available)
24
Where to Find More Information

• Additional information about released tests is
  available at www.fldoe.org
• Release includes
• Fact Sheet How to use Released Tests and how
  not to use
• Test questions formatted like the FCAT
• Includes all test items that contribute to a
  students score
• Answer Key
• Frequently Asked Questions (and answers)
• And more

25
2005-06 Testing Schedules

• FCAT Writing February 7-8
• FCAT SSS (Reading, Mathematics, Science)
• February 27-March 10
• FCAT NRT
• same period as FCAT SSS
• Summer Reading and Mathematics Retakes
• June 19-23

26
FCAT Update

• What is new for Writing, Reading, and
  Mathematics?

27
FCAT Writing

• February 2006
• Prompted Essay Test with
• Multiple-Choice (MC) Items

28
FCAT Writing

• Writing and Item Types

• Cloze Selection
• Conventions
• 3-option MC
• Stand-alone Structure
• Conventions
• 3-option MC

• Writing Plan
• Focus and Organization
• 4-option MC
• Writing Sample
• Focus, Organization, Support
• 4-option MC

29
FCAT Writing

• Standard Setting no earlier than Fall 2006
• FCAT Writing will count for graduation
  requirement beginning with the graduating class
  of 2010

30
Reading and Mathematics

• Reading Test Item Specifications
• on hold until the SSS revisions complete
• Mathematics Test Item Specifications
• Released on DOE Website in October

31
Cognitive Complexity

• Prior to 2004 based upon Blooms Taxonomy
• Level I and Level II
• Considers instructional background and students
  approach to a problem
• Movement to Cognitive Complexity to classify
  test items.
• Low, moderate, and high
• Based upon Webbs depth of knowledge and NAEP
  system
• Considers the cognitive demand of a test item

32
Low Complexity Items

• Based on recall and recognition of previously
  learned concepts and principles
• Examples include
• Identifying common examples or recognizing a
  concept
• Recognizing a standard scientific representation
  of a simple phenomenon
• Retrieving information from a chart, table,
  diagram, or graph
• Calculating or completing a familiar single-step
  procedure or equation using a reference sheet

33
Moderate Complexity Items

• Require students to decide what to do using
  informal methods of reasoning and problem-solving
  strategies from various domains
• Examples include
• Applying or inferring relationships among facts,
  terms, properties, or variables
• Describing examples and non-examples of
  scientific processes or concepts
• Predicting the next logical step

34
High Complexity Items

• Require students to solve non-routine or unique
  problems, require abstract reasoning, analysis,
  judgment, or creative thought
• Example include
• Generalizing or drawing conclusions
• Analyzing an experiment to identify a flaw and
  propose a method for correcting it

35
FAST Conference Presentation

• Florida Inquires! and the Grade 5 FCAT Released
  Performance Task
• Presented by Roberta McDonald, FCAT Science
  Development Team
• Saturday, November 5 900 a.m.

36
QUESTIONS??

• Contact Information
• Linda Oravetz
• FCAT Science Coordinator
• Test Development Center
• Florida Department of Education
• oravetzl_at_mail.leon.k12.fl.us