Assessment and Accountability in Kentucky

1
Assessment and Accountability in
Kentucky
Phyllis Shuttleworth Program Consultant Kentucky
Department of Education
2
Assessment and Accountability in Kentucky

• Assessment
• Components
• Standards
• Accountability
• Raw/Scale/Cut Scores
• Accountability Indices
• Growth Chart
• Rewards/Assistance

3

• COMMONWEALTH ACCOUNTABILITY TESTING SYSTEM

4
Assessment
5
Two Basic Types of Assessments

• Criterion-referenced or Standards-Based
• student performance is measured against a set
  standard

• Norm-referenced
• - student performance is
• measured against the
• performance of other
• students

6
Kentuckys Assessment Program
7
ELEMENTARY TESTING
8
MIDDLE SCHOOL TESTING
9
HIGH SCHOOL TESTING
10
WHO CREATED CATS?

• CTB/MCGRAW-HILL TESTING COMPANY

11
WHO CREATED CATS?

• OVER 300 TEACHERS INVOLVED Blueprint
  Construction, Content Advisory Committee,
  NRT/Core Content Comparison, and Bias Screening
• OVER 6200 EDUCATORS, PARENTS, CITIZENS SHARED
  INPUT

12
WHO CREATED CATS?

• ADVISORY GROUPS EAARS - Education Assessment
  and Accountability Review Subcommittee
• NTAPAA - National Technical Advisory Panel on
  Assessment and Accountability
• OEA - Office of Education Accountability
• SCAAC - School Curriculum, Assessment and
  Accountability Council

13
WHO CREATED CATS?
14
WHO CREATED CATS?

• STATE BOARD OF EDUCATION

15
Norm-Referenced Test

• CTBS/5 Survey Edition
• Norm-Referenced
• Spring Administration
• Multiple-Choice
• Created by CTB McGraw-Hill (testing contractor)

16
WHY HAVE NRT TESTS?

• Tests basic skills
• Counts in accountability formula
• Measures basic skills across the country
• Compares students nationally

17
Writing Portfolio

• Standards-Based
• Developed Over Time
• Collection of Students Best Work
• Created from Classroom Assignments

18
GRADE 4 PORTFOLIOS

• 4 pieces includes Reflective (letter to
  reviewer) Personal Literary Transactive
• 1 piece from content area other than language
  arts

19
GRADE 7 PORTFOLIOS

• 5 Pieces
• A. Reflective (letter to reviewer)
• B. Personal
• C. Literary
• D. Transactive
• E. Choice of B-D
• 1 piece must be from content area other than
  language arts/English

20
GRADE 12 PORTFOLIOS

• 5 Pieces
• A. Reflective (letter to reviewer)
• B. Personal C. Literary
• D. Transactive
• E. Choice of B-D
• 2 pieces must be from content area other than
  language arts/English

21
Kentucky Core Content Test
(KCCT)

• Standards-Based
• Spring Administration
• Includes Open-Response, Multiple-Choice,
  Writing Prompts
• Created by Kentucky Teachers Based on KYs Core
  Content for Assessment

22
KENTUCKY CORE CONTENT TESTS

• 8 MULTIPLE CHOICE AND 2 OPEN RESPONSE IN EACH
• ARTS HUMANITIES
• PRACTICAL LIVING
• VOCATIONAL STUDIES
• WRITING ON DEMAND - CHOICE OF TWO PROMPTS

• 24 MULTIPLE CHOICE AND 6 OPEN RESPONSE IN EACH
• READING
• MATH
• SCIENCE
• SOCIAL STUDIES

( MULTIPLE CHOICE ARE WEIGHTED 33 AND OPEN
RESPONSE ARE WEIGHTED 67)
23
WHAT ABOUT PRETEST ITEMS?

• Found on all KY. Core Content Tests
• Used to test items with Kentucky students
• Mixed with regular test

• Doesnt count in CATS accountability
• Items can be used on later forms of KY. Core
  Content Test after testing with students

24
Two Types of Standards
Content Standards
What Content
25
KCCT Test Blueprint for Science
26
Two Types of Standards

• Performance Standards

how well
27
Steps for Setting Performance Standards
Draft Performance Descriptions
Jaeger-Mills
Contrasting Groups
CTB Bookmark
Synthesis
KBE Decision
28

• NOVICE
• APPRENTICE
• PROFICIENT
• DISTINGUISHED

29
WHAT ABOUT STUDENTS WITH
SPECIAL NEEDS?

• ALL STUDENTS WITH DISABILITIES ARE INCLUDED IN
  CATS.
• THOSE WITH MILD DISABILITIES TAKE ASSESSMENTS-USE
  ACCOMMODATIONS FROM THEIR IEP.

30
WHAT ABOUT THOSE WITH MODERATE TO SEVERE NEEDS?

• FIRST - MAY BE DETERMINED TO BE EXEMPT FROM
  STANDARD ASSESSMENTS.
• NEXT PRODUCE
• AN ALTERNATE PORTFOLIO (COLLECTION OF STUDENT
  WORK)

31
Alternate Portfolio

• Involves students with profound and severe
  disabilities in the assessment program
• Developed over time
• Collection of a students best work
• Created from classroom assignments

32

Accountability
33
Raw Scores Scale Scores
Cut Scores
34
Raw Scores
6 open-response items
24 multiple-choice items
2 open-response items
8 multiple-choice items
35
(No Transcript)
36
Raw Scores
FORM 6
FORM 4
37
Scale Scores
FORM 4
FORM 6
38
Science
39
Following A Student through the
Kentucky Core Content Test (KCCT)
Students Science Answers Form 2 Open Response
Raw Score
Scale Score
19 x 2 38
525
53
Multiple-Choice
- - - - - - - - -
15
40
Accountability Index
41
Middle School Science

Weight
Calculation
0
13
Proficient
Distinguished
Total
67.72
67.7
Estimated Science Academic Index
42
(No Transcript)
43
(No Transcript)
44
(No Transcript)
45
ACCOUNTABILITY INDEX
Academic and Non-academic measures are added
together to create a schools total score or
Accountability Index.
46
Accountability Index
Non Academic Data Attendance Retention Dropout Tra
nsition

Schools Accountability Index



47
Growth Chart Every accountable school
1999/2000 Baseline Used to set goals and
determine progress

48
 
Novice Reduction Targets
Baseline
Recognition Points
Dropout Rate
Goal
Standard Error
Rewards Information
49
Rewards

• Allocations based on school size
• Meet dropout and novice reduction
• School-based decision making council decides
  how to use

A school meets or exceeds Goal Line 3 shares A
school within the progressing area ½ share
with an
index greater than it had in the last biennium
A school passes through the recognition 1
share points the first
time (at 55, 66, 77, 88, and 100)

Top five percent (exceeding
88 and 100) 1 share


50
Assistance

• All schools with indices below their assistance
  lines are
• divided into three levels.
• Level 1 Conduct Scholastic Reviews and
  Self- studies
• Level 2 Receive Scholastic Reviews
• Level 3 Receive Scholastic Audits and Assistance
  Teams with Highly Skilled Educators

51
Scholastic Audit

• The Kentucky Department of Education is required
• by House Bill 35 to conduct audits of schools
  that
• fail to meet achievement goals for each biennium.
• Team members defined by law and come from
• outside the district.
• Standards and Indicators for School Improvement
• document used as a measurement instrument.
• Allows schools to focus on their specific needs.
• Special funds available to help schools work on
• these needs.

52
WHY GIVE STATE TESTS?

• GIVES A MEASUREMENT OF STUDENTS PERFORMANCE
• TELLS WHERE STUDENTS NEED HELP
• USE TO MAKE DECISIONS IN CLASSROOMS ABOUT
  LEARNING
• INDICATES HOW SCHOOLS ARE PERFORMING

53
WHY ARE THESE TESTS SO IMPORTANT?

• PROVIDES A PICTURE OF LEVEL OF LEARNING
• DIRECTS LEARNING FOR ALL STUDENTS
• IDENTIFIES WEAKNESSES AND NEEDS
• RESULTS IN IMPROVED INSTRUCTION AND PERFORMANCE

SUCCESS IN TODAYS WORLD REQUIRES STUDENTS TO
KNOW MORE
54
HELPFUL HINTS FOR SUCCESS

• INCLUDE MULTIPLE CHOICE OPEN RESPONSE IN
  ROUTINE ASSESSMENTS
• KNOW THE CORE CONTENT AND ALIGN INSTRUCTION TO IT

• FOLLOW THE PROGRAM OF
  STUDIES

55
OPEN RESPONSE INFO

• Require students to combine content knowledge
  and application of process skills in
  communicating answers
• Open response are appropriate for all students

• CONTENT
• HIGHER ORDER THINKING

PROGRAM OF STUDIES DISK INFORMATION
56
OPEN RESPONSE INSTRUCTIONAL STRATEGIES

• Stress communication (explain expand)
• Give time to think respond
• Help to anticipate scoring of a question
• Use practical contexts (realize relevancy of what
  they are learning)
• PROGRAM OF STUDIES DISK INFORMATION

• Encourage self-evaluation (look for evidence of
  quality refinement of skills)
• Provide successful experiences with higher level
  thinking
• Evaluate frequently (helps progress by
  determining quality of own work)

57
OPEN RESPONSE ANSWERING STRATEGIES

• UNDERLINE THE QUESTION
• NUMBER PARTS
• CIRCLE KEY
  VERBS
• FOCUS ON QUALITY, NOT QUANTITY

• LOOK FOR CORE CONTENT CLUES
• BE CONCISE

58
OPEN RESPONSE FORMAT

• STUDENTS ANSWERS ARE LIMITED TO ONE PAGE FOR
  EACH OPEN RESPONSE QUESTION (EXCEPT IN WRITING)

59
THINGS TO REMEMBER ABOUT OPEN RESPONSE

• SCORED FOR SUBSTANCE-NOT WRITING STYLE
• NO CREDIT FOR INCORRECT RESPONSE
• RESTATE QUESTION ONLY IF HELPS KEEP FOCUS
• ASSESSES CONTENT, NOT WRITING
• NEED TO MANAGE TEST-TAKING TIME

60
CHECKLIST FOR OPEN RESPONSE

• ANSWERED ALL ELEMENTS
• ANSWERED CONTENT OF PROMPT
• ORGANIZED ANSWER
• GAVE SUPPORT FOR ANSWER
• (EVIDENCE EXAMPLES)
• NOTE NOTHING CAN BE ON A STUDENTS DESK DURING
  TESTING
• EXCEPT THE MATERIALS WHICH CAME WITH THE TEST, A
  PENCIL/PEN,
• AND BLANK SCRATCH PAPER.

61
KDE Website http//www.kde.state.ky.us
or http//www.kentuckyschools.org