A Look Inside the Smarter Balanced Assessment System

1
A Look Inside the Smarter Balanced Assessment
System

• Joe Willhoft, Executive Director

52nd Annual Northwest Mathematics
Conference Hyatt Regency, Bellevue,
Washington October 11, 2013
2
Smarter Balanced

• Assessment System Overview

3
Not just another testSmarter Balanced is being
built by states for states
Preparing our students for a changing world
1
Connecting learning to life after high school
career or college
2
Supporting teachers with a practical suite of
resources
3
Providing meaningful information to guide student
growth
4
Keeping educators in the drivers seat
5
3
4
A State-led Assessment Consortium

• 26 member states and territories representing 39
  of K-12 students
• 23 Governing States, 2 Advisory States, 1
  Affiliate Member
• Washington state is fiscal agent
• WestEd provides project management services

4
5
A State-led Assessment ConsortiumSustainability
for 2014-15 and Beyond

• Future affiliation with the Graduate School of
  Education and Information Studies at UCLA

5
6
Improving Teaching Learning
Summative College and career readiness
assessments for accountability
Teachers and schools have information and tools
they need to improve teaching and learning
All students leave high school college and
career ready
Common Core State Standards specify K-12
expectations for college and career readiness
Interim Flexible and open assessments, used for
actionable feedback
Formative resources Supporting classroom-based
assessments to improve instruction
6
7
Individual Score Reports for Grades 3-8 and 11
Overall Claim for Grades 3-8
ELA/Literacy
Mathematics
Overall ELA/Literacy Score
Overall Mathematics Score
Claim 1 - Reading
Claim 1 Concepts Procedures
Claim 2 - Writing
Claim 2 Problem Solving Claim 4 Modeling
Data Analysis
Claim 3 - Speaking and Listening
Claim 3 Communicating Reasoning
Claim 4 - Research/Inquiry
7
8
Claims for the Mathematics Summative Assessment
Overall Claim for Grades 3-8
Overall Claim for Grade 11
Claim 1 - Concepts Procedures
Claim 2 - Problem Solving
Claim 3 - Communicating Reasoning
Claim 4 - Modeling and Data Analysis
9
Smarter Balanced

• Educator Involvement and Progress

10
Educator Involvement Item Development
Activity Total Educators
Item Author 147
Item Review - Accessibility 53
Item Review - Bias 50
Item Review - Content 138
Pilot Test Data Review 26
Pilot Test Range Finding 96
Stimulus Review - Accessibility 28
Stimulus Review - Bias 31
Stimulus Review - Content 32
Grand Total 601
10
11
Ongoing Involvement

• Pilot Test in spring 2013 involved over 650,000
  students from all Smarter Balanced states
• Field Test in spring 2014 will involve over two
  million students in all Smarter Balanced states
• Standard setting in fall 2014 will invite all
  teachers to participate in virtual crowdsourcing

11
12
Working with Higher Education

• Ongoing state-level outreach to faculty and
  administrators
• Five Regional Coordinators
• Higher education leads in each state
• Two higher education reps on Executive Committee
• Higher education votes on major policies
• College Content-readiness Policy
• Career Readiness Task Force
• Comprehensive validity research agenda

12
13
Major Milestones in Development of Summative
Assessments
?
?
?
Full system run-through Establish performance
standards some results
Early Q.C. of items software no student
results
13
14
Field Testing

• March - June 2014
• Sampling about 20 of students in Consortium
• Purpose Evaluate items and tasks for Smarter
  Balanced pool
• Statistical data analysis of 20,000 items
• Divide items/tasks into secure (summative) pool
  and open (interim) pool
• Conduct standard setting for different
  performance levels (cut scores)
• US Dept. of Ed flexibility to avoid double
  testing

14
15
Setting Performance Standards

• Setting standards for four Performance Levels at
  each grade
• September 2014, using Field Test data
• Grade-level panels of 30 per content area
  recommend cut scores
• At least two representatives from each Governing
  State on each grade level panel
• Concurrent crowd-sourced recommendations from
  the field
• Recommendations presented to state chiefs for
  approval in late September 2014

16
Smarter Balanced

• Getting Ready for the Tests

17
Testing Window for Summative Assessments in 2015

• Each state sets its own schedule for the Smarter
  Balanced summative assessments within the
  following windows
• Grades 3-8 Testing can begin once sixty-six
  percent (66) of a schools annual instructional
  days have been completed
• Grade 11 Testing can begin once eighty percent
  (80) of a schools annual instructional days
  have been completed and
• All Grades Testing can continue up to and
  including the last day of school.
• Smarter Balanced is designed to have results
  returned within 2 weeks of of testing being
  completed.

17
18
Technology Requirements Responding to School
Needs

• Smarter states have established standards for new
  and existing hardware
• Online Readiness Tool
• Schools and districts can evaluate technology
  readiness
• Schools do NOT need one-to-one computers
• Illustrative example A 600-student school can be
  supported by a single 30-computer lab
• Smarter Balanced Readiness Calculator at
  http//www3.cde.ca.gov/sbactechcalc/
• Pencil-and-paper option available for three-year
  transition period

19
Accessibility Accommodations
19
20
Accessibility Accommodations (from S.B.
Guidelines)
20
21
Estimated Testing Times for 2014 Field Test and
2015 Operational Assessment
Test Type Grades CAT Perf. Task Only Total Class Activity Total
English Language Arts 3-5 130 200 330 30 400
English Language Arts 6-8 130 200 330 30 400
English Language Arts 11 200 200 400 30 430
Mathematics 3-5 130 100 230 30 300
Mathematics 6-8 200 100 300 30 330
Mathematics 11 200 130 330 30 400
COMBINED 3-5 300 300 600 100 700
COMBINED 6-8 330 300 630 100 730
COMBINED 11 400 330 730 100 830
Times are estimates of test length for most
students. Smarter Balanced assessments are
designed as untimed tests some students may need
and should be afforded more time than shown in
this table.
22
Take a Peek Under the HoodThe Smarter Balanced
Practice Test

• Shows item types and tools/features
• Now available Released May 29
• Expanded full-feature version in spring 2014

http//www.smarterbalanced.org/practice-test
22
23
A Deeper Dive

• Mathematics Content Specifications and Test
  Blueprints

24
Test SpecificationsClick here.
25
Then, scroll to here
26
Gr. 6 Math Blueprint Claim 1 (DRAFT)
27
Gr. 6 Math Blueprint Claims 2 4 (DRAFT)
28
Gr. 6 Math Blueprint Claim 3 (DRAFT)
29
A Few Sample Items
30
Grade 3 Problem Solving

• Steven is baking cookies. On one tray, he makes 5
  rows with 4 cookies in each row. He cannot fit
  any more cookies on the tray. He puts some more
  cookies on another tray the same size as the
  first tray.
•  
• Which number is a reasonable total for the number
  of cookies on the two trays?
•  
• 9
• 20
• 34
• 60

31
Grade 3 Communicating Reasoning
32
Grade 4 Modeling and Data Analysis
33
Attending to Coherence

• The Number Line Example

34
Grade 3
35
Grade 7
36
Grade 8
37
Learn More and Stay Engagedwww.smarterbalanced.or
g

• Visit us for the latest news and developments
• Sign up for our e-newsletter
• Follow us on Twitter at _at_SmarterBalanced