PAY ATTENTION There is going to be a test.

1
PAY ATTENTION!There is going to be a test.

• Standardized Tests
• and Education Reform
• Keith Clay, GRCC

2
Our love of tests, part I

• Are we going to be tested on this?
• An inservice teacher at a biotechnology workshop
  at UC San Francisco

3
Our love of tests Part II

• Check the web.
• The WASL is
• stupid
• evil
• AWFL
• child abuse
• terrorism

4
Tests and Education Reform

• Why Standardized Tests?
• The Art and Science of Testing
• The state of Washington
• Do tests help with education reform?
• Do education reforms help on the tests?

5
Why standardized tests?

• Some critics argue that High-Stakes Testing is
  just a bad idea

but its not
Its the law.
6
Why standardized tests?

• Assessment of curricula
• Core Knowledge? FOSS? PBI?
• Assessment of systems
• Seattle School District? Mrs. Nelsons class?
• Assessment of individuals
• Formative? Summative? Instructive?

7
Assessment of Curricula
8
Assessment of systems

• Identify substandard schools
• Identify outstanding schools
• Then what?
• Allow students to leave weak schools?
• Financially reward low performance?

9
Assessment of individuals

• Formative or Diagnostic
• Identify strengths and weaknesses
• Construct strategies for improvement
• Summative or Evaluative
• Grades
• Graduation Requirement
• 10th Grade WASL in 2008 (5th graders now)

10
Summative Eval Question

• Should a PhD historian be able to pass a state
  physics test without ever taking a physics
  class?
• Bernie Khoury,
  AAPT
• Should a state physics test be physics content
  free?
• Should a PhD historian be allowed to graduate
  from high school?

11
Assessment as Teaching Tool

• Instructive or reflective assessment
• Students study the results of their own
  assessment throughout learning
• White and Frederiksen, 2000
• Inquiry reflective assessment produced better
  results than Inquiry discussion
• Differences were greatest for weakest students
  and hardest material

12
Design of assessment tools

• Curriculum assessment probes for student
  averaged content weaknesses
• Systems assessment probes student and content
  averaged weaknesses
• Individual summative tests probe for content
  averaged weakness
• Individual formative tests probe for specific
  content and student weakness

13
High-Stakes Assessment

• NCLB mandated assessments often try to do all
  four things at once.
• Natl Assessment of Educ. Progress (NAEP) is
  high-bandwidth, low-fidelity.
• Measurement at the level of individual students
  is poor. NRC, Knowing What Students Know,
  2001

14
The art and science of testing

• THE ASSESSMENT TRIANGLE (KWSK, 2001)

15
The art and science of testing

• Psychometrics
• For statistical analysis, correct responses (cr)
  must outnumber random chance (rp)
• Usual requirement cr 2 rp
• A,B,C,D choice, cr 50
• A,B,C,D,E choice, cr 40
• Open ended Average 40 to 50

16
The art and science of testing

• Compromises Can we get 40 correct?
• Q Throw a ball straight up into the air and it
  comes back down to your hand. Define up to be
  the positive direction. When the ball is at
  the maximum height, is has an acceleration of
• A) 9.8 m/s2 D) -4.9 m/s2
• B) -9.8 m/s2 E) zero
• C) 4.9 m/s2
• Most students choose E, so psychometricians
  suggest removing it from the list of options.

17
Can we get 40 correct, if

• We frame the same question but ask
• 1) at maximum height, is has a velocity of
• A) 9.8 m/s D) -4.9 m/s
• B) -9.8 m/s E) zero
• C) 4.9 m/s
• 2) at max. height, is has an acceleration of
• A) 9.8 m/s2 D) -4.9 m/s2
• B) -9.8 m/s2 E) zero
• C) 4.9 m/s2
• 40 of students get 2 right if asked 1 first.

18
The state of Washington

• EALR, WASL, SCIF, SALT, and OIAs
• EALR Ess. Acad. Learning Requirement
• WASL Wa. Assesst of Student Learning
• SCIF Science Curr. Instr. Frameworks (they write
  the EALRs)
• SALT Sci. Assesst Leadership Team (They write
  the WASL)
• OIAs Other Important Acronyms

19
The state of Washington

• SCIF and SALT talk to each other (this is
  unusual)
• Aligned on content and level (Bloom)
• From SCIF EALR and Natl Standards alignment
  document in progress
• From SALT content specialists may override
  psychometricians

20
The state of Washington

• K8 Science according to SCIF and SALT

INQUIRY
30
Ecosystems, Circuits, Rock cycle, Nerves
Rockets, Solar system, Spiders
SYSTEMS
40
30
DESIGN
21
Do tests help education reform?

• Whereas teaching directly to the items on a test
  is not desirable, teaching to the theory of
  cognition and learning that underlies a test can
  provide a positive direction for instruction.
• NRC, Knowing What Students Know, 2001

22
Do tests help education reform?

• Large-scale standardized assessments can
  communicate across time and place, but by so
  constraining the content and timeliness of the
  message that they often have limited utility in
  the classroom.
• NRC, Knowing What Students Know, 2001

23
Do tests help education reform?

• The Frame and the Tapestry
• Thompson and Zeuli, 1999
• Michigan Educ. Assessmt Prog. (MEAP)
• MEAP has both goals and assessments
• Most local school districts aligned their
  curricula with MEAP and NSES

24
Do tests help education reform?

• considering how deep-seated most teachers
  ideas are about subject matter, teaching, and
  learning, one would not expect these
  substantially aligned documents to produce
  conceptually transformative teacher learning on a
  broad scale. Our classroom-level research
  confirms that they have not.
• Thompson and Zeuli, 1999

25
Does reform help the tests?

• Education reform for teachers is a necessary
  precondition for success
• It is now widely accepted that in order to
  realize recently proposed reforms in what is
  taught and how it is taught in math and science,
  teachers will have to unlearn much of what they
  believe, know, and know how to do, while also
  forming new beliefs, developing new knowledge,
  and mastering new skills.
• Thompson and Zeuli, 1999

26
Teaching reform requires

• For teachers
• Cognitive dissonance
• Time for reflection
• Connection with their classrooms
• A repertoire of techniques
• Continuing support
• For the system
• A scalable, sustainable process
• Thompson and Zeuli, 1999

27
How long will this take?

• This is a generation-long process.
• Pinky Nelson, WWU Project 2061