Making Formative and Benchmark Assessments Count

1
Welcome!

• Making Formative and Benchmark Assessments Count

WERA Conference December 2007
1
2
Goals for the Afternoon

• Share lessons learned from the ASK Project (an
  assessment design project funded by the National
  Science Foundation).
• Designing assessment systems for science
  curriculum.
• Understanding best practices.

2
3
Purposes

• Classroom assessment practices
• Develop assessment strategies and practices that
  
• lead to better student achievement and enhance
  instructional practices.
• Accountability
• Develop assessments with the technical quality
  needed to provide accountability information to
  districts.

3
4
Advisory Board
BEAR
External Evaluator
Formative Evaluator
LHS
Las Vegas
MS ID
LF SE EV
4
5
Two Reports from NRC
5
6
The Assessment Triangle
OBSERVATION
INTERPRETATION
COGNITION
6
7
The Assessment Triangle
OBSERVATION
INTERPRETATION
COGNITION
7
8
Cognition

• The Cognitive Perspective
• Focuses on how people develop structures of
  knowledgecontent, reasoning, and problem
  solving.
• The Situative Perspective (Sociocultural)
• Focuses on how people learn to participate in the
  practices, goals, and habits of mind of a
  particular community.

8
9
Inquiry Constructs
9
10
Physics of Sound Constructs
10
11
Progress Levels Strategic Conceptual Recogniti
on Notions
11
12
The Assessment Triangle
OBSERVATION
INTERPRETATION
COGNITION
12
13
ObservationItem Development

• Because you cant see directly into a childs
  mind, you have to find ways to infer evidence of
  learning.
• Items are written
• Panels review items
• Researchers look at cognitive demands
• Students and teachers test them out!
• Empirical data informs further revisions

13
14
ASK Assessment Items

• Embedded Assessment items (diagnostic and part of
  daily instruction)
• Notebook entries
• Notebook sheets
• Response sheets
• Benchmark Assessment items (evaluative, but also
  formativegiven periodically)
• Short Answer
• Open Response
• Multiple Choice/Multiple Answer

14
15
Benchmark Assessments

• Before starting
• After Investigation 1
• After Investigation 2
• After Investigation 3
• After Investigation 4
• After completing the module

• Survey
• Inv 1 I-Check
• Inv 2 I-Check
• Inv 3 I-Check
• Inv 4 I-check
• Posttest

15
16
The Assessment Triangle
OBSERVATION
INTERPRETATION
COGNITION
16
17
Coding (scoring) guides
Based on Progress Levels 4 Strategic 3 Conceptua
l 2 Recognition 1 Notions 0 Makes no attempt
(Coding guides are 0-2, 0-3 or 0-4 depending on
the level of the question.)
17
18
Interpretation

• Qualitative Methods
• Teacher feedback, expert panels and cognitive
  analysis.
• Quantitative Methods
• Item Response Theory (statistical analysis)
• ClassMap (computer program/reports)

18
19
The Assessment Triangle
OBSERVATION (Item development)
INTERPRETATION (Coding guides and measurement
model)
COGNITION (Construct Maps Progress Maps)
19
20
Focus on Embedded Assessment

• How is embedded assessment enacted within a
  curriculum?
• Context Physics of Sound

20
21
What students have learned before the lesson
well be doing

• Sound is caused by vibrations.
• A sound source is an object that is vibrating.
• A sound receiver detects sound vibrations.
• Sounds have identifiable properties.
• Pitch (a property of sound) is how high or low
  the sound is (volume loud or soft).
• The faster an object vibrates the higher the
  pitch (and vice versa).

21
22
Notes to Myself
Prep for Inv 2

• Review the At-a-Glance chart
• Read Background for the Teacher and Teaching
  Children About
• Read through the steps in Guiding the
  Investigation

• Review the I-Check for Investigation 2
• Review the focus questions

22
23
Notes to Myself
Prep for Inv 2

• The three big ideas
• Sounds are caused by vibrations
• Clarify distinction between pitch and volume
• The length of the instrument affects the pitch
• Shorter length higher pitch
• Longer length lower pitch
• Tension also affects the pitch
• More tension higher pitch
• Less tension lower pitch

23
24
Focus Question

• How does length affect the pitch (and therefore
  the speed of vibrations)?

Record observations Waterphone Xylophone Kalimba
String Beam
24
25
Class Discussion

• How does length affect the pitch (and therefore
  the speed of vibrations)?

Record observations Waterphone Xylophone Kalimba
String Beam
25
26
Notes to Myself
Length/Pitch
Key v Got it! rev reversed ? confused da
didnt answer the question na no answer
Gen
Cala
Derek
Halyn
Badu
Ada
Edita
Flavo
26
27
Notes to Myself
Length/Pitch

• Can students distinguish high from low pitches?
• Do they know the rule?
• Why are they inconsistent?

27
28
Next Steps

• (Can students discriminate pitch?)

Activity Close your eyes. Listen to the
sounds. If the sound has a high pitch, show
thumbs up. If the sound has a low pitch, show
thumbs down.
28
29
What are our rules?

• The longer the length,
• the ___________ the pitch,

and the ___________ the vibrations.
29
30
What are our rules?

• The shorter the length,
• the ___________ the pitch,

and the ___________ the vibrations.
30
31
Next Steps

• The longer the length, the lower the pitch, the
  slower the vibrations.
• The shorter the length, the higher the pitch, the
  faster the vibrations.

31
32
32
33
33
34
Next Steps

• The longer the length, the lower the pitch, the
  faster the vibrations.
• The shorter the length, the higher the pitch, the
  slower the vibrations.

34
35
35
36
Next Steps

• What do the dots tell you about your use of the
  rule?
• Rewrite any sentences you marked with a red or
  yellow dot.

36
37
37
38
Benchmark Assessments
Purposes Summative (measure for giving
grades) Formative (continue the
learning) Procedures Take the test Code, but
dont mark on papers Use self-assessment
strategies with students before revealing codes.
38
39
Students have Learned

• Sound is caused by vibrating objects.
• Length affects pitch
• Longer lower pitch (slower vibrations)
• Shorter higher pitch (faster vibrations)
• Tension affects pitch
• Looser lower pitch (slower vibrations)
• Tighter higher pitch (faster vibrations)

39
40
Benchmark Assessment

• Take the I-Check for Investigation 2

40
41
Behind the scenes

• After students take the benchmark, the
  teacher codes the items to determine what
  students know and what they still need help with.
• The teacher then plans which items to use to help
  students self-assess their understanding.
• (well come back to coding later)

41
42
Self-assessment Strategy Multiple-Choice Corners

• Look at item 13
• You will be assigned a letter A, B, or C
• (In the classroom students would go to the corner
  that corresponds with their actual answer.)
• Meet with your group to create an argument to
  convince the other groups in the room that your
  answer is the correct one.
• If at some point you disagree with yourself,
  you are allowed to change corners.

42
43
Self-assessment Strategy Sentence Starters

• I used to think.but now I think
• I should have gotten this one right, I just
• I knowbut Im still not sure about
• The most important thing to remember is
• Can you help me with
• Next time I will remember to
• Im confused about
• Now I know

43
44
Sentence StartersAndy
44
45
Sentence StartersAndy
45
46
Sentence StartersRachael
46
47
Sentence StartersRachael
47
48
Coding Benchmark Items
44
Note Coding Guides may be 02, 03 or 04
48
49
Levels of Progress

• 4 Strategic
• 3 Conceptual
• 2 Recognition
• 1 Notions
• 0 No attempt

49
50
Code Inv 2 I-Check

• Review the coding guide for first item.
• Code responses for all students on that item
• Go on to the next item

50
51
51
52
The Development Process

• Construct Maps and Frameworks
• Item Development
• Field Testing
• Analysis using item response theory software
  (ConstructMap)
• Item fit
• Standard setting
• Iterative revision of items
• Progress Levels and Maps

52
53
Does formative assessment matter?

• Benchmark scores were not as high as we thought
  they should be.
• When we interviewed teachers they said they
  looked over students shoulders when they
  wrote in notebooks, but did not critically
  look at embedded assessment evidence.
• Through FAST Project, worked with a small
  group of teachers to force looking at
  embedded assessments.

53
54
Magnetism and Electricity
Strategic Conceptual Recognition Notions
Posttest Results
54
55
ClassMap
Berkeley Evaluation and Assessment Research Center
55
56
Beliefs and Actions
56
57
Beliefs and Actions
57
58
Beliefs and Actions
58
59
Beliefs and Actions
59
60
Discussion

• Questions?
• Comments?

60