What do we know about classroom assessment

1
What do we know about classroom
assessment? CAESL2005 Thurs, May 19, 2005
2
Purpose
To have an evidence-centered conversation about
the challenges of classroom assessment
Desired Outcomes

• Better understanding of the challenges from
• multiple perspectives
• Recommendations for future directions
• and strategies

3
Session Overview

• Introduction
• Research presentation
• Implications of the findings
• Discuss implications in professional groups
• 10-minute break
• Feedback from mixed groups
• Posters in professional groups

4
Introduction
5
Context CAESL Assessment Model

• Mission -
• strengthen K-12 science assessment at all levels
  of the system
• Requires -
• collaboration of different professionals
• integration of psychometric and practitioner
  conceptions of assessment

5
6
(No Transcript)
7
Classroom Assessment
8

• Base of the system
• Targeted goals for student learning
• Derived from research, curriculum, standards
• Common across district, state or national levels.

9

• Base of the system
• Quality Assessment Tools
• Aligned with goals
• Appropriate to purpose
• Multiple measures are used

10

• Base of the system
• Quality Use
• Teachers use quality assessments to
• Interpret student understanding
• Communicate expectations and provide feedback
• Take action to improve learning
• Promote student accountability
• Assure equity

11
Research Presentation

• Two studies
• CAESL/FAST study
• CAESL Assessment Leadership Academy

12
Research context 1 CAESL/FAST study
13
UCLA Joan Herman Sam Nagashima Ellen Osmundson
WestEd Steve Schneider Mike Timms Jessica Partch
U.C. Berkeley Mark Wilson Cathy Kennedy Karen
Draney Nathaniel Brown Lydia Liu Diana
Bernbaum Xioahui Zheng
Stanford Rich Shavelson Susan Schultz Melody
Yin Ayita Ruiz-Primo Carlos Ayala (Sonoma State
U.)
14
CAESL/FAST Study Design

• Study context FAST curriculum on Why Things
  Sink and Float
• Developmental perspective of the curriculum
  (learning trajectory)
• Reflective Lessons embedded assessment in
  instruction
• Progress guides represent key learning goals
  used to align Reflective Lessons with instruction
  and learning goals.

15
Reflective Lessons are Formative Assessments

• Embedded in a unit of study and aligned with the
  
• curriculum
• Provide teachers and students with opportunities
  for
• reflection on learning
• Teacher, peer, and self assessments comprise
• classroom assessment
• Used to determine gap between what a student
  knows
• and knowledge goal
• Informs curriculum implementation

Source Black Wiliam, Inside the black box,
1998
16
Tracked two learning goals One example Why
things sink and float
17
/CRESST
Research Questions

• How can core types of science knowledge be used
  in
• formative assessment systems?
• How can progress variables help to link
  classroom
• assessment with learning goals?
• How do teachers use formative assessment?
• What is the impact on students learning?
• What are implications for research and practice?

18
/CRESST
Methodology

• Original sample 13 teachers were trained
• A range of implementation data
• Observations
• Interviews
• Teacher logs
• Results on reflective lessons
• Pre-Post Measures on Progress Variables
• Qualitative and quantitative methodologies

19
Research Context 2 CAESL Science Assessment
Leadership Academy
20
(No Transcript)
21
Intended contributions

• framework for classroom assessment expertise
• understanding how assessment expertise develops

22
FRAMEWORK

• Knowledge of assessment concepts

• Facility with assessment practices

23
KNOWLEDGE OF ASSESSMENT CONCEPTS
QUALITY GOALS FOR STUDENT LEARNING AND PROGRESS
QUALITY USE
QUALITY TOOLS
23
24
KNOWLEDGE OF ASSESSMENT CONCEPTS
QUALITY GOALS FOR STUDENT LEARNING AND PROGRESS

• Sound Science Content
• Developmentally Appropriate
• Coherent Coordinated

• Feasible in Scope and Implementation
• Clearly Specified

Fair Equitable
QUALITY USE
QUALITY TOOLS

• Guide for Instructional Improvement
• Feedback to Students
• Resource for Assessment Improvement

• Appropriate to Purpose
• Captures the Full Range of Student Understanding
• Clear Expectations
• Efficient and feasible

SOUND INTERPRETATION
Appropriateness Consistency Accuracy
24
25
INTEGRATED ASSESSMENT PRACTICES
ESTABLISH GOALS PLAN INSTRUCTION
ASSESSMENT
REVISE ASSESSMENTS
PROVIDE INSTRUCTION
USE INFORMATION TO GUIDE TEACHING LEARNING
ASSESS
INTERPRET STUDENT WORK
25
26
Using assessment concepts to guide practice
ESTABLISH GOALS PLAN INSTRUCTION
ASSESSMENT

• Are my goals sound?
• - science sound?
• appropriate for my students?
• clearly specified?

REVISE ASSESSMENTS
PROVIDE INSTRUCTION
Is my assessment sound? -tasks and criteria
aligned with goals? -provide evidence of
students understandings?
USE INFORMATION TO GUIDE TEACHING LEARNING
ASSESS

• Is my interpretation sound?
• criteria appropriate for purpose?
• - scoring consistent?
• - interpretation fair unbiased?

INTERPRET STUDENT WORK
26
27
How does assessment expertise develop?Science
Assessment Leadership Academy
28
PD Design

• Principles integrated with practice
• long term
• collaborative
• reflective practice
• Core strategy assessment portfolio

Repeated opportunities to build expertise within
and across portfolios
29
Program

• Assessment portfolio
• Section I Plan
• Section II Interpretation of student work
• Section III Assessment revisions
• Organization
• cross district grade level teams
• independent implementation

29
30
Research Design

• Experienced K-12 science teachers in 5 district
  teams
• Longitudinal, nested design
• Surveys and portfolios from all teachers
• Interviews observations with case teachers

30
31
Data
32
What we found out
Designing and implementing quality assessment in
the classroom is very challenging! The CAESL
model is useful for conceptualizing research and
linking research and practice
33
Findings about Targeted Goals for Student
Learning

• Clear and sound goals are very important
• When planning assessments or interpreting
  student work, teachers discovered that they first
  needed to evaluate and clarify the learning goals
  to be assessed.

34
Findings about Quality Tools

• It is hard to do good assessment with weak
  assessments, no matter how sound the curriculum
  materials.
• Teachers worked for hours (days!) developing just
  a few assessments for their units. They would
  like materials to provide them sound assessments
  that they can customize if needed.
• Teachers want the materials to guide them with
  interpretation of their students thinking.

35
Findings about Quality Tools

• Teaching and learning benefit from rubrics that
  capture student understanding, but materials
  rarely provide them.
• Teachers were frustrated with rubrics of the got
  it didnt get it variety. They would like
  criteria that capture the content of their
  students thinking.
• Teachers found developmental rubrics very helpful
  in focusing their teaching and assessment.

36
Findings about Quality Use

• Assessment feedback to students and teachers is
  beneficial, BUT scoring work to produce this
  feedback can be overwhelming.
• Many teachers completed their portfolios after
  implementing their units.
• Many teachers found the scoring system in the
  CAESL/FAST study too cumbersome to fully
  implement.

37
Findings about Quality Use

• Strong content knowledge and pedagogical content
  knowledge help in implementing science classroom
  assessments.
• Teachers were intrigued by their students
  thinking but challenged in their attempts to
  interpret it. They requested resources on how
  conceptual understanding develops in a topic
  area.

38
Findings about Quality Use

• Teachers varied in their implementation of
  formative assessment during instruction
• nature of questioning
• elicitation of student conceptions
• students engagement during small group work
• provision of feedback
• success in deeply engaging most students with
  content

39
Findings about Quality Use

• Assessment results were used but not as much as
  we expected.
• Assessment results were used to make minor
  modifications.
• Relatively rare use of results to re-teach

40
Findings about Targeted Goals for Student
Learning

• Assessment results showed that students mastery
  of concepts was generally behind where they were
  expected to be in the planned instructional
  sequence
• See the next slide for a diagram of what we
  found.

41
Comparison of results to plan Reflective Lesson
at 7--students should have mastered mass and
volume
Expected students to be here
Most students are here
42
Findings about Alignment

• Teachers discovered how critical it is to align
  learning goals, curriculum, instruction, and
  assessment.
• But many instructional units made it challenging
  for teachers to align assessments. Teachers
  wanted sound embedded assessment systems.
• Even with alignment, there were significant
  logistical challenges

43
Findings and Next Steps

• Researchers continue to probe
• The nature of quality assessment and the multiple
  capacities it requires
• The effects of quality assessment on learning

44

• Now
• Discussion in profession-alike groups
• Feedback in mixed groups
• Conclude with profession posters

45

• Guiding Questions
• Teachers What are feasible strategies for
  implementing quality assessment in the classroom?
  
• PD What should PD look like that prepares
  teachers for quality classroom assessment?
• Administrators What organizational structures
  need to be in place to help support quality
  classroom assessment?
• Researchers What further research is needed to
  inform our knowledge of developing and
  implementing quality classroom assessment?
• Curriculum Developers What kinds of assessment
  resources need to be included in curricula to
  support implementation of quality classroom
  assessment?