Using Scoring Guides

1
Using Scoring Guides

• Current perspectives on classroom assessment and
  student achievement
• L.B. Uveges
• November 27, 2007

2
Research

• Research consistently shows that regular,
  high-quality CLASSROOM ASSESSMENT increases
  student achievement

3

• The key to improvement is how students and
  teachers USE assessment information

4

• Weighing the pig, doesnt mean hell gain
  weight

5

• Assessment tools that calculate solely how well
  student achievement measures up to the standards,
  however reliable, will not suffice. Assessment
  must serve as a vehicle for improving the quality
  of learning for every student.
• - National Research Council, 2001

6
Crunching the numbers

• Grades based on averaging have meaning only when
  averaging repeated measures of similar content.
  Teachers average marks on fractions, word
  problems, geometry, and addition with marks for
  attendance, homework, and notebooks- and call it
  mathematics. In mathematics, we teach that you
  cannot average apples, oranges and bananas, but
  we do it in our gradebooks! (Ken OConnor)

7

• Ideas about assessment have undergone important
  changes in recent years. In the new view,
  assessment and learning are 2 sides of the same
  coinwhen students engage in assessments, they
  should learn from those assessments.
• - National Research Council, 1999

8
Formative Assessment

• All those activities undertaken by teachers and
  by their students (that) provide information to
  be used as feedback to modify the teaching and
  learning activities in which they are engaged.
• - Black Wiliam, 1998

9
Communication

• We must constantly remind ourselves that the
  ultimate purpose of education is to have students
  become self-evaluating. If students graduate
  from our schools still dependent on others to
  tell them when they are adequate, good, or
  excellent, then we have missed the whole point of
  what education is about
• (Costa and Kallick 1992)

10

• Grades are merely symbols in order to provide
  real information, they should be seen as only a
  part- probably a very small part- of our
  communication system
• (Ken OConnor)

11
Formative Assessment 3 Guiding Questions

• Where are you trying to go?
• Where are you now?
• How can you get there?

12

• The QUALITY of the feedback rather than its
  existence or absence is what determines its power
• - Bangert-Dewns, Kulik, Kulik Morgan, 1991
  Sadler, 1989

13

• With regard to feedback, research makes the case
  for the use of DESCRIPTIVE, CRITERION-BASED
  feedback as opposed to numerical scoring or
  letter grades without clear criteria.
• -Butler Neuman, 1995

14

• Research shows that feedback that EMPHASIZES
  LEARNING GOALS leads to greater learning gains
  than feedback that emphasizes self-esteem
• -Ames, 1992 Butler, 1998

15

• When receiving feedback emphasizing self-esteem,
  high-performing students often attribute their
  performance to effort and low-performing students
  attribute their performance to lack of ability.

16

• Student SELF-assessment is crucial for feedback
  to be used effectively. Students are the ones
  who must ultimately take action to bridge the gap
  between where they are and where they are heading
• - Sadler, 1989

17

• Effective learners operate best when they have
  insight into their own strengths and weaknesses
  and access to their own repertoires of strategies
  for learning.
• -Brown, 1994

18

• In a year-long teacher-researcher collaborative
  project, Rudd and Gladstone (1993) helped foster
  self-assessment skills through questionnaires,
  concept maps, and self-assessment maps. They
  report the following results

19

• Development of students abilities to plan and
  think through their goals and skills
• Creation of student awareness of the importance
  of evaluating their own work
• Development of students abilities to evaluate
  each others self-assessment and provide
  constructive criticism
• Increase in students abilities to manage
  resources and time more effectively.

20

• The process of engaging in self-assessment
  increases students COMMITMENT to achieving
  important goals.
• - Covington, 1992

21
SEVEN strategies for using a scoring guide as a a
teaching tool

• 1- Teach students the language of quality- the
  concepts behind strong performance.
• How do your students already describe what a
  strong product or performance looks like? How
  does their prior knowledge relate to the elements
  of the scoring guide you will use? How can you
  get them to refine their vision of quality

22
What you can do

• ASK students to brainstorm characteristics of
  good quality work
• Show samples of work (low and high quality) and
  ask them to expand their list of quality
• Ask students if theyd like to see what teachers
  think.
• Have them analyze how student-friendly versions
  of the scoring guide match to what they said

23
2- Read (view) score, and discuss anonymous
sample products or performances

• Some strong and some weak some representing
  problems they commonly experience, especially the
  problems that drive you nuts!
• Ask students to use the rubric to score real
  samples of student work. Since there is no
  single correct score, only justifiable scores,
  ask students to justify their scores using
  wording from the rubric. Begin with a single
  trait.
• Progress to multiple traits when students are
  proficient with single-trait scoring

24
3 Let students use the scoring guide to practice
and rehearse revising

• Its not enough to merely ask students to judge
  work and justify their judgments. Students also
  need to understand how to revise work to make it
  better. Begin by choosing work that needs
  revision on a signle trait

25
What to do

• Ask students to brainstorm advice for the author
  on how to improve his or her work. Then ask
  students to revise the work using their own
  advice (in pairs)
• Ask students to write a letter to the creator of
  the sample, suggesting what she/he could do to
  make the sample strong for the trait discussed
• Ask students to work on a product or performance
  of their own that is currently in process,
  revising for the trait discussed

26
4 Share examples of products or performances
from life beyond school- both strong and weak

• Have them analyze these samples for quality using
  the scoring guide

27
5- Model creating the product or performance
yourself

• Show the messy underside- the true beginnings-
  how you think through decisions along the way.
• Ask students to analyze YOUR work for quality and
  make suggestions on improvement.
• Revise your work using their advice
• Ask them to again review it for quality

28
6 Encourage students to share what they know

• People consolidate understanding when they
  practice describing and articulating criteria for
  quality

29
Ask them to

• Write self-reflections, letters to parents,
  papers describing the process they went through
  to create a product or performance. Use the
  language of the scoring guide
• Revise the scoring guide for younger students,
  make bulleted lists of elements of quality,
  develop posters illustrating the traits, or write
  a description of quality as they now understand
  it I used to but now I)
• Participate in conferences with parents and/or
  teachers to share their achievement.

30
7 Design lessons and activities around the
traits of the scoring guide

• Reorganize what you already teach and find or
  design additional lessons

31
Rubrics
32
Assessment Rubric

• TARGET- focused on target (PIs)
• METHOD- Does it match the target?
• CRITERIA- Is the purpose / goal clear and
  specific?
• TIMEFRAME- Does it meet time allotted/allowed?
• SAMPLING- Are there enough questions to get a
  valid idea of students achievement?
• VALIDITY- Does it measure what I want to measure?
• BIAS/DISTORTION- Are there any elements of either
  bias of distortion?

33
Data Skills Self Assessment

• Review the self-checklist
• Rank from 0-5 (low-high)

34
PDSA/COAU??

• Plan-Do-Study-Act
• Collect-Organize-Analyze-Use
• Collect the data needed
• Organize it (notebooks, excel, content areas)
• Analyze (data tables, charts, graphs, trangulate,
  disaggregate)
• Use- make decisions about learning- set criteria
  for measurement, improve student achievement

35
How do we get students to take responsibility?

• Student involved classroom assessment
• Student-involved record keeping
• Student involved communication (portfolios,
  conferences)

36
But what about OAT?

• Tests often are so broad in their coverage that
  the info is too imprecise for teahers to use
• The delay between testing and score reporting is
  weeks or months away by the time the scores
  return, they no longer reflect the achievement of
  the learners
• Tests are usually administered once a year while
  teachers make instructional changes daily!

37
Learning Teams

• Critique your assessments
• Assist in moving the assessment project toward
  useful, successful goals
• Follow a preplanned agenda that is management and
  applicable to your work

38
Tools

• Goal setting-
• How to teach the students how to goal set
• Examples of goals
• Collaborative goal setting
• Goal setting plan
• Long term planning
• DATA folders

39
Setting/Environment

• Ground rules
• Interviews/introductions
• Capturing expectations
• Issue Bin (Penalty Box)
• Plus/Delta
• Mission/Vision statements

40
Tools for what they know

• Brainstorming
• Affinity Diagram
• Evaluation criteria
• Light-voting
• Force-field analysis

41
Tools for thinking process

• Flowchart
• Interrelationship diagram
• Fishbone diagram
• Tree diagram
• Action planning

42
Tools for gathering data

• Check sheets
• Surveys (surveymonkey.com)
• Pareto chart
• Matrix
• Scatter diagram
• Run chart