Classroom Based Evidence

1
Classroom Based Evidence

• November 29, 2005
• Lexie Domaradzki

2
What do you know?

• Tell the person to your right everything you know
  or have experienced regarding the CBEs.
• I dont know anything is an okay answer.

3
Foundational information

• Objectives for the day
• 1. Become familiar with the content, format, and
  rubrics of the CBEs
• 2. Identify difficult patterns of implementing
  the CBEs in the reading block
• 3. Identify strategies to support a building in
  in the implementation of the CBEs
• 4. Build an understanding of why this is a pilot
  year for this project

4
Where do the CBEs fit in the Reading system?

• Why they were created.
• How they were created
• Where the fit in the total picture of the Reading
  System.

5
Why were they created?

• 1. To align our weekly practice with our outcome
  measure of the 3rd grade WASL
• 2. To ensure that students have regular practice
  with the cognitive demand that is expected in our
  state
• 3. To give teachers a picture of how students
  will likely perform on the 3rd grade WASL

6
How were they created?

• 1. An alignment expert was hired to lead the
  development team (Michelle Lewis)
• 2. The team began with several days of training
  and then began by identifying state Learning
  Targets that are measured

7
How they were created.(contd)

• 3. The team worked from an Item Development
  checklist (in handouts)
• 4. Team identified learning target for each
  story, wrote questions for 2 pt. and 4 pt.
  responses.
• 5. The team wrote a task specific rubric for
  each questions and gave examples for each point
  possible

8
How they were created.(contd)

• 6. Each program had 2-4 writers identifying
  learning targets, developing the questions and
  the rubrics.
• 7. Each program had 2-4 reviewers for accuracy,
  alignment with rubric, and format

9
Learning Targets

• Writing team worked hard to be sure that all
  learning targets were covered.
• Writing team worked hard to be sure the story
  could support the learning target.
• In some core programs, some of our state learning
  targets are never taught.

10
How were the CBEs distributed?

• Each reading program burned onto a cd
• One cd mailed to Reading Coach at each building
• Teacher materials and student materials are all
  on one cd.
• All but two programs have both Multiple choice
  and short answer questions on test
• Harcourt Trophies and Collections were both done
• Open Court 2000 and 2002 are on one cd.

11
Things that we learned.
12
Lets take a look at the CBEs

• Teachers materials
• Learning target, item (question), rubric
  (examples
• Format, content difficulty
• Student materials
• Item (question) and response area
• Format, content difficulty

13
A further look.

• Compare/contrast WASL sample tests and our pilot
  CBEs
• Complete Venn diagram as a team for Sample
  tests/CBEs
• WASL sample tests in left oval, CBEs in right
  oval, similarities in middle

14
Points of interest

• Third grade WASL
• No 4 point responses
• All written responses scored on a 0-2 point
  rubric
• 0-2 point rubric similar to 4th grade WASL in
  expectation of text based evidence
• Multiple choice answers worth 1 point

15
Vocabulary

• List words found in the questions of the CBEs
  and the WASL sample assessment that may be
  unfamiliar to your students.
• 1. Post the words in the classroom
• 2. Directly teach and discuss vocab words
• 3. Show examples and non examples of answers
  utilizing the vocabulary

16
Implementing the CBEs

• Very similar process that many of you began when
  the 4th grade WASL was implemented.
• This process takes a very strong leader who holds
  firm on expectations and yet understands that
  this increased expectation will create
  dissonance.

17
4th grade WASL

• When the 4th grade WASL began, what type of
  difficulties do your teachers and students have?
• What has changed sine the beginning?
• How did the transformation happen?

18
Lettermans Top Ten List

• Letterman Top Ten list of comments from the
  Teachers Lounge about CBEs

19
Number 10

• How am I supposed to translate this test score
  into a letter grade?
• 8/8A
• 7/8B
• 6/8C
• 5/8D
• Eight points possible
• Points earned divided by points possible
• Example 6/8 points 75

20
Number 10

• More importantly..
• How are my students doing with multiple choice?
• How are my students doing with short answer?
• How likely are my students going to be able to
  hit the target in the spring?
• What do I need to do to assist my students with
  increased performance?

21
Number 9

• My students are failing all of their weekly
  tests. Am I really supposed to fail all my
  students?
• Currently some core program say take any and all
  answers for story tests.
• Strategies for increasing performance.

22
Strategies for increasing performance

• 1. Model 1 short answer question, have students
  do one on own.
• 2. Do not score test for several weeks until
  they are familiar with the format and
  expectations.
• 3. Give students a chance to improve a response
  after you model a good answer.

23
Strategies for increasing performance

• 4. Be sure that students use their books during
  the test.
• 5. Be sure if it is a text to text question,
  that your students have read both pieces of text.
  
• 6. Understand that this is an opportunity for
  them to experience, practice and become
  proficient at handling an outcome measure that is
  to come in the spring.

24
Number 8

• The wording of these questions are not
  developmentally appropriate for 3rd graders.
• After comparing the WASL sample test
• How could you use the sample test to help you
  with this discussion in your building?

25
Number 7

• I showed this test to another 3rd grade teacher
  and he gave it a completely different score.
• Inter-rater reliability
• Practice scoring sessions with grade level
• 4th grade teacher helping with reliability
• 3rd grade anchor papers

26
Number 6

• This students answer really wasnt good enough
  for a 2, but I thought they deserved more than a
  1 so I gave them a 1.5
• That score is out of alignment with the WASL
  expectations
• The better question is what would it have taken
  for that student to score a 2? How can I give
  that student feedback so that they could hit the
  target next time?

27
Number 5

• You cant expect a 3rd grade to extend
  information beyond the text. That is a higher
  level skill.

28
Number 4

• I cant give this student full credit. They
  didnt write in complete sentences and misspelled
  the title of the story
• 1. Examine the rubrics in the sample tests.
• No mechanics are measured on the Reading
  Assessment
• For the Reading Assessment, scoring must be
  aligned

29
Number 3

• Open book tests do not measure students
  comprehension.
• 1. Students need to be able to scan the text,
  not completely re-read it. If they comprehend
  the big ideas of the text, they can scan more
  quickly.
• 2. Most assessments in our life give
  opportunities to re-visit the text.

30
Number 2

• We should not be teaching to the test.
• Format, cognitive demand, and aligned scoring
• When students are familiar with these things,
  they actually spend their testing time activating
  comprehension, rather than trying to decipher
  what will be expected of them.

31
Number 1

• Thank goodness we gave our students some
  experience with the CBEs. The opportunities
  they had to practice with format, cognitive
  demand, and scoring alignment really paid off.

32
Ways to introduce CBEs to your staff

• 1. Run them off and put them in the teachers
  boxes (not, not, not recommended)
• 2. Explain the purpose behind them, how they
  were created and the building plan for
  implementing them.
• 3. Run off one test and compare/contrast them in
  a grade level meeting with WASL as we did here
• 4. Find one or two teachers willing to test them
  out

33
Ways to introduce contd.

• 5. Have teachers work with the CBEs for 4-6
  weeks and collect students papers.
• 6. Teachers bring papers to grade level
  meetings to compare and see how students are
  doing.
• 7. Be sure to utilize the feedback form with
  teachers so that we receive information on how to
  improve the CBEs next spring.
• 8. Send a 3rd grade teacher to training on Jan.
  31 training for further work on data collection,
  data analysis and instruction implications of
  student papers.

34
Panel discussion

• Hearing from peers who have begun to implement
  the CBEs.

35
Important considerations

• CBEs are a support and a supplement to your
  third grade work.
• If you are at risk for meeting the 3rd grade
  Continuation Criteria, CBEs probably will not be
  an area of focus for your teachers.
• Excellent leaders know where to place priorities.
  CBEs may not be the highest priority in your
  building.
• CBEs were provided for you as a tool of support,
  not a mandate or implementation expectation at
  this time.
• This is a pilot year. We are asking the CBEs
  NOT distributed in your districts as there is
  necessary testing and reliability work that needs
  to be done with them. This is not available
  outside of Reading First at the time.

36
Planning together

• If you have already begun implementing the CBEs,
  how did you set it up and what have you done? How
  did it work?
• If you are just ready to begin, what will your
  plan be?
• If you will not be placing CBEs on your priority
  radar, share the reason behind your decision.

37
Sharing with others

• Line discussion
• Already implemented
• Planning to implement
• Not a priority for this year

38
Future Professional Development Opportunities

• Jan 31, 2006
• 3 participants per school, coach, principal and 1
  teacher
• Session taught by Lexie
• Data collection
• Data analysis
• Adjusting instruction based on student work
• Summer Institute
• Session taught by Michelle Lewis
• Data collection
• Data analysis
• Adjusting instruction based on student work

39
Foundational information

• What do you understand now regarding
• Objectives for the day
• 1. Become familiar with the content, format, and
  rubrics of the CBEs
• 2. Identify difficult patterns of implementing
  the CBEs in the reading block
• 3. Identify strategies to support a building in
  in the implementation of the CBEs
• 4. Build an understanding of why this is a pilot
  year for this project