Learning Innovations at WestEd

1
Learning Innovations at WestEd

• Local Accountability Professional Development
  SeriesBuilding a Local Accountability System
  with Standards, Assessments, and Standards-Based
  Instruction

2
Overview of Series
Session Objectives
November Session Module 1 Understand the value of a comprehensive local accountability system Learn about essential or power standards Begin to think about and identify power standards Re-write essential standards in student-friendly language Begin creating assessment calendars
February 14-15 Session Module 2 5 Unpacking Standards Creating Reliable and Valid assessments. Unpack essential standards by examining the content/concepts, skills, and level at which to teach and assess (Blooms Taxonomy) Understand elements of reliable and valid tests Begin process of creating informative assessments to assist with instructional programs and practices to support improved Student Learning Objectives
3
Fill out the initial Pre-Assessment

• Do the best that you can
• If you are unable to write a paragraph at this
  time, feel free to write a list of question you
  might have.

4
Seasonal Partners

• When looking for partners
• Find folks you have not worked with in the past
  couple of weeks if possible.

5
Backward Design

• To begin with the end in mind means to start
  with a clear understanding of your destination.
  It means to know where youre going so that you
  better understand where you are now so that the
  steps you take are always in the right
  direction.
• From The Seven Habits of Highly Effective People
  by Stephen R. Covey

6
Assumptions about a Local Accountability System
and Data-Driven Instruction

• Collaboratively building a local accountability
  system enhances school/district culture
• Limiting and unpacking essential standards
  supports the development of formative assessment
  practice, differentiated instruction, and
  improved student performance and understanding

• Timely, targeted assessment data provides
  teachers with the opportunity to inform
  instruction and individualize learning
  opportunities
• Immediate access to and use of data significantly
  influences teacher pedagogy and student
  performance

7
How your work on assessments connects with your
development of SLOs and the development of
related compensations

• Identifying mastery learning objectives is
  essential in creating effective performance
  assessments
• Benchmark (interim) testing is the glue for an
  effective local accountability system

• Formative Assessment practice informs
  differentiated instruction and provides
  opportunities for all students to mastery
  concepts and skills, and perform at higher
  achievement levels

8
Step 1
Benchmark and Formative Assessments to support
student outcomes
Identifying and Using Essential Standards
Student Learning and Achievement
Standards-Based Instruction Using Data
Building Your Assessment System
Instruction
Curriculum
Assessment
Monitoring Progress
9
Benchmark and Formative Assessments to support
student outcomes
Identifying and Using Essential Standards
Student Learning and Achievement
Standards-Based Instruction Using Data
Building Your Assessment System
Step 2
Instruction
Curriculum
Assessment
Monitoring Progress
10
Today and Tomorrows Agenda

• Unpacking Standards the big picture, content
  skills
• Determining Rigor New Blooms Taxonomy and
  Webbs Depth of Knowledge
• Practice Unwrapping Essential Standards
• Elements of developing valid and reliable
  assessments
• Work sessions -
• Unpacking essential (Power) standards
• Writing standards in student friendly language
• Developing essential (Power) standards
  assessments

11
Four Essential Questions

• What do we want our students to know?
• What will it look like when we get there?
• How will we know when they got there?
• What will we do to get them there?

12
Four Essential Questions

• What do we want our students to know?
• What will it look like when we get there?
• How will we know when they got there?
• What will we do to get them there?

13
Local Accountability System

• Six Elements to consider for implementation.

14
Instruction
Formative Assessment
Reporting and Using Results
Standards
Leadership
Professional Development
15
Essential Standards

• Find your Fall partner and read page 35.

Be prepared to share your answers with the
group.
16
Why do we need essential standards?

• If American educators were to adequately cover
  all of the knowledge identified in the current
  set of standards for the core subject areas, it
  might take as much as 22 years of schooling
  (literally!) within the current structure.
• From Awash in a Sea of Standards by Robert J.
  Marzano and John S. Kendall

17
Why do we need essential standards?

• Or to put it another way..
• We would be going to school from
• kindergarten through grade 21!

18
Essential (or Benchmark) Standards

• Subset of state standards
• Prioritized in terms of
• Endurance
• Leverage
• Readiness
• Required for state assessments
• Vertical and horizontal alignment

• Taught on calendar
• Mastery guaranteed
• Formative assessments
• Intervention opportunities
• Not license to eliminate other standards from
  curriculum

19
Essential (or Benchmark) Standards
What makes a standard essential?

• It has endurance. Will this provide students with
  knowledge and skills that will be of value beyond
  a single test date?
• It provides leverage. Will this provide knowledge
  and skills that will be of value in multiple
  disciplines?
• It ensures readiness for the next level of
  learning. Will this provide students with
  essential knowledge and skills that are necessary
  for success in the next grade or next level of
  instruction?
• It is regularly assessed by the state. Is this a
  standard that students have failed to master?

20
Standards Based Curriculum, Instruction, and
Assessment
21
Essential (or Benchmark) Standards
What considerations need to be made to identify
essential standards?

• Consider teaching order
• Re-write in student friendly language
• Consider allotment of time to teach to mastery
• Review previous performance data
• Sequence across grade levels
• Vertical and horizontal alignment

22
Assessment Beliefs (Card activity)

• Count off by (Total number of participants
  divided by 3)
• Work in teams of three
• find the statement(s) that resonate for you
• Try to reach consensus as a group
• Be prepared to report out to the whole group

23
Local Accountability (Return to your seats)

• Read Element 3, Assessments on pages 5-7.
• As you read, underline or highlight key quotes,
  points, or phrases.

23
24
Local Accountability Assessment

• Go around your table, giving each person a
  chance to share a favorite quote or idea.
• Decide upon one quote or idea that youd like to
  share with the whole group. Give a reason why
  that quote resonates for your group.
• Write your quote and reason on the chart paper
  provided.

24
Used with permission from Learning Innovations at
WestEd for Northern Humbolt presentations
25
Self Scoring Guide Pages 22-27
22-27
26
Review pages 21 22 and then as a group at your
table, identify where you would place yourself
on the rubrics on pages 23 25 27 for
Assessments, Un-packed standards.. And Common
Formative Assessments.Be prepared to share your
findings
Where are you now.. ?
26
27
Quick review of student friendly language See
Handouts
28

• Unpacking Standards is critical in the process of
  identifying foundational skills
• After data analysis has helped to define an
  instructional focus and Learning Objectives.
• See Module 5, Page 15

29
Formative and Common Formative Assessment(s) to
pre-assess student understanding, inform and
individualize instruction (inform intervention
support), and determine mastery.
30
Break!
31
Welcome Back!
32

• Module 5 Unpacking Standards
• Building the foundation for instruction guided
  by local assessments

33
Standards Based Curriculum, Instruction, and
Assessment
34
Unpacking Essential or Power Standards
35
Workshop Essential Questions

• How are grade level expectations (standards)
  unpacked? Why do we need to unpack them?

36
Rationale (See page 17 Module 5)
Unpacking grade level expectations will provide a
simple and highly effective way to manage
standards. Unpacked grade level expectations
can help you address the appropriate cognitive
depth for classroom instruction and assessment.
37
Definition
Unpacking standard expectations means to
identify the concepts and skills found in the
state standards. IN OTHER WORDS. . .
38
It means to examine the standards to determine
exactly what students need to

• know (the content/concept)
• be able to do (the skill) considering
• New Blooms (identifying the level of rigor to
  teach students the concepts and skills)
• And
• Webbs Depth of Knowledge (to focus on how deeply
  a student has to know the content in order to
  respond)

39
(No Transcript)
40

What Are Concepts?
Abstract ideas that point to a largerset of
understandings Concepts the important nouns

41

What Are Skills?
Skills the verbs When we unpack skills in a
standard, we are looking for the verbs.
42
Math Concepts Skills
Concepts
Skills

• Linear
• Non-linear
• Patterns
• Models
• Tables
• Sequences
• Graphs
• Problem situations
• Words and symbols
• Expression
• Equation

Identifies Generalizes Writes Extends Using
43
Reading Concepts Skills
Skills
Concepts

• Text Features
• Main Idea
• Supporting details
• Organizing information
• Questions
• Inferences
• Information
• Informational text

Stating Organizing Generating Synthesizing Evaluat
ing Drawing Inferences
44
Guided Practice Unpacking the Standards Content
Skills
45
RLT42.2 Analyze and interpret elements of
literary texts, citing evidence where appropriate
by- describing main characters physical
characteristics or personality traits or
providing examples of thoughts, words, or actions
that reveal characters personality traits

• Work in groups of 2 or 3 and use the graphic
  organizer
• Write the standard above in the box at the top
• Write the nouns in the concepts box
• Write the verbs in the skills box

46
Blooms Original Revised Taxonomy
Taxonomy
Separate dimension

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

• Remembering
• Understanding
• Apply
• Analyze
• Evaluate
• Create

C O G N I T I V E
47
Categories of the Knowledge Dimension

There are 4 categories of knowledge.

Factual Knowledge Conceptual Knowledge
Procedural Knowledge Metacognitive Knowledge
48
What kind of knowledge is it?
49
Blooms Original Revised Taxonomy
Taxonomy
Separate dimension

• Knowledge
• Comprehension
• Application
• Analysis
• Synthesis
• Evaluation

• Remembering
• Understanding
• Apply
• Analyze
• Evaluate
• Create

C O G N I T I V E
50
Categories of the Cognitive Processes

There are 6 categories of the Cognitive
Processes.

Remember Understand
Apply Analyze
Evaluate Create
51
Remember
Remembering means to retrieve relevant
knowledge from long-term memory and is essential
for meaningful learning and problemsolving. 1.
Recognizing 2. Recalling
52
Understand
Understand means to construct meaning from
instructional messages, including oral, written,
and graphic communication. 1. Interpreting
2. Exemplifying3. Classifying 4.
Summarizing 5. Inferring 6.
Comparing 7. Explaining
53
Apply
Apply involves using procedures to perform
exercises or solve problems. 1. Executing 2.
Implementing
54
Analyze
Analyze involves breaking material into its
constituent parts and determining how the parts
are related to one another and to an overall
structure. 1. Differentiating 2.
Organizing 3. Attributing
55
Evaluate
Evaluate is defined as making judgments based
on clearly defined criteria and standards. The
criteria most often used are quality,
effectiveness, efficiency, and consistency. 1
. Checking 2. Critiquing
56
Create
Create involves putting elements together to
form a coherent of functional whole. In other
words, drawing upon elements from many sources
and putting them together into a structure or
pattern relative to ones own prior knowledge.
(It may or may not include originality or
uniqueness.) 1. Generating 2. Planning
3. Producing
57
Guided Practice Unpacking the Standards Blooms
58
Practice determining Blooms

• RLT42.2 Analyze and interpret elements of
  literary texts, citing evidence where
  appropriate by-
• describing main characters physical
  characteristics or personality traits or
  providing examples of thoughts, words, or actions
  that reveal characters personality traits
• Using the standard below, determine the Knowledge
  and Cognitive Process boxes to check on the
  graphic organizer.
• Using the Taxonomy Table on page ___ of the
  handout, determine how you would check boxes for
  this standard.

59
Depth of Knowledge (DOK)

• Norman Webbs Depth of Knowledge Levels (1997,
  2003)
• Descriptive not a taxonomy
• Focuses on how deeply a student has to know the
  content in order to respond
• Interpreting and assigning depth of knowledge
  levels to content standards

Kentucky Department of Education
60
Depth of Knowledge (DOK)
Level 1 - Recall and Reproduction
Level 2 - Skills and Concepts
Level 3 - Strategic Reasoning
Level 4 - Extended Reasoning
61
Level 1 - Recall and Reproduction

• Focus is on specific facts, definitions, details,
  or using routine procedures

Kentucky Department of Education
62
Level 2 - Skills and Concepts

• Focus is on applying skills (in a
  familiar/typical situation) and concepts,
  relationships (compare, cause-effect), main ideas
  requires deeper knowledge than definition
  explaining how or why making decisions
  estimating, interpreting in order to respond one
  right answer

Kentucky Department of Education
63
Level 3 - Strategic Reasoning

• Focus is on need for reasoning and planning in
  order to respond (e.g., write an essay, apply in
  new/novel situation)
• complex and abstract thinking is required
• often need to provide support for reasoning
  or conclusions drawn
• more that one correct response or approach
  is often possible

Kentucky Department of Education
64
Level 4 - Extended Reasoning

• Requires complex reasoning, planning, and
  thinking generally over extended periods of time
  for the investigation or to complete the multiple
  steps of the assessment item.
• Students may be asked to relate concepts
  within the content area and among other content
  areas or to real-world applications in new
  situations.

Kentucky Department of Education
65
Applying Webbs DOK

• Handout - Ceiling Levels of DOK
• Read and highlight key points and phrases for
  each DOK level

66
Find the Webbs DOK

• Handout
• Kentucky Released Items
• We will work in groups of 3 4
• Choose the DOK level for your question and write
  your reason(s) for choosing the level

67
Guided Practice Unpacking the Standards Webbs
DOK
68
RLT42.2 Analyze and interpret elements of
literary texts, citing evidence where appropriate
by- describing main characters physical
characteristics or personality traits or
providing examples of thoughts, words, or actions
that reveal characters personality traits
Check the appropriate box(s) on the Webbs DOK on
the Graphic Organizer.
69
Closure

• Look back at your Unpacked Standards,
• ask yourself
• Could you put away the original
  standard(s)confidently, plan instruction and
  assessment using only your graphic organizer,
  knowing that you have faithfully captured every
  important concept and skill students need to
  learn?
• 2. Would other educators identify the same
  concepts and skills?
• If the answer to both questions is yes,
  congratulations!You have successfully Unpacked
  a Standard!

70
Two Buck Summary
On your white board, please write a two buck
summary for unpacking the standard. Each word is
worth 10 cents.
71
Where to Find the Standards

• Content Standards
• http//www.cde.ca.gov/be/st/ss/
• Curriculum Frameworks
• http//www.cde.ca.gov/be/st/fr/
• Common Core Standards http//www.corestandards.o
  rg/the-standards

72
Turn to page 31
Module 2Alternately read sections of this
article to each other following along while
someone else is reading.Highlight or underline
sections that resonate for you.
Skills for Teachers to Implement Standards-based
Instruction
72
73
Read page 33 on your own.
As you encounter sections you.
Understand, mark with a check, Question.
Mark with a ? Think important Mark with
a When you are done, read page 33 and discuss
with your neighbor.
Developing Benchmark Assessments
73
74
Flag Tag (Sorting activity)

• Find your Spring partners and join one other
  Spring partner pair
• Work in teams of four
• Order the assessment questions from least
  difficult to most difficult
• Try to reach consensus as a group
• Be prepared to defend your decision using your
  understanding of Blooms taxonomy, Webbs DOK and
  other considerations.

75
Why do we do this?

• Two reasons.
• Universal design.
• Matching the appropriate level of assessment
  items to the rigor and cut scores we have
  identified.

76
Performance Levels

• Turn to page 36 and 37
• Box 1 show the achievement distributions
  comparing conventional, mastery and tutorial
  approaches to learning and instruction.
• Box 2 notes the six key characteristics of
  effective assessments.
• Box 3 shows that a comprehensive school program
  is needed to inform practice and program
  decisions leading to students performing at the
  proficient levels.

77
Performance Levels

• The purpose of Benchmark and Formative
  Assessments is
• to provide timely appropriate feedback to inform
  instruction,
• to diagnose and move students up levels, and
• to find out what students have and have not
  learned at a proficient level in time to
  intervene.

78
Performance Levels

• Why do we suggest 4 to 5 performance levels?
• Performance levels become the foundation for
  scoring scales defined by the district.
• The performance levels help to group students
  for targeted interventions for improvement.
• A purpose of assessment becomes the
  identification of student performance to match
  targeted instructional strategies and assist each
  student in mastery of district benchmark and
  state standards.

79
Setting Performance Levels
These have often already been set by the DOE
with ATI Galileo for current benchmark/interim
assessments. How through what process would you
choose different performance levels within the
district?

80
LUNCH

• We will return in 60 minutes
• At..

81
Read pages 10
11Write down any question that come up for you
as you read this story.Cynthia Lee to share her
experience as principal involved in a similar
effort.
Monache High School
81
82
Test Construction - Universal DesignHouse of the
Future, Disneyland 1957-1967Architects,
Richard Hamilton and Marvin Goody
83
Creating Assessments - Guidelines Packet

• Local Accountability Professional Development
  Series
• Learning Innovations at WestEd

84
Taking a test - Franzipanics

• Individually, take the test on page 47, then join
  with a partner if you finish early and discuss
  what you think are the correct answers.
• Hint There is one correct answer for each
  question.
• Why did we take this test?

85
Read pages 51
52Discuss at your table the questions on page
53.We will discuss as a whole group the need
for rubrics when utilizing performance based
assessments.(See page 46)
Developing Quality Classroom Assessments
85
86
We will start off
tomorrow using a rubric to evaluate some writing
samples. (Performance task)You will also be
creating new and/or evaluating and modifying
existing assessments you already use. (Galileo
allows you to do this). Bring standards,
materials and assessments that you would like to
work with.Please complete todays evaluation
before leaving.Thanks!
Framing of Day 2 - End of Day 1
86
87
Day Two
88
Welcome Back!
89
8th Grade English and Writing (performance)
Assessment

• Read a student writing sample and using the 6
  Traits Rubric, score the sample as best you can
  in all areas on the scale of 1 to 5.
• Compare your results with a partner
• If you have time, look at an additional sample
  and go through the same procedure of scoring the
  sample by yourself, and comparing your results to
  your partners findings.
• Be prepared to discuss as a whole group.
• See p. 46 Building Performance Assessments

89
90
(No Transcript)
91
(No Transcript)
92
(No Transcript)
93
(No Transcript)
94
(No Transcript)
95
8th Grade English Assessment

• Now review the assessment as a whole.
• Find your summer partner and discuss changes to
  the assessment that you think would improve on
  its usefulness.
• Be prepared to discuss as a whole group.

95
96
Taking a test - Franzipanics

• Individually, take the test on page 47, then join
  with a partner if you finish early and discuss
  what you think are the correct answers.
• Hint There is one correct answer for each
  question.
• Why did we take this test?

97
Read pages 51 Skim
52As you read, underline or highlight key
quotes, points, or phrases. Discuss at your
table the questions on page 53.
Developing Quality Classroom Assessments
97
98
Local Accountability (Return to your seats)

• Read Element 3, Assessments on pages 5-7.
• As you read, underline or highlight key quotes,
  points, or phrases.

98
99
Formative Assessment

• Read the CCSSO definition of Formative
  Assessment
• As you read, underline or highlight key points,
  or phrases that resonate for you or that raise
  questions in your mind.
• When you are finished reading, turn to a neighbor
  and discuss what you noted.
• (Assessment Literacy Handout)

100
Formative Assessment Making it happen in the
classroom

• Read and ? (understand), ? (question) or
  (important).
• When you are done, discuss with your summer
  partner.
• Be prepared to discuss as a whole group.

100
101
Ms. Tolivers Classroom

• Review the Group 1, 3, and 4 questions on the
  handout.
• As you watch the video. Take notes on the
  different formal and informal assessment
  practices you are observing in her classroom
• When you are done, discuss with a neighbor.
• Be prepared to discuss as a whole group.

101
102
Break!
103
Welcome Back!
104
Work Period.. Start by unpacking the
standard(s) you will work with Continue work on
Student Friendly Language. Begin work creating
or reviewing assessments that will inform your
new accountability system.
105
Grant Information and
updates from Bob

• 1115 to Noon

106
LUNCH

• We will return in 60 minutes
• At..

107
Unpacked Standards Exchange

• Exchange at least one of your unwrapped standards
  with another group
• Take time to read the other groups unwrapped
  standards, discuss the work, answer questions,
  make changes, notice differences and similarities
  in each of your unwrapped standards

108
Work Period Continued.. Continue work on
Student Friendly Language. Begin work creating
or reviewing assessments that will inform your
new accountability system.
109
Draft Assessment exchange

• Meet with another group to share the work you
  have been doing in creating or revising
  assessments.
• Take time to hear about each others work, ask
  questions, and notice differences and
  similarities in the work you are doing.

110
Final
QuestionsPost Assessment Workshop
Assessment.Please complete todays evaluation
before leaving.Thanks!
End of Day 2
110
111
Thank you!
If you have any questions, please contact

• Steve Hamilton
• Tel [phone]
• shamilt_at_wested.org