Depths of Knowledge

1
The Florida Center for Research in Science,
Technology, Engineering, and Mathematics
Depths of Knowledge In Standards, Instruction,
and Assessment Danielle Sherdan,
Ph.D. Assistant Director for Technical Assistance
at FCRSTEM August, 2008
2
Outline

• What are the Depths of Knowledge (DOK)?
• How and why are tasks, standards, and assessment
  items classified in terms of DOK?
• Practice rating benchmarks and assessment items
• How can I, as a teacher, use the DOK ratings of
  the standards?

3
Levels of Cognitive Complexity/Depths of Knowledge

• Refer to cognitive processes involved in
  completing a task (as in standards) or an
  assessment item
• Assume student is familiar with basic concepts of
  the task and focuses on the task, not the student
• Webb, N.L. Wisconsin Center for Education
  Research and the National Institute for Science
  Education
• 4 Levels (1, 2, 3, 4) of Cognitive Complexity
  originally developed to rate assessment items,
  but adaptable for standards
• FCAT and Sunshine State Standards (each benchmark
  rated)
• 3 Levels (low, moderate, high) approximately
  equivalent to Webbs levels 1, 2, 3

4
Depth of Knowledge Levels - Science
Modified from Webb N.L., 2002 DOK Levels for
Four Content Areas
5
Depth of Knowledge Levels - Science
Modified from Webb N.L., 2002 DOK Levels for
Four Content Areas
6
Depth of Knowledge Levels - Science
Modified from Webb N.L., 2002 DOK Levels for
Four Content Areas
7
DOK depends on taskDifficulty depends on students
Tasks can be anywhere in this graph
High DOK
Moderate DOK
Any one task will be in one row only, but can be
in any column
Low DOK
Easy
Challenging
Average
Based on percentage of students likely to respond
correctly
8
Why are tasks, standards, and assessments
classified in terms of DOK?

• Alignment between standards and assessment can be
  based on
• Content
• Complexity

A mile wide and an inch deep
9
How are tasks, standards, and assessments
classified in terms of DOK?

• Process relies heavily on expert and reviewer
  judgment
• Reviewer agreement is essential
• Focuses on nature of items rather than
  perspectives of reviewers
• Process Reviewer training, practice, and rating
  reliability checking

10
Floridas Next Generation StandardsDepth of
Knowledge Ratings (and more) available at
http//www.floridastandards.org
11
Get a feel for the process of rating benchmarks
and apply your understanding of depths of
knowledge
Rate this benchmark
??
12
Get a feel for the process of rating benchmarks
and apply your understanding of depths of
knowledge by rating the benchmarks below
??
??
13
Depth of Knowledge/Level of Cognitive Complexity
on the FCAT
Information available on http//fcat.fldoe.org/fca
trelease.asp
The following slides will give an assessment
example at each of the three levels.
14
Depth of Knowledge/Level of Cognitive Complexity
on the FCAT
Example of low complexity item (grade 5)
From the FCAT Cognitive Complexity Classification
of Science Test Items document, January, 2008
15
Depth of Knowledge/Level of Cognitive Complexity
on the FCAT
Example of moderate complexity item (grade 5)
From the FCAT Cognitive Complexity Classification
of Science Test Items document, January, 2008
16
Depth of Knowledge/Level of Cognitive Complexity
on the FCAT
Example of high complexity item (grade 5)
From the FCAT Cognitive Complexity Classification
of Science Test Items document, January, 2008
17
Practice rating assessment items
??
From the 2007 FCAT Grade 8 released test book
18
Practice rating assessment items
??
This question asks for the next logical step and
is therefore considered a moderate complexity
question
From the 2007 FCAT Grade 8 released test book
19
Practice rating assessment items
??
This question asks for recall of a definition and
is therefore a low complexity item
From the 2007 FCAT Grade 8 released test book
20
Practice rating assessment items
??
This question asks to draw conclusions or
generalizations based on experimental findings
and is considered a high complexity task
From the 2007 FCAT Grade 8 released test book
21
Aligning DOK levels of standards and
assessmentsStandards ratings may serve as a
ceiling for assessment
From the support materials for core content
assessment for Science, 2007 Kentucky Department
of Education
22
Take-home messages

• Depths of knowledge classification is based on
  the task, not the student
• DOK is different from task/item difficulty
• DOK ratings aid in alignment of standards and
  assessment, and therefore instruction

23

• Danielle Sherdan dsherdan_at_lsi.fsu.edu
• The Florida Center for Research in Science,
  Technology, Engineering, and Mathematics