Depth of Knowledge

1
Depth of Knowledge

• Its Not Just a Verb

2
Depth of Knowledge (DoK)

• A variety of models has been developed to
  evaluate the Depth of Knowledge of content
  standards and the corresponding assessments.
• One of the most popular is Norm Webbs Depth of
  Knowledge Levels.
• http//www.education.ky.gov/KDE/AdministrativeRe
  sources/SchoolImprovement/InstructionalSupportN
  etwork/ISNRefinedCoreContentforAssessmentMat
  erials.htm

3
Depth of Knowledge (DoK)

• CATS Items are coded using three DoKs.
• These three broad categories (based on Norm Webb)
  used are as follows
• Level 1 Recall and Reproduction
• Level 2 Skills and Concepts
• Level 3 Strategic Thinking
• While Webbs model includes a fourth level,
  Extended Thinking, it is reserved for activities
  that require extended amounts of time, such as
  authentic writing.

4
Depth of Knowledge (DoK)

• Some general distinctions can be made among the
  levels.
• Level 1 Requires students to recall or
  recognize facts or to use simple skills or
  abilities.
• Level 2 Requires students to engage in some
  form of mental processing beyond recalling or
  reproducing a response.
• Level 3 Requires students to reason, analyze,
  evaluate, or synthesize information.

5
(No Transcript)
6
Bloom and Webb How do They Compare?
7
(No Transcript)
8
DOK is not a Verb

• Verbs must be considered in context.
• Explain to me where you live does not raise the
  DOK of a simple rote response.
• Even if the student has to use addresses or
  landmarks, the student is doing nothing more that
  recalling and reciting.

9
DOK is About What Follows a Verb

• What comes after the verb is more important than
  the verb itself.
• Analyze this sentence to decide if the commas
  have been used correctly does not meet the
  criteria for high cognitive processing.
• The student who has been taught the rule for
  using commas is merely using the rule.

10
A Verb Across 4 DOK Levels Depth of Knowledge is
not verb dependent. It is based on the cognitive
complexity of the standard.
11
DOK is not about difficulty
Difficulty is a reference to how many students
answer a question correctly. How many of you know
the definition of exaggerate? DOK 1-recall (If
all know the answer, then it is easy.) How many
of you know the definition of prescient? DOK
1-recall (If most do not know the definition,
this question is a difficult question, but that
alone does not change the DoK level.)
12
The Bell-Shaped Curve
13
Depth of Knowledge (DoK)

• The Core Content for Assessment Version 4.1
  specifies DoKs for the content.
• The DoKs that are specified represent the
  ceiling for the assessment, not the target.
• The ceiling represents the highest level of
  knowledge students are expected to demonstrate on
  the assessment.
• The target represents the depth of knowledge
  reflected in the item.

14
REMEMBER-

• When testing at DoK level 2, you must teach at
  DoK levels 3 and 4.

15
Depth of Knowledge (DoK)

• It is important not to confuse DoK with
  difficulty.
• A Level 1 item can be very difficult. Think of
  Final Jeopardy or Who Wants to be a Millionaire.

16
Depth of Knowledge (DoK)

• It is important not to confuse DoK with
  difficulty.
• 3x2
• 2,563,907 x 3,458,212

17
DoK IS Not about Difficulty.

• Its about the INTENDED OUTCOME!

18
DOK is reference to the complexity of mental
processing that must occur to answer a question,
perform a task, or generate a product.

• Adding is a mental process.
• Knowing the rule for adding is the intended
  outcome that influences the DOK.
• Once someone learns the rule of how to add 44
  is DOK 1 and is also easy.
• Adding 4,768,348 9,876,543 is still DOK 1 but
  may be more difficult.

19
DoK is Also About Complexity

• The intended student learning outcome determines
  the DoK level.
• Instruction and classroom assessments must
  reflect the DoK level of the intended learning
  outcome.

20
Content Standards Across 4 Levels It shows four
core content statements around the same idea
using information.
21
Depth of Knowledge Distinctions

• Some general distinctions can be made among the
  levels.
• Level 1 Requires students to recall or recognize
  facts or to use simple skills or abilities.
• Level 2 Requires students to engage in some
  form of mental processing beyond recalling or
  reproducing a response.
• Level 3 Requires students to reason, analyze,
  evaluate, or synthesize information.

22
DoK Level 1 Recall and Reproduction

• Focus is on the following
• Specific facts
• Definitions
• Details
• Routine procedures

23
DoK 1Know it! Do it!

• Level one requires students to receive or recite
  facts or to use simple skills or abilities.
• Its a shallow understanding consisting of
  verbatim recall from text or a simple
  understanding of a single word or phrase.

24
DoK Level 1 TasksRecall and Reproduce

• Recall elements of a story
• Apply a math formula
• Follow a clearly defined series of steps

25
DoKLevel 2Skills, Concepts, Basic Reasoning

• Focus is on the following
• Basic application of skills and concepts (in a
  familiar/typical situation)
• Decision-making approaches
• Relationships such as compare/contrast
• or cause-effect
• Main ideas
• Explanations of how or why

26
DoK 2What do I do? Decide upon approach, then
apply.

• Depth of Knowledge level 2 requires students to
  engage in mental processing beyond recalling or
  reproducing a response.
• May require students to apply some of the skills
  and concepts that are covered in DoK 1, but not
  in a complex way. Thinkers summarive, interpret,
  infer, classify, organize, collect, display,
  compare

27
DoK Level 2 TasksSkills, Concepts, Basic
Reasoning

• Design a market strategy.
• Paint a mural.
• Summarize events.
• Determine fact and opinion.
• Cite evidence.

28
DoKLevel 3Strategic Thinking/Complex Reasoning

• Focus is on the following
• Reasoning and planning
• Complex and abstract thinking
• Going beyond text
• Explaining, connecting
• Applying prior knowledge

29
With DoK Level 3

• More that one correct response or approach is
  often possible

30
Depth of Knowledge Level 3 Tasks

• Describe, compare solution methods.
• Solve a multiple-step problem and provide support
  with an explanation that justifies the answer.
• Indicate priority and ratings.
• Conduct a debate.

31
Depth of Knowledge Sorting Activity
32
Depth of Knowledge Level 1 Example

• Bill lives on the side of the street with
  even-numbered addresses. Which addresses below
  would be found on Bills side of the street?
• A) 1020, 1022, 1024
• B) 2021, 2023, 2025
• C) 3168, 3169, 3170
• D) 4167, 4168, 4170

33
Level 1 Example
List three properties that can be used to
classify objects.
1 2 3
34
DoK Level 1 Example

• Name the basic purposes of government in the
  United States (as stated in the Preamble to the
  United States Constitution).

35
DoK Level 1 Example

• From any vertex of a 4-sided polygon, 1 diagonal
  can be drawn. From any vertex of a 5-sided
  polygon, 2 diagonal can be drawn. From any vertex
  of a 6-sided polygon, 3 diagonal can be drawn.
  From any vertex of a 7-sided polygon, 4 diagonal
  can be drawn. How many diagonals can be drawn
  from any vertex of a 20-sided polygon?
• This is not really a geometry item. Rather, it
  simply requires students to notice an easy,
  routine pattern. DOK levels are difficult to
  assign for many pattern-recognition problems,
  because they depend upon how routine the pattern
  is. This particular pattern is immediately
  recognizable and requires no processing, but a
  more complex pattern could make this a Level 2 or
  a even Level 3 item.

36
DOK Level 2 Example

• Explain how the Preamble defines the basic
  purposes of the U.S. Government.

37
Depth of Knowledge Level 2 Example

• Construct two-dimensional patterns for
  three-dimensional models, such as cylinders and
  cones.
• This requires at least two steps first,
  recognizing the shape and, second, drawing a
  two-dimensional object to reflect the shape in
  three dimensions.

38
Depth of Knowledge Level 2 Example

• The class went on a field trip. The students
• left the school at 900 a.m. They returned to
• school at 130 p.m. How long were they gone?
• A) 8 hours 30 minutes
• B) 8 hours
• C) 4 hours 30 minutes
• D) 4 hours
• The choices offered indicate that this item is
  intended to identify students who would simply
  subtract 9 minus 1 to get 8. More than one step
  is required here. The students must first
  recognize the difference between a.m. and p.m.
  and make some decisions about how to make this
  into a subtraction problem, then do the
  subtraction.

39
Depth of Knowledge Level 2 Example

• In which set are the numbers equivalent?
• A) 1/3, 3/27, 33
• B) 0.090, 90, 0.90
• C) 88, 88/100, 22/25
• D) 0.66, 2/3, 66.7
• There are a number of different concepts and
  procedures that can be used for this problem,
  rather than an obvious, simple one. Students
  must not only be able to identify different
  representations of rational numbers (Level 1),
  but also to manipulate and compare these
  representations (Level 2). This means that
  numerous interdependent and non-trivial steps are
  involved here. However, this does not require
  ant conjecturing, planning, abstracting, or
  explaining, so it is not Level 3.

40
Depth of Knowledge Level 3 Example

• Think carefully about the following question.
  Write a complete answer. You may use drawings,
  words, and numbers to explain your answer. Be
  sure to show all of your work.
• Laura wanted to enter the number 8375 into her
  calculator. By mistake, she entered the number
  8275. Without clearing the calculator, how could
  she correct her mistake? Without clearing the
  calculator, how could she correct her mistake
  another way?
• An activity that has more than one possible
  answer and requires students to justify the
  response they give would most likely be a Level
  3. Since there are multiple possible approaches
  to this problem, the student must make strategic
  decisions about how to proceed, which is more
  cognitively complex than simply applying a set
  procedure or skill.

41
DoKLevel 3Example

• S a/b c/d
• If Oltaltbltcltd in the equation above, then the
  greatest increase in S would result in adding 1
  to the value of which variable?
• A) a
• B) b
• C) c
• D) d
• E) There is not enough information to know for
  certain.
• If a multiple-choice item is Level 3, often it is
  because the multiple choices do not constrain or
  guide the possible solutions. The choices here
  allow for all possible responses to this item,
  including the response that the problem cannot be
  solved. This gives such an item the character of
  an open-ended item, even though it is not one.

42
DoK 4How its Different

• Multiple sources
• Extended amounts of time
• Numerous variables
• Several connections
• New situations
• Deep awareness
• Synthesis
• Creativity

43
DoK 4Extended Reasoning

• Focus is on the following
• Complex analysis
• Alternative, new perspectives
• Abstractions

44
Remember

• Depth of Knowledge (DOK) is a scale of cognitive
  demand.

45
Remember

• DOK requires looking at the
• assessment item/standard-
• not student work-in order to
• determine the level.
• DOK is about the item/standard-not
• the student.

46
Remember

• The context of the assessment item/standard must
  be considered to determine the DOK not just a
  look at what verb was chosen.

47
Lets Practice DOK WALK
48
Reading Grade 4Birbal and the BarberBy T.V.
Padma

• Near the end of the passage, the king laughed
  because he
• a. was happy that his bee sting and stubbed toe
  felt better
• b. realized that he did not have to give away
  any bags of gold.
• c. Was happy that he would not have to see the
  barber anymore
• Realized he had been wrong to blame the barber
  for his bad luck.
• DoK level 2

49
Reading Grade 4Birbal and the BarberBy T.V.
Padma

• 6. Birbal would MOST likely agree that
• a. Kink Akbar had a very unlucky face
• b. The barber should not live in the kingdom
• c. King Akbar had guards who where very rude.
• People do not really have unlucky faces.
• DoK level 2

50
Read all parts of the question before you
begin. 7. The last sentence of the passage says
that Birbal went to tell the barber the good
news, with a little smile tugging at the corners
of his mouth. a. Describe the good news Birbal
was going to tell the barber. b. Explain TWO
reasons why Birbal might have been smiling. Use
examples from the passage to support your
answers. DoK Level 3
51
Reading Grade 7

• When the king says, Alas, I am old and ill, and
  my son is mad, he means that his son is
• A. insane
• B. angry
• C. excited  
• D. strange
• DoK Level 2

52
Reading Grade 7
Tom looks distressed after the Prince leaves him
because A. position of an object. B. time that
the object is observed. C. frequency of
appearance of the object. D. duration of time
that the object is evident. DoK level 2
53
Open ResponseReading, 7th Grade
The Prince and the Pauper tells the story of two
people whose lives are very different. a.
Describe three ways in which Toms life is
different from the Princes. b. Explain why
these differences in their lives cause them to
want to change places. DOK 3
54
Reading Grade 10
The main reason the passage is written from the
first-person point of view is to A. convince
readers that the author is right. B. provide
readers with the authors perspective. C. enable
the author to express unbiased opinions. D. show
the authors professional qualifications.
This is a DOK 3 because it requires the student
to process the text the student must understand
first-person point of view and be able to
conclude why the author chose it.
55
Reading Grade 10 Multiple Choice
Which of these words best describes the tone of
this passage? A. cynical B. humorous C.
reflective D. exhuberant DOK 3
56
5th Grade Math
2. Which number is a multiple of 3, a factor of
42, and odd? A. 6 B. 9 C. 14 D. 21 DOK 1
57
5th Grade Math
Look at the table. If 56 is input into the table,
what will be the output? A. 6 B. 7 C. 8 D. 9 DOK 2
58
8th Grade Math
7. Ashley and her cousin Stella have been
invited to go to their grandmothers house next
summer. The girls live in different towns. In
order to go, each girl must save enough allowance
money to buy her own bus ticket. Ashley has
saved 2/3 of the 66 she needs for her ticket,
and Stella has saved 4/5 of the 75 she needs for
hers. A. Determine the amount of money that
Ashley and Stella have each saved so far. Show
your work. B. Ashley gets an allowance of 5 at
the end of each week. If she saves all of her
allowance, how many more weeks must she save
until she has the 66 needed for her bus ticket?
Show your work. DOK 2
59
4th Grade Science
6. A certain type of bird lives in Kentucky
during the summer. In the winter, the bird flies
much farther south. Which is the MOST likely
reason the bird flies much farther south in the
winter? A. Kentucky is too cold in the
winter. B. Kentucky gets too much wind in the
winter. C. Too many other animals live in
Kentucky in the winter. D. There are too many
thunderstorms in Kentucky in the winter. DOK 2
60
11th Grade Science
7. A space station must be designed so that the
oxygen and carbon dioxide cycles that occur on
Earth will also occur in the space station. On
Earth, these two gases are continually produced
and recycled by the natural processes that take
place in plants and animals. A. Describe the two
processes that occur in plants and animals to
produce and recycle these gases (i.e., oxygen and
carbon dioxide). B. Name four organisms that
could be placed in the space station to ensure
that the oxygen and carbon dioxide cycles take
place. Explain how each organism will continue
these cycles. C. Explain why your selection of
organisms is reasonable given the condition of a
space station. DOK 3
61
5th Grade Social Studies
4. In what are of the US would settlers have had
the greatest need to conserve water? A.
Northeast B. Southeast C. Northwest D.
Southwest DOK 2
62
5th Grade Social Studies
5. As a symbol of friendship, France gave
America the A. Liberty Bell B.
Statue of Liberty C. Lincoln Memorial
D. Washington Monument DOK 1
63
11th Grade Social Studies
6. During World War II, the United States
experienced many shortages of resources. A.
Identify two shortages the US experienced during
World War II. B. Explain how the US dealt with
each of the shortages you identified in part a.
Use details to support your answer. DOK 2
64
4 Guiding Questions to Explore About Assessing
Student Learning at the Classroom Level

• How is understanding of the content best
  measured?
• How are assessments aligned with content
  standards and what we expect students to know and
  be able to do?
• What learning experiences will deepen student
  knowledge and application of content?
• How can we incorporate DOK into our classroom
  assessment design?

65
Remember
When an assessment item is written at a DOK 2
level, instruction needs to require DOK 3 or 4
depth of content and thinking in order for
students to be successful on that item.
66
Whats Next?