Group Exercise: Quadrant Analysis

1
Group Exercise Quadrant Analysis

• Value-Added Summer Workshop
• July 28th 2009

2
Purpose of Exercise

• Examine Value-Added with the added dimension of
  Attainment
• Understand exactly what interpretations can and
  cannot be made regarding the data
• Discuss policy implications regarding the four
  quadrants of value-added and attainment data

3
Exercise Format

• Divide into four sub-groups representing each of
  the four quadrants
• Quadrant 1 (High Value-Added, High Attainment)
• Quadrant 2 (High Value-Added, Low Attainment)
• Quadrant 3 (Low Value-Added, High Attainment)
• Quadrant 4 (Low Value-Added, Low Attainment)

4
Exercise Format

• For each quadrant discussion
• 5-10 minute sub-group discussion
• Report from the sub-group representing the
  quadrants school team
• Group discussion
• Key Points

5
Quadrant Analysis

• The mapping of value-added and attainment data in
  one graphic.

6
Quadrant Analysis

• Mapping can be done using either the state or
  district as the reference point.
• Data plots can be for districts (state
  value-added system),
• or for schools, grades or classrooms (either a
  state or district value-added system)

7
Quadrant Analysis

• Attainment data can be utilized from either the
  pre-test or the post-test.
• The pre-test is the preferred choice, since it is
  a pre-growth measure.

8
Quadrant Analysis

• Perspectives of Policymakers
• SEA analyzing districts and/or schools (statewide
  system)
• Superintendent analyzing schools
• Principal assessing school and analyzing
  grade-level performance

9
Quadrant Analysis

• Plotting Value-Added data
• The following graphic address the nature of
  standard deviations and the application of
  quadrants to the data.

10
Value-Added vs. Attainment
Plotting Value-Added
Tier Values
11
Quadrant Analysis

• Various Representations
• Quadrants
• Iron Cross (Standard Deviation bars)
• Nine Box

12
(No Transcript)
13
Quadrant 1 High Value-Added, High Attainment

• Be careful to accurately interpret the data.
• Characteristics
• These are schools (could also be districts or
  grades) that are both high value-added and high
  attaining.

14
Quadrant 1 High Value-Added, High Attainment

• Questions
• Are all Quadrant 1 schools equal?
• What does it mean to be within one of the
  standard deviation bars?
• What does it mean to be within both of the
  standard deviation bars?
• While the standard deviation bars help to gauge
  to meaningfulness of the school position in the
  quadrant, what part of the story is still
  missing?

15
Quadrant 1 High Value-Added, High Attainment

• Quadrant 1 School Team Report

16
Quadrant 1 High Value-Added, High Attainment

• Key Points
• It is critical that policymakers understand the
  dangers of over-interpreting the data.
• Expert Perspective
• Michael Christian, Assistant Scientist, VARC

17
Quadrant 3 Low Value-Added, High Attainment

• The Ceiling Effect Myth
• Characteristics
• Prior to value-added data, these high attaining
  schools had no reason to believe they were
  anything but highly successful.
• In reality their high attainment has little to do
  with the impact of the school.
• Some might refer to these schools as coasters,
  resting on the outside influences that lead to
  student attainment.

18
Quadrant 3 Low Value-Added, High Attainment

• Questions
• Why might the introduction of value-added to
  schools from this quadrant be a particularly
  painful process?
• Some would argue that because they are already
  high attaining schools, Quadrant 3 schools face a
  more difficult challenge in achieving high
  value-added status. Might there exist a ceiling
  effect that is curtailing their ability to be
  high value-added schools.?

19
Quadrant 3 Low Value-Added, High Attainment

• Quadrant 3 School Team Report

20
Quadrant 3 Low Value-Added, High Attainment

• Key Points
• There is no ceiling effect.
• Expert Perspective
• Brad Carl, Associate Researcher, VARC

21
Quadrant 2 High Value-Added, Low Attainment

• The Equitable Nature of Growth
• Characteristics
• When only viewing attainment data these schools
  (could also be districts or grades) are deemed to
  be failing.

22
Quadrant 2 High Value-Added, Low Attainment

• Questions
• Does attainment data alone paint an accurate
  picture of schools falling into Quadrant 2? If
  not, why not?
• Is there any benefit to studying the school
  culture and curriculum programs of schools from
  Quadrant 2?
• Is it enough to simply be a high value-added
  school? Can we expect Quadrant 2 schools to
  eventually become Quadrant 1 schools?

23
Quadrant 2 High Value-Added, Low Attainment

• Quadrant 2 School Team Report

24
Quadrant 2 High Value-Added, Low Attainment

• Key Points
• It is critical that we recognize the success
  these schools are having regarding value-added.

25
Quadrant 4 Low Value-Added, Low Attainment

• The kids cant learn.
• Characteristics
• These are the most challenged of schools (could
  also be districts or grades), for they are
  neither high value-added nor high attaining.

26
Quadrant 4 Low Value-Added, Low Attainment

• Questions
• While value-added allows for an apples to
  apples comparison of schools (meaning that
  schools from any of the quadrants can be utilized
  as relevant examples), why might Quadrant 2
  schools serve as a more palatable example for
  schools from Quadrant 4?
• From a Superintendents perspective, how might
  schools from Quadrant 4 be viewed regarding the
  use of district resources?

27
Quadrant 4 Low Value-Added, Low Attainment

• Questions (Continued)
• Say a Principal has just reviewed the quadrant
  plot for her school and sees that the school has
  moved from a Quadrant 1 school last year to a
  Quadrant 3 or a Quadrant 4 school this year. How
  might looking at a schools grade-level
  value-added plots be helpful in possibly
  diagnosing the change?

28
Quadrant 4 Low Value-Added, Low Attainment

• Quadrant 4 School Team Report

29
Quadrant 4 Low Value-Added, Low Attainment

• Key Points
• Low attainment is no excuse for being low
  value-added. Regardless of starting point, all
  kids are capable of growth.

30
Value-Added Tutorials

• Very soon well be making live a new section of
  the VARC website. Please do be sure to visit
• varc.wceruw.org/tutorials