CSAGS

1
Using run charts and control charts to monitor
quality in healthcare
CIST Workshop for ISD Scotland http//www.show.sco
t.nhs.uk/indicators
2
Workshop format

• Types of variation
• Illustrative example
• Run charts
• Illustrative example
• Simple exercises
• Control charts
• Illustrative example
• Some examples of CISTs work

3
Variation process levels

• Variation exists in all processes around us.
• Examples
• Everyone is different
• No two snowflakes are the same
• The weights of two cans of baked beans are never
  the same!

4
Worked example 1Coloured balls pulled from a bag
Suppose a bag contains 100 balls, 20 of which are
red. Scoopfuls of 20 are repeatedly drawn out,
with replacement. The number of red balls in
each scoop is observed, and the result of 25
scoops is plotted on a graph.
5
What causes the variation? This is an example of
a dynamic process This is how we typically
present data that arises over time Example
6
Worked example 2Coloured balls pulled from a bag
Suppose each group in this room has a bag, and
each bag contains 20 balls, 4 of which are red.
Draw out 10 balls, and count the number of red
balls observed in your scoop. The result of this
process can be plotted on a graph.
7
There is variation between the number of red
balls that each group observed This is an example
of a static process This is how we typically
present data that arises at a single point in
time Example
8
Sources of variation
Variation in our processes can be due to a number
of elements

• People
• Materials
• Machines

• Methods
• Measurements
• Environment

9
Causes of variation

• Example variability between patients
• Effect ever present, little or no effect, natural

• Example Poor methods, faulty materials
• Effect Infrequent, large effect, man-made(?)

10
The two types of variation

• Common cause variation
• Common cause variation is an inherent part of
  every process
• A process that exhibits common cause variation is
  said to be stable and in control
• A stable process enables prediction, within
  limits, about the future behaviour of the process
• Special cause variation
• Special cause variation is due to irregular or
  unnatural causes that are not an inherent part of
  the process
• A process that exhibits special cause variation
  is said to be unstable and out of control
• An unstable process will be unpredictable

11
Statistical process control (SPC) charts

• Simple graphical representation of current
  process performance
• Relatively easy to construct
• Easy to interpret
• Designed to identify which type of variation
  exists within your process
• Two of the most popular in common use today are
  the run chart and the control chart

12
How to construct a run chart (for dynamic
processes)

• Definition Run chart
• A time ordered sequence of data, with a
  centreline drawn horizontally through the graph
• Construction
• Typically, you should have a minimum of 15 data
  points
• Draw a horizontal line (the x-axis), and label it
  with the unit of time

13

• Draw a vertical line (the y-axis), and scale it
  to cover the current data, plus sufficient room
  to accommodate future data points. Label it with
  your outcome
• Plot your data on the graph in time order, and
  join neighbouring points with a solid line
• Calculate the mean or median of your data. This
  will be your centreline. Draw the centreline in
  your graph

14
Worked example 1 (cont)Coloured balls pulled
from a bag
Suppose that only the first 15 scoops had been
obtained, and we wish to produce a run chart for
this data. If we choose to use the mean as the
centreline, this is calculated as (35 4)/15
3.73. The resulting run chart is as follows
15
Now suppose the number of reds observed in the
16th scoop is obtained, and is 5. This new
observation can be included in the run chart.
Note that the centreline (the mean in our case)
needs to be recalculated. The mean is
(153.735)/16 3.81. The resulting run chart is
as follows
16
In fact, we could do this for observations 17,
18, , 25. So, when observation 17 is added the
resulting run chart is as follows
17
18
18
19
19
20
20
21
21
22
22
23
23
24
24
25
25
The run chart for our data is shown below. The
centreline is the mean. For our 25 observations,
the mean is 3.96. Before we proceed any further
we need some more definitions.
26
Definition Useful observations Those
observations that do not fall exactly on the
centreline. The number of useful observations in
your sample is equal to the total number of
observations minus the number of observations
falling on the centreline.
27
Worked example 1 (cont)Coloured balls pulled
from a bag
The total number of useful observations in our
sample is equal to the total number of
observations, i.e. 25.
28
Definition Run A sequence of one or more
consecutive observations on the same side of the
centreline. Ignore those observations that fall
directly on the centreline.
29
Worked example 1 (cont)Coloured balls pulled
from a bag
The runs in our run chart have been circled.
There are a total of 16 runs.
30
Interpreting the run chart to determine your
variation type
Recall, a run chart is a useful tool for
identifying which type of variation exists within
your process. There a number of rules which can
be applied to the run chart to determine the
variation type Rule 1 Number of runs? Rule 2
Shift Rule 3 Trend Rule 4 Zig-zag
31
Rule 1 Number of runs?
The purpose of this test is to see whether or not
there are too many or too few runs in the
process. The number of runs is compared to the
following table, which contains the upper and
lower critical values. If the number of runs in
your process is outwith these values, then there
is sufficient evidence that your process contains
special cause variation.
32
Table of critical values
33
Rule 2 Shift
The purpose of this test is to identify a shift
in the process. If you have less than 20 useful
observations, then a run containing 7 or more
observations indicates a special cause. If you
have 20 or more useful observations, then a run
containing 8 or more observations indicates a
special cause.
34
Rule 3 Trend
The purpose of this test is to identify a
statistical trend in the process. Definition
Trend A sequence of successive increases or
decreases in your observations Identification of
a trend is sufficient evidence that your process
contains special cause variation.
35
If you have 20 or less observations, then a trend
containing 6 or more observations indicates a
special cause. If you have more than 20
observations, then a trend containing 7 or more
observations indicates a special cause. Note that
if an observation is the same as the preceding
observation, this observation is not counted,
however, the counter still continues.
36
Rule 4 Zig-zag
The purpose of this test is to identify
non-random behaviour in the process. If the lines
between 14 or more consecutive points alternately
go up and down, this is an indication of special
cause variation in your process. As with Rule 3,
if an observation is the same as the preceding
observation, this observation is not counted,
however, the counter still continues.
37
Some fun Non-strenuous exercises!

• For the run charts in your handout, decide
• if the run chart indicates common cause or
  special cause variation in the process, and
• if the process does exhibit special cause
  variation, which rule it breaks.

38
How to construct a control chart (for dynamic
processes)
Definition Control chart A time ordered
sequence of data, with a centreline drawn
horizontally through the graph. This centreline
is the mean of the process. In addition, control
limits are included in the chart to provide an
extra tool for identifying the variation type.
39

• Construction
• The first step is to decide the most appropriate
  control chart for your data (see the following
  slide)
• Proceed as for a run chart, except that it is the
  mean that is calculated for the centreline
• Calculate the standard deviation (s.d.) using the
  formula for your chosen control chart type
• Calculate the control limits as
• centreline 3 s.d.
• Warning limits can be included in the chart.
  These are calculated as
• centreline 2 s.d.

40
Types of control chart

• I-chart
• For data that can be measured on a continuous
  scale (variable data), and where only a single
  observation is made at each time point.
• Example

41

• c-chart
• For countable data, where the area of
  opportunity is constant or roughly constant
  across the time points. For this chart it is the
  number of non-conforming units that is plotted.
• Example

42

• u-chart
• For countable data, where the area of
  opportunity is heterogeneous across the time
  points. For this chart it is the proportion of
  non-conforming units that is plotted.
• Example

43

• np-chart
• Where the data collected are countable data of
  nonconforming units, often referred to as
  attribute data. This chart is appropriate when
  the denominator is constant across the time
  points. For this chart it is the number of
  non-conforming units (the numerator) that is
  plotted.
• Example

44

• p-chart
• Where the data collected are countable data of
  nonconforming units, often referred to as
  attribute data. This chart is appropriate when
  the denominator is heterogeneous across the time
  points. For this chart it is the proportion of
  non-conforming units that is plotted.
• Example

45
Worked example 1 (cont)Coloured balls pulled
from a bag
Suppose that we wish to produce a control chart
for our data. We could construct a np-chart (if
we were interested in the number of red balls in
our 25 scoops) or a p-chart (if we were
interested in the proportion of red balls in our
25 scoops). Here we show how to construct a
np-chart for our data. (the p-chart for our data
is left as an exercise) We know that the mean of
our process, is 99/253.96. In addition, we
have and n20.
Hence the standard deviation can be calculated
as Warning limits and control limits can now be
included in the run chart to yield our control
chart.
46

• The np-chart for our data is shown below, where
• the upper and lower warning limits are
  represented by the orange and yellow dashes, and
• The upper and lower control limits are
  represented by the red and green dashes.

47

• The p-chart for our data is shown below, where
• the upper and lower warning limits are
  represented by the orange and yellow dashes, and
• The upper and lower control limits are
  represented by the red and green dashes.

48
Interpreting the control chart to determine your
variation type
The same rules (Rules 1-4) for determining the
variation type from a run chart also apply to a
control chart. However, the extra power of the
control chart to detect for special cause
variation arises from the addition of 2 further
rules. If either one of the following rules
applies to your control chart, your process
contains special cause variation.
49
Rule 5 2 successive observations outwith
the same warning limitRule 6 1
observation outwith the control
limitApplying Rules 1-6 to the control chart in
Worked Example 1, what would you deduce about our
process?
50
How to construct a control chart (for static
processes)
Recall the type of data in Worked Example 2,
where instead of data arising in a temporal
fashion, we actually obtained data from m groups
at a single point in time. It is possible to
produce a control chart for this type of
data. The construction of a control chart for
static processes, however, is different to that
for dynamic processes.
51
For the data in Worked Example 2, we have both
the denominator (n20), and the numerator (the
number of reds in each scoop) for each group. We
could plot the results on a chart, where the
x-axis is labelled with the denominator, as
opposed to time (as is the case for dynamic
processes). The y-axis is still our outcome, or
numerator. The centreline and control limits are
also included in the chart. The calculation of
the standard deviation of your process depends
crucially on your data type.
52
Recently, Mohammad et al., (2001) and Adab et
al., (2002) have constructed control charts of
this kind for healthcare data. They have been
suggested as an alternative to performance league
tables, as a method for presenting outcomes
data. The purpose of the charts is to compare
units (i.e. practices, surgical wards, etc) from
a single process (i.e. the NHS) that differ
wildly from the (national) average.
53
Example Control chart for number of deaths in
hospital within 30 days of admission for patients
admitted with myocardial infarction (patients
aged 35-74 years admitted to the 37 very large
acute hospitals in England during 1998-9)From an
example by Adab et al., (2002)
54
Worked example 2 (cont)Coloured balls pulled
from a bag
Suppose each group in this room has a bag, and
each bag contains 40 balls. The number of red
balls in each bag will differ 10 groups will
each have 20 red balls in their bag, 2 groups
will each have 4 red balls in their bag, and 2
groups will each have 36 red balls in their bag.
Each group will draw out a scoop whose size is
determined by your ticket. Once you have drawn
out your scoop, count the number of reds in your
scoop, and give your tutor this number, as well
as your scoop size. The aim is to identify, via
the construction of a control chart, which groups
have wildly different proportions of red balls in
their bag, compared to the average.
55
The control chart for our data is shown below,
where the upper and lower control limits are
represented by the red and green dashes. Can you
identify the bogus groups that had a different
proportion of red balls in their bag?
56
Further information
CIST website http//www.show.scot.nhs.uk/indicato
rs Adab P, Rouse AM, Mohammed MA, Marshall T.
Performance league tables the NHS deserves
better. BMJ 200232495-8. Carey R. How do you
know that your care is improving? Part II Using
control charts to learn from your data.
J Ambulatory Care Manage 2002,
25(2).78-88. Carey R, Lloyd R. Measuring quality
improvement in health care A guide to
statistical process control applications.
Milwaukee, WI ASQ Press. 2001. Mohammed MA,
Cheng KK, Rouse A, Marshall T. Bristol, Shipman,
and clinical governance Shewharts forgotten
lessons. Lancet 2001 357463-7.
57
Formulae for the calculation of standard
deviations
58
I-charts (Control chart 1) Assume you have m
observations, Xi , i 1, 2,, m. Calculate the
process average, Calculate the absolute moving
ranges (MRs) between adjacent observations, that
is MRi,i1 Xi - Xi1 , i 1, 2,,
m-1. Calculate the mean range, , as Set the
control limits at
59
For c- and u-charts (Control charts 2 3) Assume
you have m observations from a Poisson(µ)
distribution, i.e. Xi Poisson(µ), i 1, 2,,
m, where Xi is the number of non-conforming
units for observation i, and µ is the process
average. Since the true process average, µ, is
not known, we replace its value by the observed
process average, which is given by
60
c-chart Since it is the number of non-conforming
units that is plotted, i.e. X1, X2, , Xm , we
are required to calculate the s.d. for each Xi ,
i 1, 2,, m. Since the area of opportunity is
constant for all i, the s.d. is simply It is
this s.d. that is used for the calculation of the
control (and warning) limits.
61
u-chart Since it is the proportion of
non-conforming units that is plotted, i.e. Y1,
Y2, , Ym , where Yi Xi /ni Poisson(µ/ni), and
ni is simply a scaling constant that allows for
the heterogeneity of the area of opportunity, we
are required to calculate the s.d. for each Yi ,
i 1, 2,, m, which is given by It is this s.d.
that is used for the calculation of the control
(and warning) limits. Also note that since a
different s.d. is required to be calculated for
each proportion, Yi , i 1, 2,, m, different
control (and warning) limits are required to be
calculated, too.
62
For np- and p-charts (Control charts 4
5) Assume you have m observations from a Bin(ni
,p) distribution, i.e. Xi Bin(ni ,p), i 1,
2,, m, where Xi is the number of
non-conforming units for observation i, ni is
the number of units for observation i, and p
is the probability of success. In addition, let
63
np-chart Since it is the number of non-conforming
units that is plotted, i.e. X1, X2, , Xm , we
are required to calculate the s.d. for each Xi ,
i 1, 2,, m, which is given by However, since
nn1n2 nm, the s.d. is simply It is this
s.d. that is used for the calculation of the
control (and warning) limits.
64
p-chart Since it is the proportion of
non-conforming units that is plotted, i.e. Y1,
Y2, , Ym , where Yi Xi /ni , we are required to
calculate the s.d. for each Yi , i 1,2,,
m, which is given by It is this s.d. that is
used for the calculation of the control (and
warning) limits. Note that since a different s.d.
is required to be calculated for each proportion,
Yi , i 1,2,, m, different control (and warning)
limits are required to be calculated, too.