Users Guide to the QDE Toolkit Pro

1
Users Guide to the QDE Toolkit Pro
Ch 9 111
Sept 5, 2003
National Research Conseil national Council
Canada de recherches
Excel Tools for Presenting Metrological
Comparisons byB.M. Wood, R.J. Douglas A.G.
Steele
Chapter 9. Graphing and Pooling Measurement
Distributions (ii)
In this chapter, we present the use of the QDE
Toolkit Pros facilities for graphing
distributions and pooled distributions. We also
present some summary statistics, calculated by
the QDE Toolkit Pro, that are useful even before
a KCRV is chosen.
2
Ch 9 112
QDE Toolkit Pro Graphs - working with these
Pooled Distributions
After running the macro tk_pool_PlotBuilder, the
graphs can be adjusted with Excel. Heres a quick
review of some things you will want to do
Point at a corner or edge of a selected graph and
drag (left mouse button) to resize.
Point at a graph, left mouse click to select it
and pop it to the top of the stacked graphs.
Tip We find Excels undo capability is very
useful in reversing unwanted formatting of
graphs. Select EditUndo with the mouse, or
control Z from the keyboard.
Double-click on the selected charts y axis (the
vertical line at x0) to get a dialog box, select
the scale tab, enter the new y maximum...
Similarly, the x-axis range, etc. can be
adjusted. There is a very wide range in x of data
points available (1000 uniformly spaced points
across the default graph range and 1000 more on
an expanding mesh to cover 200 x the breadth
(for integrating Student distribution tails).
3
Ch 9 113
QDE Toolkit Pro Graphs - working with these
Pooled Distributions
After running the macro tk_pool_PlotBuilder, the
graphs data can be edited or recalculated with
Excel (for example to convolute with either a
uniform or a goalpost distribution).
The data (2000 rows x 10-20 columns) are
extensive enough so that some Excel tricks (shift
EditPaste Picture, or using F9 to convert a
graph series into an explicit formula array) for
taking a frozen picture of a graph no longer
always work well. It is often very helpful to
take a frozen snapshot of a graph, andgoing
beyond screen resolution can be a challenge!
After getting the Excel graph about right, one
trick seems to give the best for-printing
conversion to Windows metafile form (continued
on next page)
Prior to Excel 97 SR2, Excel had a memory leak
when creating lots of charts that could give a
not enough memory error and required restarting
Windows occasionally.
4
Ch 9 114
QDE Toolkit Pro Graphs freezing a graph (as a
picture)
For Office XP, we copy the selected graph to the
clipboard Control-C Paste it into Microsoft
Word Control-V, then click on the clipboard
icon beside the pasted graph to select the
Paste Options select Picture of Chart
(smaller file size) Press the Esc key to
get rid of the icon Select and copy the graph in
Word Paste it back into Excelas a frozen
picture.
For Office 97, or for Office 2000, we copy the
selected graph to the clipboard, and do a Edit
Paste Special into Microsoft Word as a Picture
(Enhanced Metafile). Then, copying from this
graph in Word, it can be pasted (not Paste
Special this time) back into Excel as a frozen
picture.
5
Ch 9 115
QDE Toolkit Pro Graphs - working with
Reference-Value Distributions
The macro tk_pool_PlotBuilder can also graph the
PDFs of the candidate KCRVs. The RVs are graphed
with thick lines. In column A of the input table,
the block of reference values starts beneath the
block of Lab comparison data, with the first name
containing the character pair RV (case
insensitive). It ends with the first blank in
column A.
Here, two RVs are graphed. One is labeled the
KCRV, with no uncertainty given. (This is
sometimes done to avoid profound difficulties,
for example by the CCT.) This first RV (its value
happens to be the simple mean of the 8 pooled
Labs) is plotted as a delta-function (really .001
of the initial graph width). The second RV is an
inverse-variance weighted mean of the 8 pooled
labs, with its formal standard uncertainty (the
same as the product of PDFs, since they are all
normal) and correlation coefficients with the
contributor labs (which are not used in plotting
the RVs PDF).
6
Ch 9 116
QDE Toolkit Pro Graphs - working with the
supplementary information
After running the macro tk_pool_PlotBuilder,
there is a block of supplementary information in
the 16 columns to the right of the correlation
coefficient matrix. We will discuss them in turn,
but heres the overview
Tip the Toolkit Pros output italic numbers are
unitless, and regular font numbers have units.
Col after rij 1 Lab names comments hold reduced
chi-square of differences with all other pooled
Labs. 2-4 Input Comparison Data for Labs, similar
statistics for pools 5-6 Du/u for degrees of
freedom variance-based and tail-based. 7-8
Coverage Factors for 68.0 and 95.0 confidence
(from trapezoidal integration of the
distributions). Mostly of interest for the pooled
distributions. 9-12 Error in the symmetric vs
rigorous confidence intervals. Mostly of interest
for quantifying the un-importance of this effect
for the asymmetric pooled distributions. 13-15
Mean, Median and (first) Mode of each
distribution. Mostly of interest for the pooled
distributions, 16 Lab Names, just for
convenience
7
Ch 9 117
QDE Toolkit Pro Graphs - col 1 of the
supplementary information
1
Col after rij 1 The comment in the top row of the
column contains the date and time of creation of
the block of supplementary information. The
column is of Lab names for each row, but some
really useful information is in the comments of n
in-pool labs, which give the reduced chi-square
statistical information about their rms En with
all other (n-1) in-pool Labs (n-1) terms,
between 1 and (n-1) degrees of freedom. The
bottom (Pair Difference) name has a comment
that gives the all-pairs chi-squared for the
pool there are n(n-1) terms to sum, with an
obvious 21 redundancy Despite having n(n-1)
terms, the degrees of freedom is still
(n-1). Note that this chi-square is independent
of any choice of KCRV. Its use is illustrated in
Hill, Steele and Douglas, Metrologia 39, 269
(2002). It is also discussed in a bit more detail
on the following page. It uses the Excel
statistical function ChiDist to convert from an
all-pairs-variance to a probability that it could
be exceeded by chance.
8
Ch 9 118
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION
about Lab j
The chi-squared of each of N in-pool Labs, with
respect to the N-1 other Labs, is calculated and
put into this column as comment boxes. This is
the Labs rms En ?j2 (N-1)-1
?i1Ni?j(xi xj)2 / (ui2 uj2 - 2rijuiuj)If
the differences are independent and normally
distributed about zero, then this is EXACTLY a
reduced chi-squared statistic with N-1 degrees of
freedom. It is perhaps the best test as to
whether Lab j agrees with the rest of the
world, and is independent of any choice of KCRV.
In practice, the differences are only independent
when the other Labs uncertainties dominate, and
the degrees of freedom for ?j2 may be as small as
1 when uj dominates.
If the differences were independent and there was
perfect agreement, within the stated
uncertainties, for Lab j, this value of ?j2 is
expected to be exceeded by chance with only this
probability a minimum probability that may be a
substantial underestimate if the differences
lack of independence is considered. In the
Toolkit Version 2.07, the probability, between 0
and 1, is given in scientific number format.
9
Ch 9 119
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION
about comparison
The reduced chi-squared Labs j with respect to
the N-1 other Labs, is ?j2 (N-1)-1
?i1Ni?j(xi xj)2 / (ui2 uj2 - 2rijuiuj)By
averaging the N ?j2s, ?2 (N)-1 ?i1N ?j2
the all-pairs variance. It is perhaps the best
test as to whether these Labs agree with each
other within their uncertainties, and is
independent of any choice of KCRV. The degrees of
freedom of this reduced chi-squared is N-1.
If there were perfect agreement, within the
stated uncertainties, for all Labs, this value of
?2 is expected to be exceeded by chance with only
this probability. If this probability is very
small, then the measured differences are not
described very well by the stated
uncertainties. In the Toolkit Version 2.07, the
probability, between 0 and 1, is given in
scientific number format.
10
Ch 9 120
QDE Toolkit Pro Graphs - VERY USEFUL INFORMATION
a TIP
There are so many comment boxes that the workbook
needs to have its comments hidden unless mouse is
pointing to a commented cell. (Tools Options
View and select Comment indicator only. )
To keep one or more comment boxes on
display, Right-mouse-click on the cell Select
Show comment
To hide one or more comment boxes that is on
display, Right-mouse-click on the cell Select
Hide comment
11
Ch 9 121
QDE Toolkit Pro - Using Pair Differences
The bilateral pair differences of laboratories
can incorporate all knowledge about the pairs,
including correlations, so the reduced chi-square
of all pair differences is an appropriate tool to
consider for characterizing a comparison model
without having to choose a specific reference
value. The all-pairs variance or APV of a
comparison with N laboratories can be
calculated APV ???i 1N ?j 1N (xj - xi)2 /
(ui2 uj2 - 2 rij ui uj) / (N(N-1)) for normal
PDFs for the xis , the APV is distributed as a
reduced chi-square with N-1 degrees of
freedom. Aside Here we use the name APV here
since it sometimes may be appropriate to
interpret the APV in terms of its model
distribution, which is not necessarily a reduced
chi-squared if we include the effects of the
stated effective degrees of freedom for each
laboratory. In the QDE Toolkit Pro Versions
2.042.07, the analysis is all in terms of the
normal distribution limit and the reduced
chi-squared.

12
Ch 9 122
QDE Toolkit Pro - Using Pair Differences
The all-pairs variance or APV of a comparison
with N laboratories is APV ???i 1N ?j 1N (xj
- xi)2 / (ui2 uj2 - 2 rij ui uj) /
(N(N-1)) for normal PDFs for the xis , the APV
is distributed as a reduced chi-square with N-1
degrees of freedom.

At the left, the Monte Carlo simulation of the
distribution of the APV of a 12-Lab comparison is
compared with the reduced chi-squared curves
having 10, 11 and 12 degrees of freedom. Although
the APV sum is over 132 non-zero terms, with 66
terms that look distinct, the distribution of
APVs is just the reduced chi-squared distribution
with 11 degrees of freedom. The APV has been
constructed to be independent of any change of
reference value. Although we may have not yet
chosen a specific KCRV, one degree of freedom is
used by this possibility.
13
Ch 9 123
QDE Toolkit Pro - Using Pair Differences
The all-pairs variance or APV of a comparison
with N laboratories is the master chi-squared for
the comparison. This is true in the sense that
if a comparison has failed the APV
chi-squared test, then NO choice of reference
value can, by itself, rescue the
comparison This can be demonstrated by
considering what happens to the pair differences
when the reference value is changed the pair
differences are invariant, and so the APV chi
squared statistic is also invariant.

14
Ch 9 124
QDE Toolkit Pro Graphs - col 2-4 of the
supplementary information
2
4
3
Col after rij 2-4 Input Comparison Data for Labs
value, uncertainty, degrees of freedom. In
columns 2 and 3, similar stastistical information
is calculated for the pools, and explained in
comments. Degrees of freedom is just as input,
with a default to normal.
3
4
15
Ch 9 125
QDE Toolkit Pro Graphs - col 5-6 of the
supplementary information
5
6
Col after rij 5 Du/u for degrees of freedom
estimate based on the chi distributions
variance-about-the-mean based and tail-based. The
overly broad tails of the Student distribution at
low degrees of freedom arise from the small
arguments (and large value) of the chi
distribution.Approximate fractional uncertainty
in the standard uncertainty from variance of the
chi-square family of distributions that are
highly asymmetric for degrees of freedom less
than 10. See ISO Guide Eq.E-7 Table E-1. 6
Du/u for degrees of freedom Improved estimate Du
for the fractional uncertainty in the standard
uncertainty, derived from the correct inverse-chi
distribution's interval 0,uDu with 84
confidence (the same Du as Col 5 in the limit of
large degrees of freedom). See discussion on
pages 49-52 of this Guide, in the context of
discussing Tables of Equivalence.
Two definitions for degrees of freedom?No. The
variance-based method is simply a bad
approximation for degrees of freedom lt 10, if the
degrees of freedom is to be used for evaluating
coverage factors from Student distributions.
Nonetheless, we suggest using the symbol nS for a
degrees of freedom aimed at describing the tails
of the Student distribution. Fortunately, in
precision metrology, usually n gt 10 and there is
no difficulty.
16
Ch 9 126
QDE Toolkit Pro Graphs - col 7-8 of the
supplementary information
7
8
Col after rij 7-8 Coverage Factors for 68.0 and
95.0 confidence (from trapezoidal integration of
the distributions). Mostly of interest for the
pooled distributions. Because the product PDF is
narrow, there may be only a few tens of samples
within s. An easy way of monitoring this effect
is to include a normal RV of about the same value
and width. Any variation of the coverage factor
from the expected norm (here 0.1) is an
indication of the accuracy if the product were
over Student distributions instead of normal
distributions. If the accuracy is not sufficient,
in the Visual Basic code in module
QDE_Toolkit_PlotBuilder, in subroutine
tk_pool_PlotBuilder_With_Anchor, near comment
line C140, change poolpoints 1002 to a larger
number, such as poolpoints 10002 (Excel will
limit you to 60000)... near comment line C110
in the same subprogram Confidence_k1
0.68 Confidence_k2 0.95 can be edited to the
values of your choice. These do not affect the
MRA value of 95 confidence.
17
Ch 9 127
QDE Toolkit Pro Graphs - col 9-12 of the
supplementary information
9
10
11
12
Col after rij 9-12 Error in the symmetric vs
rigorous confidence intervals. For the symmetric
input data, you see the effects of round-off in
the trapezoidal integration. These columns are
mostly of interest for quantifying the
un-importance of asymmetry for the asymmetric
pooled distributions How close are the
intervals mean-u, meanu and mean-U, meanU
to the true asymmetric confidence intervals,
starting for example at X where CDF(X)0.025 and
running to X where CDF(X)0.975.
18
Ch 9 128
QDE Toolkit Pro Graphs - col 13-15 of the
supplementary information
13
14
15
Col after rij 13-15 Mean, Median and (rightmost)
Mode (the zero-slope peak) determined from the
table of each distribution. Now this is mostly
of interest for the pooled distributions,
So as you see, theres lots of information
included in the supplementary information
columns that is eminently ignore-able, most of
the time. If and when it is wanted, it will be
waiting!