Incorrect Results from Weighted Fits to Experimental Data

1
Incorrect Results from Weighted Fits to
Experimental Data

• This presentation will probably involve audience
  discussion, which will create action items. Use
  PowerPoint to keep track of these action items
  during your presentation
• In Slide Show, click on the right mouse button
• Select Meeting Minder
• Select the Action Items tab
• Type in action items as they come up
• Click OK to dismiss this box
• This will automatically create an Action Item
  slide at the end of your presentation with your
  points entered.

• Thomas M. Huber
• Steven H. Mellema
• Matthew C. Miller
• Gustavus Adolphus College
• http//physics.gac.edu/huber/fitting/

2
Summary of Presentation

• Overview of Weighted Least Squares Fitting
• Incorrect Uncertainty Calculations with
  Commercial Fitting Packages
• Monte Carlo Method of Estimating Fit Parameters
  and Uncertainties
• Status of Commercial Fitting Packages
• Status of Program-Independent Fitting Library
• Conclusions

3
Weighted Least Squares Fitting

• Physicist References for Weighted Least Squares
• Bevington (1969) Data Reduction and Error
  Analysis for the Physical Sciences
• Press, et al (1986-92) Numerical Recipes
• Weighted Least Squares Fit with Uncertainties in
  Y values
• Algorithm Adjusts Parameters a0, a1, to
  Minimize ?2
• Weighting Depends on Y Uncertainties 1/syi2

4
Problems With Commercial Packages

• Starting Two Years Ago, We Compared Commercial
  Packages (PsiPlot, Sigmaplot, Axum, ) to Results
  from Bevington Numerical Recipes Subroutines
• Most Tested Commercial Packages had Incorrect
  Uncertainties for Weighted Fits!
• Multiplied by Factor of Square Root of Reduced ?2
  relative to Bevington/Numerical Recipes
• Common to All Fit Functions (Linear, Power, )
• One implication, the Uncertainties in Fit
  Parameters were Independent of the Absolute
  Magnitude of the Errors Only Relative Scaling
  Mattered

5
Example of Implications of the Problem
Slope 0.989 /- 0.019 Intercept 0.061 /-
0.068 ?2 2.2
Slope 0.989 /- 0.189 Intercept 0.061 /-
0.681 ?2 0.022

• Same Data Set with Different Scaling of Y Error
  Bars
• Fit Parameters from Bevington/Num Rec. shown on
  Graph
• Commercial Packages Indicated Both Data Sets Have
  Same Uncertainty in Fit Parameters!
• Slope 0.989 /- 0.028
• Intercept 0.061 /- 0.101
• Regardless of how error bars are scaled

6
Which Algorithm is Correct?

• Needed to Verify Which Method was Correct for
  Calculating Uncertainties in Weighted Fit
  Parameters
• Analytically Solve for Equal Error Bars
• Agreement with Bevington/Numerical Recipes
• Developed a Monte Carlo Method For Arbitrary
  Error Bars
• Agreement with Bevington/Numerical Recipes

7
Monte Carlo Method For Estimating Fit Parameters

• Generate and Fit Large Number of Data Sets
• Vary Y Values Using Gaussian Errors in Data
  Points
• Fit Using Bevington Weighted Fit Algorithm
• Ignore Uncertainty in Fit Parameters

• Accumulate the Fit Parameters for Thousands of
  Varied Copies of the Data Set
• Accumulate Histograms and Statistics
• Compare to Weighted Fit Results

8
Sample Results from Monte Carlo
Bevington/Numerical Rec. 0.989 /-
0.189 Monte Carlo 0.990 /- 0.189 Original
Sigmaplot,Psiplot,.. 0.989 /-
0.028 Regardless of Scaling of Y Error Bars
9
Summary of Monte Carlo Analysis

• To Date, Weighted Fit Parameters from
  Bevington/Numerical Recipes and Uncertainties are
  Statistically Consistent with the Monte Carlo
  Analysis
• Includes Uncertainties in X and Y
• Monte Carlo Can Also Incorporate Additional
  Information About Data Set, such as
• X or Y Values Must be Greater Than Zero
• Asymmetric Error Bars
• Poisson Distribution for Counting Experiment

10
Status of Commercial Packages
11
Current Project Fitting Subroutine Library

• We Have Written a Program-Independent DLL
  Subroutine Library
• Simple Subroutine Calls from Visual Basic/C,
  Excel, Sigmaplot, Origin,
• Calculates Fit (User Interface and Graphics
  Written with the Calling Program Sigmaplot,
  Excel, etc.)
• Incorporates Uncertainties in both X and Y
• Algorithm by M. Lybanon (AJP, v. 52, 22, 1984)
• Allows Monte Carlo Analysis
• Planning To Use in Fall 2001 Classes

12
Conclusions

• Verified that there is Common Error in Weighted
  Fitting Packages
• Some Commercial Packages have been Updated to
  Eliminate this Error
• DLL Subroutine Library and Interfaces Should be
  Available in Fall 2001
• http//physics.gac.edu/huber/fitting/