aapt2001

Incorrect Results from Weighted Fits to Experimental Data: 

Incorrect Results from Weighted Fits to Experimental Data Thomas M. Huber Steven H. Mellema Matthew C. Miller Gustavus Adolphus College http://physics.gac.edu/~huber/fitting/ This presentation will probably involve audience discussion, which will create action items. Use PowerPoint to keep track of these action items during your presentation In Slide Show, click on the right mouse button Select “Meeting Minder” Select the “Action Items” tab Type in action items as they come up Click OK to dismiss this box This will automatically create an Action Item slide at the end of your presentation with your points entered.

Summary of Presentation: 

Summary of Presentation Overview of Weighted Least Squares Fitting Incorrect Uncertainty Calculations with Commercial Fitting Packages Monte Carlo Method of Estimating Fit Parameters and Uncertainties Status of Commercial Fitting Packages Status of Program-Independent Fitting Library Conclusions

Weighted Least Squares Fitting: 

Weighted Least Squares Fitting “Physicist” References for Weighted Least Squares Bevington (1969) Data Reduction and Error Analysis for the Physical Sciences Press, et al (1986-92) Numerical Recipes Weighted Least Squares Fit with Uncertainties in Y values Algorithm Adjusts Parameters a0, a1, … to Minimize χ2 Weighting Depends on Y Uncertainties 1/σyi2

Problems With Commercial Packages : 

Problems With Commercial Packages Starting Two Years Ago, We Compared Commercial Packages (PsiPlot, Sigmaplot, Axum, …) to Results from Bevington & Numerical Recipes Subroutines Most Tested Commercial Packages had Incorrect Uncertainties for Weighted Fits! Multiplied by Factor of Square Root of Reduced χ2 relative to Bevington/Numerical Recipes Common to All Fit Functions (Linear, Power, …) One implication, the Uncertainties in Fit Parameters were Independent of the Absolute Magnitude of the Errors – Only Relative Scaling Mattered

Example of Implications of the Problem: 

Example of Implications of the Problem Same Data Set with Different Scaling of Y Error Bars Fit Parameters from Bevington/Num Rec. shown on Graph Commercial Packages Indicated Both Data Sets Have Same Uncertainty in Fit Parameters! Slope = 0.989 +/- 0.028 Intercept = 0.061 +/- 0.101 Regardless of how error bars are scaled Slope = 0.989 +/- 0.019 Intercept = 0.061 +/- 0.068 χ2 =2.2 Slope = 0.989 +/- 0.189 Intercept = 0.061 +/- 0.681 χ2 =0.022

Which Algorithm is Correct?: 

Which Algorithm is Correct? Needed to Verify Which Method was Correct for Calculating Uncertainties in Weighted Fit Parameters Analytically Solve for Equal Error Bars Agreement with Bevington/Numerical Recipes Developed a Monte Carlo Method For Arbitrary Error Bars Agreement with Bevington/Numerical Recipes

Monte Carlo Method For Estimating Fit Parameters: 

Monte Carlo Method For Estimating Fit Parameters Generate and Fit Large Number of Data Sets Vary Y Values Using Gaussian Errors in Data Points Fit Using Bevington Weighted Fit Algorithm Ignore Uncertainty in Fit Parameters Accumulate the Fit Parameters for Thousands of Varied Copies of the Data Set Accumulate Histograms and Statistics Compare to Weighted Fit Results

Sample Results from Monte Carlo: 

Sample Results from Monte Carlo Bevington/Numerical Rec. 0.989 +/- 0.189 Monte Carlo 0.990 +/- 0.189 Original Sigmaplot,Psiplot,.. 0.989 +/- 0.028 Regardless of Scaling of Y Error Bars

Summary of Monte Carlo Analysis: 

Summary of Monte Carlo Analysis To Date, Weighted Fit Parameters from Bevington/Numerical Recipes and Uncertainties are Statistically Consistent with the Monte Carlo Analysis Includes Uncertainties in X and Y Monte Carlo Can Also Incorporate Additional Information About Data Set, such as X or Y Values Must be Greater Than Zero Asymmetric Error Bars Poisson Distribution for Counting Experiment

Status of Commercial Packages: 

Status of Commercial Packages

Current Project: Fitting Subroutine Library: 

Current Project: Fitting Subroutine Library We Have Written a Program-Independent DLL Subroutine Library Simple Subroutine Calls from Visual Basic/C, Excel, Sigmaplot, Origin, … Calculates Fit (User Interface and Graphics Written with the Calling Program: Sigmaplot, Excel, etc.) Incorporates Uncertainties in both X and Y Algorithm by M. Lybanon (AJP, v. 52, 22, 1984) Allows Monte Carlo Analysis Planning To Use in Fall 2001 Classes

Conclusions: 

Conclusions Verified that there is Common Error in Weighted Fitting Packages Some Commercial Packages have been Updated to Eliminate this Error DLL Subroutine Library and Interfaces Should be Available in Fall 2001 http://physics.gac.edu/~huber/fitting/