Understanding PT statistics

Importance of the topic

Understanding the statistical parameters reported in proficiency testing (PT) schemes is essential for laboratories, QA/QC personnel and method developers to correctly interpret interlaboratory comparison results, identify method bias or poor precision, and choose suitable PT schemes. Clear knowledge of location estimators, measures of dispersion and the associated uncertainties enables participants to assess their performance relative to peers and reference values and to make informed decisions about corrective actions or method improvements.

Objectives and overview of the guidance

The document provides concise guidance to PT participants on the statistical concepts that commonly appear in PT reports and on how those statistics are obtained, with reference to ISO 13528. It emphasises that PT reports should state the assigned value (x_pt), its uncertainty u(x_pt) and the standard deviation used for proficiency assessment (σ_pt). The leaflet explains common summary statistics for the location (center) and dispersion (spread) of results and gives practical points to consider when interpreting them.

Methodology and statistical approaches

The leaflet summarises standard approaches for estimating the location and dispersion of participant results and highlights ISO 13528 recommendations:

• Location estimators: arithmetic mean, median and several robust estimators (e.g., Algorithm A, Hampel-type estimators).
• Dispersion estimators: classical standard deviation (s) and robust alternatives such as scaled median absolute deviation (MADe), normalized interquartile range (nIQR), robust standard deviation s* (from Algorithm A, Qn or Q methods) and other robust measures.
• Standard uncertainty of the location: for the arithmetic mean, u(mean)=s/√p (where s is the classical standard deviation and p the number of results). For other estimators ISO 13528 recommends a conservative factor of 1.25, giving u(location)=1.25·s*/√p when s* is a robust standard deviation estimate.

Key practical points discussed include:

• Robust estimators are recommended when outliers exist or data symmetry is uncertain.
• If the arithmetic mean is used, PT providers should normally screen and remove outliers before reporting it.
• All estimators can be unreliable for small numbers of results; uncertainty and estimator choice should be interpreted cautiously in such cases.
• Significant differences between estimated location and external reference values may indicate method bias; providers may report method-specific locations when participants use different analytical methods.
• Dispersion measures are expressed in the measurand units or as percent; classical s requires outlier screening, while robust measures resist individual discrepant results better.

Main results and discussion

The leaflet summarises strengths and limitations of the different estimators (as in the reference table): robustness to outliers, efficiency for normal data, resistance to minority-mode biases (RMM) and breakdown point. In short: classical estimators (mean and s) have high efficiency for normally distributed data but are sensitive to outliers; robust estimators sacrifice some efficiency under ideal normality in order to provide reliable central location and dispersion in the presence of contamination or multimodality. The guidance stresses transparent reporting by PT providers of how assigned values, uncertainties and σ_pt were derived so participants can interpret performance scores correctly.

Benefits and practical applications

• Improved scheme selection: participants can prefer PT schemes whose statistical methods and reporting align with their needs (e.g., robust methods where heterogeneity or outliers are expected).
• Better performance interpretation: knowing the estimator and its uncertainty allows correct interpretation of z-scores or other performance metrics and helps distinguish random variation from systematic bias.
• Method comparison and troubleshooting: method-specific locations and dispersion estimates permit identification of method-related bias or precision issues and guide corrective actions.
• Regulatory and accreditation support: clear, standardised statistical reporting helps meet accreditation requirements and supports reproducible interlaboratory comparison results.

Future trends and opportunities

• Wider adoption of robust statistical methods in PT reporting, particularly for heterogeneous or multimodal data.
• Enhanced transparency from PT providers: more frequent publication of how x_pt, u(x_pt) and σ_pt are derived and of method-specific summaries where appropriate.
• Improved handling of small-sample PTs: use of conservative uncertainty estimates, bootstrap or simulation approaches, and clear communication of limitations when p is small.
• Integration of automated tools and interactive visualisations to explore distributions, detect multimodality and illustrate the impact of different estimators.
• Development of tailored PT schemes for groups of comparable methods and increasing use of statistical techniques that explicitly account for method groups or known covariates.

Conclusion

The leaflet provides a practical primer on the statistical measures commonly reported in quantitative PT schemes. Key recommendations for participants are to check which estimators the PT provider used, to pay attention to reported uncertainties (especially for small p), and to prefer schemes that report transparent, method-aware statistics. Robust estimators should be favoured when outliers or asymmetry are present; classical estimators remain valuable for well-behaved, approximately normal data after appropriate outlier handling.

Reference

1. Brookman B., Mann I. (eds.) Eurachem Guide: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes, 3rd ed., Eurachem, 2021.
2. Eurachem leaflet: Understanding PT performance assessment, Eurachem Proficiency Testing Working Group, first English edition, May 2024.
3. ISO 13528:2022, Statistical methods for use in proficiency testing by interlaboratory comparison, International Organization for Standardization.