Understanding PT performance assessment

Significance of the topic

Proficiency testing (PT) is a cornerstone of laboratory quality assurance and accreditation. Quantitative PT schemes provide objective external evaluation of a laboratory’s measurement capability, support metrological traceability, and identify weaknesses in measurement results or uncertainty estimation. Clear understanding of how PT providers assign values, estimate uncertainties, and score participant results is essential for laboratories to interpret performance, improve methods, and demonstrate competence to accreditation bodies.

Objectives and overview of the leaflet

This leaflet explains the statistical principles and decision rules used by PT providers to assess quantitative laboratory performance. It summarizes how assigned values (x_pt) are determined, options for estimating the uncertainty of the assigned value u(x_pt), methods to set the standard deviation for proficiency assessment (s_pt), the commonly used performance scores (D%, z, z', zeta, En), and the conventional interpretation thresholds for these scores. The guidance is aligned with ISO 13528:2022 and Eurachem recommendations.

Methodology — assigned value strategies

The assigned value x_pt is the reference against which participant results (x_i) are compared. ISO 13528 suggests five general assignment strategies; these are commonly described as:

• Reference (certified) value established independently of participant results (highest metrological confidence).
• Value derived from results of reference or expert laboratories.
• Consensus value from robust statistical treatment of participant results (e.g., robust mean, median).
• Defined or fixed value known a priori (e.g., spike concentration), independent of participants.
• Hybrid or alternative approaches combining elements above.

Key practical point: the PT provider must choose an assignment method consistent with scheme objectives and consider traceability and how u(x_pt) will be estimated.

Uncertainty of the assigned value

The uncertainty u(x_pt) quantifies the standard uncertainty of the assigned value and may be evaluated by methods that correspond to the chosen assignment strategy. Options include:

• Using the certified uncertainty from a reference material or method.
• Propagating uncertainties from calibration/measurement models used by a reference laboratory.
• Estimating the standard deviation of a set of reference laboratory results.
• Estimating standard uncertainty from the dispersion of participant results (with appropriate robust statistics) plus additional components.
• Combining multiple sources of uncertainty using standard uncertainty propagation.

Uncertainty may be reported as a standard uncertainty u(x_pt) or as an expanded uncertainty U(x_pt) = k · u(x_pt) where k is a coverage factor (commonly k = 2 for ~95% confidence). The magnitude of u(x_pt) affects choice of scoring (e.g., whether to use z or z').

Standard deviation for proficiency assessment (s_pt)

ISO 13528 provides five options to set s_pt, the characteristic standard deviation used for performance assessment. These options range from using an a priori performance specification (based on fitness-for-purpose or regulatory limits), to estimating s_pt from robust statistics of participant results, to using values derived from historical data or expert judgment. The chosen s_pt must align with PT scheme goals: either to judge analytical performance against a predefined target or to reflect current interlaboratory variability.

Performance scores and normalisation

Performance scores are normalized differences between the participant result x_i and x_pt. Main scores described are:

• D%: Percent difference normalized to x_pt. Success is assessed against a provider-defined maximum relative permissible error dE% (dE/x_pt).
• z score: (x_i - x_pt) / s_pt. It is unitless and indicates how many s_pt a result deviates from x_pt. Use z' instead of z when u(x_pt) is significant (u(x_pt) > 0.3·s_pt); z' includes u(x_pt) in the denominator.
• zeta (ζ) score: (x_i - x_pt) / sqrt[u(x_pt)^2 + u(x_i)^2]. This compares the numerical difference to the combined standard uncertainty of the assigned value and the laboratory’s reported standard uncertainty. A poor ζ may reflect an inaccurate measurement result, an underestimated measurement uncertainty, or both.
• En score: (x_i - x_pt) / sqrt[U(x_pt)^2 + U(x_i)^2] where U are expanded uncertainties (typically ≈95% confidence). En is frequently used in metrological comparisons between calibration laboratories.

A summary table in the leaflet maps each score to its normalization factor and typical use-case (assessment against performance specification vs. metrological agreement). Figures referenced in the leaflet illustrate assignment and uncertainty estimation options, and the different ways to set s_pt.

Interpretation of performance scores

ISO 13528:2022 conventional thresholds for z, z' and ζ scores are:

• |score| ≤ 2.0: acceptable (satisfactory) performance.
• 2.0 < |score| < 3.0: warning (questionable) performance.
• |score| ≥ 3.0: unacceptable (action) signal.

Analogously, |En| < 1 and |D%| < dE% indicate satisfactory performance. These decision limits are widely accepted but laboratories and PT providers should agree on scheme-specific criteria where justified.

Main results and discussion

The leaflet consolidates best-practice recommendations for PT result evaluation: the choice of x_pt and s_pt must reflect scheme purpose, and the method for estimating u(x_pt) must be transparent. When u(x_pt) is non-negligible relative to s_pt, standard scoring must be adjusted (use z' or ζ) to avoid misleading conclusions. Emphasis is placed on interpreting failing or borderline scores by investigating both result accuracy and uncertainty estimation. The leaflet highlights that different scores provide complementary information: z/z' and D% evaluate numerical agreement relative to an interlaboratory dispersion or specification, while ζ and En explicitly incorporate uncertainty budgets and thus are more appropriate for metrological comparisons.

Practical benefits and applications

• Supports laboratories in interpreting PT reports correctly and prioritizing corrective actions (method bias vs. uncertainty estimation).
• Helps PT providers design schemes aligned with accreditation and regulatory needs by selecting appropriate x_pt, u(x_pt) methods and s_pt values.
• Enables transparent communication with accreditation bodies about the metrological basis of PT decisions.
• Facilitates targeted method improvement and quality control by differentiating between disagreement due to bias and disagreement due to poor uncertainty estimation.

Future trends and possibilities for use

• Stronger integration of robust statistical methods and uncertainty propagation in assigned value estimation, especially for consensus-based x_pt.
• Increased use of uncertainty-aware scores (ζ, En) in accreditation and interlaboratory comparisons as laboratories improve uncertainty reporting.
• Adoption of harmonized, scheme-specific performance specifications (s_pt) tied to clinical, environmental or regulatory fitness-for-purpose criteria.
• Growing automation and digital reporting of PT results combined with machine-readable uncertainty metadata to enable large-scale trend analysis and benchmarking.
• Expanded use of reference laboratories and certified reference materials to strengthen metrological traceability in PT schemes.

Conclusion

This leaflet provides concise guidance on how PT providers assign values, estimate uncertainties, set s_pt, and apply statistical scores to assess quantitative laboratory performance. Understanding the differences among assignment methods and scoring rules is essential for correct interpretation of PT outcomes. Laboratories should consider both numerical agreement and uncertainty conformity when responding to unsatisfactory or borderline results. Transparent documentation by PT providers of x_pt, u(x_pt) and s_pt justifications improves the value of PT for quality assurance and accreditation.

Reference

1. ISO 13528:2022, Statistical methods for use in proficiency testing by interlaboratory comparison.
2. Brookman B., Mann I. (eds.), Eurachem Guide: Selection, Use and Interpretation of Proficiency Testing (PT) Schemes, 3rd ed., Eurachem, 2021.
3. Eurachem leaflet: How can proficiency testing help my laboratory.
4. Eurachem leaflet: Understanding PT statistics.