Use of uncertainty in compliance

When test results are used to assess compliance, i.e. to decide whether specifications or regulations are met, the measurement uncertainty of the test results has to be taken into account. Assessment of compliance for cases i and v in the Figure below is clear – the measurement results including the uncertainty interval are clearly above or below the limit value. In the other cases the decision is not clear since the uncertainty interval overlaps the limit value. The Eurachem/CITAC guide [1] gives guidance on cases ii, iii and iv. 

Eurachem: Upper control limit

We need acceptance & rejection zones 

In order to judge whether the results in cases ii, iii, and iv comply with the limit value we need a decision rule, based on the risks associated with making a wrong decision. This decision rule enables a guard band, g, to be calculated which defines an acceptance zone and a rejection zone. If the measurement value is within the acceptance zone the specifications are met and we can conclude compliance. If the measured value is in the rejection zone, we can conclude non-compliance. The intersection between these two zones is called the decision limit. A guard band is normally chosen so that for a measured value in the acceptance zone the probability of correct acceptance is more than or equal to a defined confidence value α. Note that a guard band, g = 0 can also be used. This is called simple acceptance. 

Decision rule 

A decision rule should have a well-documented method of determining the location of acceptance and rejection zones, ideally including acceptable levels of probability, P, that the value of the measurand 1) lies within the specification limit, high confidence of correct acceptance (low probability of false acceptance) or 2) lies outside the specification limit, high confidence of correct rejection (low probability of false rejection).

Information needed for assessing compliance 

The following information is needed: 

• measurand clearly specified; 
• specification stating upper or lower limit or both; 
• decision rule; 
• measured value: 
• measurement uncertainty for a measured value at the limit(s).

Example focusing on correct acceptance 

• Measurand: Mass fraction of nickel, Ni in a batch of steel delivered to a customer. 
• Uncertainty: U = 0.2 % Ni, k = 2 (95 %). Standard uncertainty, u = 0.1 % Ni. This uncertainty includes both sampling uncertainty for the batch and analytical uncertainty. 
• Specification: The specification zone is from the lower limit 16.0 % Ni to the upper limit 18.0 % Ni. 
• Decision rule High confidence of correct acceptance: The acceptance range is the mass fraction where it can be decided with a confidence level of approximately 95 % (α = 0.05) that the batch has a mass fraction above the lower limit and below the upper limit. 
• Guard band: Each guard band is calculated as 1.64u ≈ 0.17 % (rounded upward for safety) with k value 1.64 from the one-tailed upper 95 % quantile of the normal distribution. 
• Acceptance zone: 16.2 % Ni to 17.8 % Ni , after rounding to one decimal place. 
• Measured value: 16.1 % Ni

Eurachem: Guard bands (g), and acceptance and a rejection zones based on a lower and upper limit and a decision rule stating high confidence of correct acceptance

The measured value, 16.1 % Ni, is below the lower acceptance limit of 16.2 %; that is, in the rejection zone. The batch is non-compliant. 

Note – if the decision rule stated simple acceptance the acceptance zone would be 16.0 % to 18.0 % and the batch would be compliant. 

Example focusing on correct rejection 

• Measurand: Mass fraction of a banned substance in a sample. 
• Uncertainty: The relative standard uncertainty urel is 35 %. 
• Specification: Upper limit is 2 ng/g. 
• Decision rule High confidence of correct rejection: The concentration of the banned substance will be deemed to be above the limit if the probability of the value of the concentration being greater than the limit is 95 % or greater.
• Guard band: The guard band g for correct rejection is 1.6 ng/g assuming a lognormal distribution (due to the high relative uncertainty – see further Guide Annex A, Case 4 [1]). 
• Acceptance limit: 3.6 ng/g 
• Measured value: 3.3 ng/g 

Eurachem: Guard band (g), and acceptance and a rejection zones based on a decision rule stating high confidence of correct rejection

The measured value, 3.3 ng/g, is below the acceptance limit of 3.6 ng/g; that is, in the acceptance zone. The sample is compliant. Note that the assumption of the type of distribution is crucial. If assuming a normal distribution in this case the acceptance limit would be 3.2 ng/g and the sample would not be compliant.