See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/343829008

Determination Risk Level Associated to Decision of Conformity to Specification

for Environmental Laboratories

Article · August 2020

DOI: 10.23880/ijoac-16000194

CITATIONS READS

0 415

1 author:

Mohamed H. Shaltout
HCWW
7 PUBLICATIONS 19 CITATIONS

SEE PROFILE

All content following this page was uploaded by Mohamed H. Shaltout on 24 August 2020.

The user has requested enhancement of the downloaded file.

 International Journal of Oceanography & Aquaculture
MEDWIN PUBLISHERS ISSN: 2577-4050
Committed to create value for Researchers

Determination Risk Level Associated to Decision of Conformity to

Specification for Environmental Laboratories

Shaltout M*
Case Report
Quality Assurance Manager, Reference lab of drinking water, Holding Company for Water &
Volume 4 Issue 3
Wastewater, Egypt
Received Date: August 01, 2020
Published Date: August 21, 2020
*Corresponding author: Dr. Mohamed Shaltout, Quality Assurance Manager, Reference lab of
DOI: 10.23880/ijoac-16000194
drinking water, Holding Company for Water & Wastewater, Egypt, Tel: +201270000612 ; Email:
mhshaltout@hotmail.com

Abstract
Before 2017, there aren’t standards obligated analytical laboratories even accredited to take measurement uncertainty into
account in the decision for acceptation or rejection the samples result to a specification, so the decision of conformity for all
results closed to specification limits became doubtable. Now in the recent edition of ISO Standard (ISO/IEC 17025:2017)
decision rules become obligatory for both testing and calibration laboratories, a clause number 7.8.3 titled “Reporting
statement of conformity” stated the laboratories shall document the decision-rule employed and taking into account the risk
level. The environmental measurements are critical and mistakes occur in decision lead to a disaster, so this paper concerned
to explain different methods of decision-rules used and calculate the risk level relevant to each decision through the practical
implementation these decisions by Kafe EL Seikh accredited water laboratory.

Keywords: ISO/IEC 17025/2017, Decision Rules ; Uncertainty ; Specification/ Tolerance Limits ; Statement of Conformity;
Risk Level

Introduction Clause 3.7, illustrated the definition of the decision-

rules with conformity, which is taking the uncertainty of
Previous version of ISO standard (ISO/IEC 17025:2005) measurement into account to stat the conformity with
recommended only to testing laboratories to write a specified requirement [2]. Clause 6.2.6 obligate the
a statement of conformity where necessary for the laboratory to authorized personnel for results analyses
interpretation of the test results. Statement of conformity not including the writing of the conformity statement, opinion
obligatory in this version because this version didn’t confirm and interpretation. Clause 7.1.3 requires when the customer
the implementation in terms of the shall and should but it needs lab to write a statement of conformity to a specification
was mentioned as where relevant (ISO 17025:2005, Clause or standard a binary decision shall be taken ( pass/fail, in-
5.10.3.1b) [1]. and there was lake in references that explain tolerance/out-of- tolerance), the specification or standard
that, especially for testing laboratories. and the decision-rule shall be defined unambiguously in test
report or in calibration certificate unless mentioned clearly
On the other hand, recent ISO Standard (ISO/ in the requested specification or standard, the decision-rule
IEC:17025/2017) obligated both testing and calibration selected shall be discussed , and agreed with, the customer
laboratories to write the statement of conformity in case before the lab initiated customer requests [2-4]. Clause 7.8.3.
of requested by the customers or the competent authority (1c) where applicable, the uncertainty of measurement
and mentioned that at different positions such as resources written in the same unit as absolute value or a relative
and processes related to personnel, contract review and percentage to the measurand, when the value of uncertainty
reporting.

Determination Risk Level Associated to Decision of Conformity to Specification for Environmental Int J Oceanogr Aquac
Laboratories
2 International Journal of Oceanography & Aquaculture

affects the validity of the test results, when requested by a lower tolerance limits and double-sided tolerance limit [6,7].
customer, or when the uncertainty of measurement affects
statement of conformity to specifications. Clause 7.8.6.1
When a statement of conformity to a specification or
standard for test or calibration is required, the laboratory
shall mentioned the decision-rule used also the risk level
(like false of acceptation and false of rejection and statistical
assumptions used) which related to the decision-rule used
[2] .Clause 7.8.6.2 the laboratory shall report the statement
of conformity clearly to identify which results that statement
of conformity implemented; which specifications and
requirements are compiled or not; the decision-rule applied
(unless it is stated in the specification or standard used) [2]. Figure 2a: Guard band for lower tolerance limit decision.
Some analytical laboratories working in the
environmental field, acceptance and rejection decisions are
inaccurate due to they haven’t taken uncertainty in their
decisions due to lack in statistical knowledge. So in this
paper we explain different methods of decision-rules used
and associated risk levels with a practical example for each
method.

Methods for Determination Risk Levels of

Decision Rules
When performing a measurement and subsequently Figure 2b: Guard band for lower tolerance limit decision.
making a statement of conformity, there are two possible
outcomes; correct decision regarding to specification or
incorrect decision is regarding to specification [5].

Figure 2c: Guard band for double side tolerance limit

decision, (1) Tolerance/ specification Zone, (2) Acceptance
Figure 1: Level of risk for different laboratories decision. Limit, (3) Acceptance Zone, (4) Rejection Zone, (5) Guard
band(W) for lower Tolerance Limit, (6) Guard band(W)
As shown in Figure 1 The expanded measurement for Upper Tolerance Limit, (LTL) Lower Tolerance/
uncertainty in the lower result (A) totally lies within the Specification Limit and (UTL) Upper Tolerance/
tolerance limit. The upper result (B) has significantly larger Specification Limit.
measurement uncertainty. The risk of falsely accepting a
result in case B is higher due to the larger measurement
uncertainty, so using of a guard bands (W) can reduce the This document refers to Guard Bands, where the Guard
probability of making an incorrect conformance decision. Band (W) equal the Tolerance/specification Limit (TL) minus
A safety factor built into the measurement decision to the Acceptance Limit (AL)
reduce the acceptance limit below that of the specification/
tolerance limit [5]. There are four cases of decision-rules that W = TL-AL Eq. (1)
often encountered to the workers in field of environmental This means that if the measurement result is below
laboratories; simple acceptance, upper tolerance limits,

Shaltout M. Determination Risk Level Associated to Decision of Conformity to Specification Copyright© Shaltout M.
for Environmental Laboratories. Int J Oceanogr Aquac 2020, 4(3): 000194.
3 International Journal of Oceanography & Aquaculture

the Acceptance Limit (AL), the measurement is accepted risks provided in the following table are also accepted as
as conforming to specification. If measurement exceed conforming to specification, reject otherwise [8,9].
Acceptance Limit (AL) by the amount less than specific

ID Decision rule Guard Band W(r*U) Specific risk Conformance probability (pc)
1 6 Sigma 3U < 1ppm PFA ≥ 99.99999%
2 3sigma 1.5U <0.16 PFA ≥ 99.84%
3 ILAC G8:2009 rule 1U <2.5% PFA ≥ 97.5%
4 ISO 14253-1:2017 [5] 0.83U <5% PFA ≥ 95.0%
5 Simple acceptance 0 <50%PFA ≥ 50%
Item reject for measured value
6 Uncritical -U ≥ 97.5%
greater than AL=TL+U <2.5 PFR
Customers may define arbitrary
7 Customer defined rU multiple of r to have applied as Defined by customers
guard band
Table 1: Specific risk of decision rules used in environmental laboratories.
PFA-Probability of False Accept and PFR- Probability of False Reject (Assumes a single sided specification and normal distribution
of measurement results).

Case (1): Simple Acceptance Possible decision rules for this conformity decision might
therefore be defined in terms of conformance probability(pc)
Customers ask a laboratory to make a conformity or probability false acceptance (PFA) [11,12].
decision that “ignores uncertainty”. In case of uncertainty
ignorance, the guard band length equal to zero, W = 0, Example: What is the lab decision of compliance with
infers that acceptance occur when a measurement result the WHO specification for measuring iron in drinking
is below a tolerance limit. This is called simple acceptance water sample, while the sample result is 0.28 mg/l, WHO
or shared risk because the probability to be outside the specification limit is 0.3 mg/l, expanded uncertainty
tolerance limit may be as high as 50% (risk level is 50%) associated with measuring of iron in this lab is ±8%, lab
in the case when a measurement result is exactly on the using ILAC-G8 decision-rule Table (1).
tolerance limit (assuming a symmetric normal distribution
of the measurements) [10]. According to the accreditation Answer: Decision-rule lab used is ILAC-G8, where guard
requirements this decision is not permitted by ISO/IEC band length equal 1* Expanded uncertainty(U) and the
17025:2017, which require that uncertainty should be accept risk level is <2.5% so the result ACCEPT when 𝑝𝑐 ≥
taken into account (directly or indirectly) when conformity 97.5 %; REJECT otherwise or equivalently, also the result
decisions are made [11]. ACCEPT when 𝑃𝐹𝐴 < 2.5 %; REJECT otherwise”

Case (2): Single Sided Upper Tolerance Limit Frist of all calculate standard uncertainty(u) with the
same unit of measurand by applying the following equation
Conformance probability for an upper tolerance limit is [9-11].
therefore
measured value ( ym ) × Expanded uncertinity (u)
Standard uncertinity ( u ) =
pc= 100- NORM. DIST (TL, ym, SD, TRUE) Eq. (2) K average value ( K = 2 )

Eq (4)
Where: TL is specification limit, ym is lab result, SD standard
deviation which equal standard uncertainty(u) of the result Standard uncertinity (u)=±0.0112mg/l
with the same unit * factor specified in different types of Standard deviation (SD)= r*u=1*0.0112=±0.0112 mg/l
decision rules (r) Table (1). By applying the values of: specification limit TL, lab result 𝑦𝑚,
Standard deviation SD in Conformance probability (pc) Eq.
Probability false acceptance PFA (risk level) is (2), using Excel program
pc = 100-NORM. DIST (0.3, 0.28, 0.0112, TRUE) =96.29%
PFA = 100 – pc Eq. (3) Calculation probability false acceptance using Eq. (3).

Shaltout M. Determination Risk Level Associated to Decision of Conformity to Specification Copyright© Shaltout M.
for Environmental Laboratories. Int J Oceanogr Aquac 2020, 4(3): 000194.
4 International Journal of Oceanography & Aquaculture

PFA = 100 – pc = 3.71% Decision Rules for this conformity might therefore be defined
This result might therefore be reported as: in terms of 𝑝c or 𝑃𝐹𝐴 [12-14].
Example: Dissolved oxygen(O2) of a river was 2.2 mg/L as
“REJECT, with a conformance probability of only 96.29% O2, dissolved oxygen needed for fish living ranges from 2 - 6
which does not meet acceptance criterion of 𝑝c ≥ 97.5%” or mg/l, expanded uncertainty associated with O2 measurement
“REJECT with a probability false acceptance of 3.71%, unable is ±6% and the decisions rule lab always uses is 3 Sigma
to meet acceptance criterion of PFA < 2.5%” Table (1).

Case (3): Single Sided Lower Tolerance Limit Answer: Decision-rule which lab used is 3 Sigma where
guard band length equal.
Conformance probability(pc) for lower tolerance limit is 1.5* expanded uncertainty(U) and the accept risk level is
therefore [11,12]. <0.16% so the result ACCEPT when 𝑝𝑐 ≥ 99.84 %; REJECT
otherwise or equivalently, also the result ACCEPT when 𝑃𝐹𝐴
pc = NORM. DIST (TL, ym, SD, TRUE)Eq. (6) < 0.16 %; REJECT otherwise”
Calculate standard uncertainty as the previous examples
Probability false acceptance PFA (risk level) is u= ± 0.066 mg/l
Standard deviation (SD) = 1.5 * u= 1.5 * 0.07= ±0.10 mg/l
PFA = 100 – PFA Eq. (7) pc = NORM. DIST (6, 2.2, 0.10, TRUE) −NORM. DIST (2, 2.2,
0.10, TRUE) = 97.83%
Example: Dissolved oxygen(O2) of treated sewage plant PFR= 2.17%
was 4.2 mg/L as O2, specification limit lab follows to
determine statement of conformance is O2 not less than 4 This result might therefore be reported as:
mg/l, expanded uncertainty associated with O2 measuring “REJECT, with a conformance probability of only 97.83%
is ±6% and the decisions rule lab always uses is ISO 14253- which does not meet acceptance criterion of 𝑝c ≥ 99.84%” or
1:2017[5] Table (1). “REJECT with a probability false acceptance of 2.17%, unable
to meet acceptance criterion of PFA < 0.16 %”
Answer: Decision-rule which lab use is ISO 14253-
1:2017[5] where guard band length equal 0.83* expanded Conclusion
uncertainty(U) and the accept risk level is < 5.0% so the
result ACCEPT when 𝑝𝑐 ≥ 95.0 %; REJECT otherwise or Laboratories working in the environment field must
equivalently, also the result ACCEPT when 𝑃𝐹𝐴 < 5.0 %; accurately determine the uncertainty values associated
REJECT otherwise Calculate standard uncertainty(u) as the with the tests results due to its extreme importance in the
previous example different decision-rules and also, they must apply decision-
rules methods whether or not they are internationally
u= ± 0.126 mg/l accredited unless it is inherent in the requested specification
Standard deviation (SD)= 0.83 * u= 0.83 * 0.126= ±0.104 or standard.
mg/l
pc = NORM. DIST (4, 4.2, 0.104, TRUE) = 97.21% Highlights
PFA = 2.79%
This paper concerned to explain different methods of
This result might therefore be reported as: decision-rules used and calculate the risk level relevant to
“Accepted, with a conformance probability (𝑝c) of only each decision through the practical implementation these
97.21% which meet acceptance criterion of 𝑝c ≥ 95.0%” or decisions by Kafe EL Seikh accredited water laboratory.
“accepted with a probability false acceptance of 2.79%, meet
acceptance criterion of PFA < 5.0%” References

Case (4): Double Sided Tolerance Limit 1. Ricci U (2014) Establishment of an ISO 17025:2005
accredited forensic genetics laboratory in Italy, Accred
Conformance probability for a two-sided tolerance limit is Qual Assur 19: 289-299.
therefore.
pc = NORM. DIST (TL(max), ym, SD, TRUE) − NORM. DIST 2. (2017) ISO/IEC 17025 General requirements for the
(T(min), ym, SD, TRUE) Eq. (8) competence of testing and calibration laboratories.
International Organization for Standardization/
PFR= 100-Pc Eq. (9) International Electro-Technical Committee, Geneva.

Shaltout M. Determination Risk Level Associated to Decision of Conformity to Specification Copyright© Shaltout M.
for Environmental Laboratories. Int J Oceanogr Aquac 2020, 4(3): 000194.
5 International Journal of Oceanography & Aquaculture

3. (2001) ASME B89.7.3.1 (2001) Guidelines for Decision 9. (2012) ISO/IEC Guide 98-4, Uncertainty of
Rules: Considering Measurement Uncertainty in measurement-part 4: role of measurement uncertainty
Determining Conformance to Specifications. The in conformity assessment. International Organization
American Society of Mechanical Engineers. for Standardization, Geneva.

4. (2003) ISO 10576-1 (2003) Statistical methods- 10. (2008) ISO/IEC Guide 98-3, Guide to the expression of
Guidelines for the evaluation of conformity with uncertainty in measurement.
specified requirements.
11. ASME (2005) B89.7.4.1, Measurement Uncertainty and
5. (2019) ILAC-G8:09 (2019) Guidelines on Decision Rules Conformance Testing: Risk Analysis. American Society of
and Statements of Conformity. Mechanical Engineers.

6. (2017) BS EN ISO 14253-1 (2017) Geometrical product 12. Tsimillis KC (2018) Measurement uncertainty:
specifications (GPS)-Inspection by measurement of requirements set in the accreditation standards, Accredit
workpieces and measuring equipment Part 1: Decision Qual Assur 23(253): 109-114.
rules for verifying conformity or nonconformity with
specifications. 13. Ricci U (2014) Establishment of an ISO 17025:2005
accredited forensic genetics laboratory in Italy. Accred
7. UKAS (2019) M3003, The Expression of Uncertainty and Qual Assur 19: 289-299.
Confidence in Measurement, 4th (Edn.), United Kingdom
Accreditation Service. 14. Silva GMP, Nogueira R, Faria ACO (2014) the lead
assessor role in the ISO/IEC 17025:2005 accreditation
8. UKAS (2020) LAB 48, Decision Rules and Statements of of Brazilian calibration and testing laboratories by the
Conformity, 2nd (Edn.), United Kingdom Accreditation General Coordination of Accreditation (Cgcre), Accred
Service. Qual Assur 19: 127-132.

Shaltout M. Determination Risk Level Associated to Decision of Conformity to Specification Copyright© Shaltout M.
for Environmental Laboratories. Int J Oceanogr Aquac 2020, 4(3): 000194.

View publication stats