Analytical Performance Evaluation of Hematology Analyzer Using Various TEa Sources and Sigma Metrics

Introduction

Clinical laboratories are dynamic and complex organizations that have a critical role in patient diagnosis, treatment, and management.¹ In clinical medicine, it’s critical to standardize and harmonized testing procedures to deliver precise, timely, and accurate results.² This improves the clinical decision making and allows good adherence to available guidelines. However, standardization and harmonization of total testing process (TTP) is challenging.² Studies have shown that laboratory defects are more common in the pre- and post-analytical phases than in the analytical phase.^1,2

Advanced laboratory technologies and automation have reduced the incidence of defects during the analytical phase.³ Also, the frequency of defects in the analytical phase has decreased as a result of the application of quality control mechanisms such as internal quality control (IQC) and external quality assurance (EQA).⁴ Nevertheless, advance technologies, automation, and application of quality control have not been achieved intended improvement paralleled with a similar reference method. As a result, assessing analytical performance of hematology analyzer using direct and comprehensive assessment tool such as sigma metrics is important.^2,5

Sigma metrics is widely accepted summary process assessment tools. It was initially proposed for industry by the Motorola Company in the 1980s. At later it also applied as performance assessment tool in clinical laboratories.^4,6 Sigma metrics quantifies the degree to which a particular process deviates from the accepted international specifications.⁷ Sigma-metric quantifies the performance of a process in the laboratory as a rate of defects per million opportunities (DPMO).⁸ Assessment of the analytical performance of a laboratory in terms of sigma metrics is more meaningful than assessing the number of defects alone because, based on the sigma metrics value; the laboratory optimizes its IQC plan, identifying the number and frequency of IQCs needed for clinical uses.^6,8

Six-sigma performance attainment requires 3.4 DPMO. It indicates the performance of the laboratory as world-class.⁹ In the process to be applied, a sigma value of three is the minimum acceptable value.¹⁰ A lower sigma metrics values indicates higher defects, and many acceptable test results are falsely rejected, which makes it more difficult to use in the analysis of patients samples. A higher sigma metrics values, on the other hand, indicates fewer defects and fewer falsely rejected acceptable test results.¹¹

A hematology analyzer is an instrument used to perform a complete blood count (CBC) test. It is used for the counting of blood cells, measures hemoglobin, measures hematocrit, and calculates blood cell indices. Consequently, the laboratory must ensure that instrument performance is enough to provide good-quality test results.^4,12

The studies conducted in different areas show that the sigma level of hematology analyzer is inconsistent for common hematological parameters. For example, the studies done in India,¹³ Indonesia,¹⁴ Pakistan,¹⁵ Peru,¹⁶ Romania,¹⁷ Turkey,¹⁸ and the United States¹⁹ showed poor to world-class sigma values for common hematological parameters. The aim of this study is to evaluate the sigma metrics performance levels of hematology analyzer at study setting.

Materials and Methods

The study was conducted at the University of Gondar, Comprehensive Specialized Hospital, Hematology Laboratory. The laboratory was assessed for accreditation, but its performance was poor.²⁰ In the laboratory, the CBC test is performed by an automated hematology analyzer (Beckman Coulter, Unicel DxH 800). The IQC data having the same lot number were included prospectively from August 1 to October 30, 2022. In the laboratory, every morning, three commercially available IQC materials (L1, L2, and L3) provided by the trademarks of Beckman coulter, Inc, USA (A63013-AF, code; L1; 123174000, L2; 133184000, L3; 14319400) were analyzed once a day for CBC analysis. The laboratory has not participated in the EQA program.

All daily IQC data and manufacturer target value data of CBC parameters included in this study were consecutively collected during study period. The daily IQC data of CBC parameters were collected by two trained laboratory professional’s using record format on daily bases. In addition, the professionals collected the manufacturer target value data form insert kit. The TEa were obtained from different sources to calculate sigma value of laboratory using different guidelines.²¹ The source of TEa used to determine sigma value of the laboratory were standards of European Federation of Clinical Chemistry and Laboratory Medicine (EFLM) 2022 minimum,²² State of the Art (SOTA),²³ Spanish Minimum External Quality Assurance (MIN SPANIH EQA),²⁴ Clinical Laboratory Institute Amendment (CLIA),²⁵ and Guidelines of the German Federal Medical Council (Rilibak)²⁶ (Table 1).

Table 1 Sources of TEa Specifications Used to Calculate Sigma Metrics

The mean and standard deviation of IQC data were calculated using Microsoft excel software. The CV of all CBC parameters was calculated from the observed mean and SD of the daily IQC data using formula: .¹³ The bias of all CBC parameters was calculated from the observed mean of the daily IQC data and target value of manufacturer in the insert kit using formula: . Finally, sigma value of each CBC parameter were calculated separately based of different TEa sources using the formula: .²⁷ The average sigma value of each parameter was the mean sigma value of three levels of quality control material.

Furthermore, based on the calculated value of sigma metrics, each CBC parameter had got a proper Westgard rule. A 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) was required for the parameters achieved ≥ 6 sigma value. For parameters, achieved ≥5 sigma value requires 2of 3₂s and R4s rules with one run of each of the three levels of controls (R = 1, N = 3). For parameters, achieved ≥ 4-sigma value, quality requires 2of 3₂s, R₄s and 3₁s rules with one run of each of the three levels of controls (R = 1, N = 3). For parameters, achieved <4-sigma value requires 1₃s, 2of 3₂s, R₄s, 3₁s and 6_x/9_x rules with two run of each of the three levels of controls (R = 2, N = 3) or run three levels of controls with one measurement (R = 1, N = 6) or the day’s work divided into two runs with 3 control measurements per run (N = 3, R = 2). If a 9_x rule were used instead of the 6_x rule, then a day’s work could be divided into three runs with 3 controls per run (N = 3, R = 3).²⁸

Results

Laboratory Means, SD, CV and Bias of IQC Material

In the current study, mean, SD, CV% and bias% were calculated for fifteen CBC parameters. The highest CV (%) value were observed in Basophil (L1: 10.78*103/µL, L2: 12.46*103/µL and L3: 37.77 *103/µL) and Eosinophil (L1: 12.19 *103/µL), while the lowest CV (%) were observed in Hb (L3: 0.52 *103/µL) and MCV (L2: 0.63 *103/µL and 0.60 *103/µL). The highest bias (%) value were observed in Basophil (L3: 12.36 *103/µL), Neutrophil (L3: 8.44 *103/µL) and Monocyte (L1: 7.44 *103/µL), whereas the lowest bias % were observed in WBC (L1: 0.039*103/µL), Hb (L1: −0.04 *103/µL) and MCHC (L2: −0.057 *103/µL). Based on the desirable biological variation database specifications,²⁹ WBC, RBC, Hb, HCT, PLT, Neutrophil, Lymphocyte, Monocyte, Eosinophil and Basophil, CV (%) values obtained were within acceptable limits in all level control. However, RDW was within unacceptable limits in all level of control. In this study, desirable accuracy were observed in WBC, RBC, Hb, HCT, MCH, PLT, Neutrophil, Lymphocyte, Monocyte, Eosinophil and Basophil in all level of control based on the desirable biological variation database specifications (Table 2).

Table 2 The TV, Laboratory Mean, SD, CV and Bias of Three Levels Control Material

Sigma Value of CBC Parameters

In the current study, the sigma metric value of CBC parameters were calculated using TEa from different sources, from 5 parameters TEa obtained from CLIA guideline except HCT all parameters were meet the average sigma value greater than three. Out of parameters achieved >3 sigma value, only WBC and PLT were meet six sigma quality performance. On the other hand, out of 15 CBC parameters TEa obtained from EFLM 2022 minimum guideline, except MCV, MCHC, RDW, MPV and Basophil all were achieved sigma value greater than three. The CBC parameters such as WBC, RBC, Hb, Neutrophil, Lymphocyte and Monocyte were meet average sigma value greater than 6 based on TEa of EFLM 2022 minimum guideline. Moreover in this study, based on the TEa from both Rilibak (five parameters) and Standards of Spanish (seven parameters) guidelines all were achieved average sigma value greater than three. The sigma calculation based on SOTA TEa showed that seven parameters were achieved sigma value ≥3, but remaining were achieved sigma value <3. The CBC parameters such as WBC, RBC and Hb in Rilibak guideline, Hb and MCV in Standards of Spanish guideline, and MCV, MCH and MCHC in SOTA guideline were meet ≥ 6 average sigma values (Table 3).

Table 3 Sigma Metrics of CBC Parameters Using Different Standards

Quality Control Strategy Based on Sigma Values

Based on calculated the sigma value, proper Westgard rule was selected for each CBC parameters using different guideline. Of five parameters included based on CLIA guideline except Hb and HCT for all parameters, 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) were selected. On the other hand, out of 14 parameters included based on EFLM 2022 guideline, 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) were selected for only WBC and RBC. The CBC parameters such as WBC, RBC and Hb had got 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) were selected out five parameters included. In addition, out of seven parameters included based on Standard of Spanish, 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) were selected for only Hb and MCV. Furthermore, out of nine CBC parameters included 1_3S rule with one run of each of the three levels of controls (R = 1, N = 3) were selected only for MCV, MCH and MPV. In the current study, for CBC parameters the Westgard rule was vary based on the guidelines (Table 4).

Table 4 Quality Control Strategy Based on the Sigma Value of the CBC Test Parameters Using Different Guidelines

Discussion

Hematology analyzers are highly complex instrument used to CBC analysis. In fact, CBC results obtained from the analyzers are very important for diseases screening, diagnosis, monitoring and treatment.^30,31 Thus, the hematology laboratory should follow all regulatory requirements to detect analytical errors, test system failure, and environmental factors before patient results are reported to release accurate and precise results on nearly all specimens.^32,33 Consequently, the performance of the CBC analyzer should be checked routinely.³⁴ The aim of the current study is also to assess the performance of hematology analyzer using sigma metrics tool.

In the current study, the accuracy and precision of the analyzers were assessed, compared to standard of desirable biological variation database specifications the study found desirable precision and accuracy for all CBC parameters except RDW. The imprecision and inaccuracy of RDW value might be relate to different reasons.^35,36 One of the reasons may be due to existence of small differences in the TV of commercial controls for RDW. Another possible explanation may related to storage condition of quality control materials in the laboratory, since RDW is reflect of RBC size variation any change during storage of RBC might results size variation. Consequently, the laboratory suggested to follow the right strategies and took possible remedial action for RDW result in order to minimize the wastage and delivery of wrong results that are beyond allowable inherent errors.

Beside, the current study used sigma metrics tool to assess the performance of hematological analyzer using TEa from five different guidelines. Since, there is no harmonized and universally accepted source of TEa.^7,18 The results of the study shows different sigma value can be obtained depending on the TEa values. The current study provided alternative to the laboratory in order to optimize local Westgard rules and find most appropriate TEa.

Using CLIA guideline, the study included five parameters. Out of parameters included except HCT all parameters achieved marginal to word class performance. The studies done in Spain,³⁷ Romania,³⁸ Indonesia,⁴ India⁷ and Egypt,¹ also reported similar finding with current study. The poor performance for HCT may relate to study period. The current study and most of previous studies reported poor performance for HCT used short period to collect IQC data. Even though, clinical and laboratory standards institute recommends at least 3 month data to calculate the bias.³⁹ However, some previous study reported better finding for HCT than current study.^14,15,40 The discrepancy of the result may be due to differences in the calculation of bias, hence those studies used proficiency test data, while current used manufacturer TV from insert kit. In addition, biological variation might contribute for the difference.^32,41

In side of other parameters, the finding of the previous studies,^1,18 is inconstant for WBC, Hb and PLT with compared with current study. The possible reason for the inconsistence of the result could be difference of analysis method, study period, instrument of analysis. In addition, limited stability and scarce volume of pooled samples collected from different subjects may be cause for difference. Beside, the production process may also generate a certain degree of diversity between control material and whole-blood samples routinely used.

On the other hand, in the current study except MCV, MCHC, RDW, MPV and Basophil out of 14 parameters included based on EFLM 2022 minimum guideline and Hb and PLT out of 9 parameters included based on SOTA guideline other parameters meets minimum specification. Similarly, all parameters included in Rilibak and Standards of Spanish guideline achieved minimum specification. The finding of the current study is inconstant with studies done in Spain,³⁷ Turkey,¹⁸ and Pakistan.¹⁵ The discrepancy might be related to difference of storage condition and lot number of IQC materials. Beside, the difference might be related with calculation of bias, analyzers used of analysis of the tests, and other pre-analytical and analytical conditions. As a limitation of this study, the study used manufacturer TV data for calculation of bias and the study was carried out a relatively short duration. Consequently, CV and bias used to calculate sigma metrics may not represent the real image of the laboratory.

Conclusion and Recommendation

The finding of the current study shows sigma value of CBC parameters is inconsistent due to variability TEa source. As a result, harmonized TEa source is necessary to standardize sigma value calculation. Based on the Sigma value, the authors of this study suggest the laboratory directors to use best practical and professional judgment to choose appropriate TEa source and optimize local Westgard rules for each parameter by considering the parameters that have no TEa goal. In general, the laboratory is better to implement periodic and inclusive internal and external audit to take evidence based corrective measures.