Application of Sigma metrics in the quality control strategies of immunology and protein analytes

Abstract Background Six Sigma (6σ) is an efficient laboratory management method. We aimed to analyze the performance of immunology and protein analytes in terms of Six Sigma. Methods Assays were evaluated for these 10 immunology and protein analytes: Immunoglobulin G (IgG), Immunoglobulin A (IgA), Immunoglobulin M (IgM), Complement 3 (C3), Complement 4 (C4), Prealbumin (PA), Rheumatoid factor (RF), Anti streptolysin O (ASO), C‐reactive protein (CRP), and Cystatin C (Cys C). The Sigma values were evaluated based on bias, four different allowable total error (TEa) and coefficient of variation (CV) at QC materials levels 1 and 2 in 2020. Sigma Method Decision Charts were established. Improvement measures of analytes with poor performance were recommended according to the quality goal index (QGI), and appropriate quality control rules were given according to the Sigma values. Results While using the TEaNCCL, 90% analytes had a world‐class performance with σ>6, Cys C showed marginal performance with σ<4. While using minimum, desirable, and optimal biological variation of TEa, only three (IgG, IgM, and CRP), one (CRP), and one (CRP) analytes reached 6σ level, respectively. Based on σNCCL that is calculated from TEaNCCL, Sigma Method Decision Charts were constructed. For Cys C, five multi‐rules (13s/22s/R4s/41s/6X, N = 6, R = 1, Batch length: 45) were adopted for QC management. The remaining analytes required only one QC rule (13s, N = 2, R = 1, Batch length: 1000). Cys C need to improve precision (QGI = 0.12). Conclusions The laboratories should choose appropriate TEa goals and make judicious use of Sigma metrics as a quality improvement tool.


| INTRODUC TI ON
Detection of immunology and protein analytes is widely conducted in medical laboratories in China. How to ensure test performance, provide patients with accurate and reliable results, and provide support for doctors' diagnosis and treatment are the primary goals of medical laboratories. For this reason, Six Sigma is obviously a rare good tool for quality control (QC). Sigma, which has a statistical appellation-"Standard Deviation," represents the data dispersion. 1,2 As we all know, the higher the Sigma value is, the better the quality is. In medical laboratories, Sigma metrics have been widely used for the quality control of the whole clinical test processes, including pre-3 , during-4 and post-analytical 5 phases. A scientific and reasonable quality control strategy of medical laboratories can be achieved by combining Sigma quality management with Westgard multirules quality control charts.
Quality control is an important part of clinical laboratory management. As a commonly used quality management tool, Six Sigma management program can effectively evaluate the performance indicators of analytes, help the laboratories find problems in time.
The "Six" in Six Sigma represents the ideal goal that anything beyond those tolerance specifications is considered a defect. 1 6σ means 3.4 defect per million with world class performance, while 3σ means 66800 defect per million with marginal performance, that is, for example, if the minimum standard of quality control is set at the 3σ level, 66800 out of 1 million human immunodeficiency virus (HIV) carriers may be misdiagnosed. Therefore, the minimum standard set at the 3σ level is not fully applicable to clinical laboratories. The laboratory needs to increase the value of Sigma, minimize those defects, and increase the probability of error detection.
Six Sigma quality management provides a new perspective for the quality control strategy of clinical laboratories. Not only through Six Sigma can we identify whether our methods are appropriate for clinical but also it can help determine the QC rules, guide our risk management efforts.
In this study, we evaluated the performance of 10 immunology and protein analytes by calculating their Sigma values based on Bias%, CV%, and four different sources of TEa%. Based on the calculated Sigma value, the QC strategies were personalized and Sigma Method Decision Charts were established. Improvement measures of analytes with σ below 6 were recommended according to the quality goal index (QGI).

| Study design
This study included four steps ( Figure 1). The study was conducted in the Laboratory Department of Guangdong Provincial Hospital of Chinese Medicine from January 1 to December 31, 2020. According to the formula Sigma = (TEa%-Bias%)/CV%, 6 we first determined the source of TEa%, CV%, and the sample used for Bias% calculation, second calculated the Sigma, and then Sigma Method Decision Charts were constructed, and finally the QGI analysis and corrective actions were performed to find and eliminate the potential causes of poor clinical performance of the analytes.

| Evaluation of bias
Bias is an indicator of systematic errors. The laboratory was involved in the EQA program by analyzing five different concentration proficiency test (PT) samples provided by NCCL. PT samples were dissolved in pure water according to the NCCL's instructions, each PT sample was tested for 3 days to obtain three results, and the mean of the three results was calculated and assigned as "Measured mean in our laboratory." NCCL groups the submitted data according to the instruments or reagent manufacturers that participants use, and takes the ISO13528 robust average value in the group as the target mean.
Seven analytes (C3, C4, IgG, IgM, IgA, ASO, and RF) were grouped according to the instrument, while the other three analytes (CRP, PA, and Cys C) were grouped according to reagent manufacturers due to non-Roche reagents. The target mean assigned by NCCL of each analyte was considered as target value. Excluding unqualified PT data (Data exceeding two standard deviations of the mean), the calculation equation was as follows 10,11 : Bias%=│Measured mean in our laboratory-Target mean assigned by NCCL│/(Target mean assigned by NCCL)×100%.
The average bias of each analyte, which was calculated by 1-year accumulative bias of each analyte sourced from the NCCL plans in 2020, was used in the calculation of Sigma.

| Allowable total error
TEa (or "total allowable variation") represents the allowable difference between measured value and trueness. Four different TEa targets were used in this study: (Ⅰ) TEa derived from the quality goals issued by the China National Center for Clinical Laboratories (NCCL) in 2017, 12 designated as TEa NCCL . (Ⅱ, Ⅲ, Ⅳ) the biological variation database specifications (minimum, desirable, optimal), designated as TEa BVmin , TEa BVdes and TEa BVopt . BV provided by EFLM.
The TEa BV are calculated using the formula: Here CV I means CV within-subject, CV G means CV between-subject.

| Composition of Sigma Method Decision Charts
Logging in the NCCLnet (https://www.nccl.org.cn/loginCn), entering TEa%, Bias%, and CV% of each analyte which is obtained through the above steps in the interface of the Six Sigma management menu, the Sigma Method Decision Charts are composed with CV%/TEa% along the x-axis and Bias%/TEa% along the y-axis. 10

| Quality goal index ratio
The QGI ratio was calculated from the analyte with a Sigma<6. Cumulative Mean, SD, and CV% of two IQC levels were shown in Table 2. The RMS CV% ranged from 1.83% (IgG) to 4.96% (Cys C).
The EQA data, which were all falling within ±2SD of the mean, were all satisfactory. The average bias values were displayed in Table 2.
They ranged from 0.93% (Cys C) to 4.42% (C3). Cys C had the highest CV% and lowest Bias%. Four different source of TEa were also shown in Table 2. Mean assigned by NCCL and relative bias% were shown in Table S1.

TA B L E 1 Westgard Sigma multi-rules
Sigma value Rules adopted

| Composition of Sigma Method Decision Charts
We constructed Sigma Method Decision Charts of the ten analytes as per TEa NCCL (Figure 2). Nine out of ten analytes were displayed in the Six Sigma zone, while Cys C appeared in the 3σ-4σ zone. This chart could provide us with a visual view of the analytes' performance. We could intuitively judge the performance of the analytes through this chart.

| Resetting QC strategies and improvement measures
QC strategies were reset according to σ NCCL and the QGI of analytes with σ<6 were calculated (

| DISCUSS IONS
In this study, we evaluated the performance of 10 immunology Therefore, we would not include TEa CLIA in subsequent calculations.
The Sigma metrics of 10 immunology and protein analytes were shown in Table 3 based on four different TEa standards.
As mentioned above, surprisingly, while using the TEa NCCL ,  procedure with two QC levels once per day for the 10 immunology and protein analytes empirically to supervise the performance of analytes in the past. By using Sigma metrics, nine analytes for σ>6 can safely use the 1 3s procedure with one measurement of two QC levels to gain an appropriate level of analytical quality assurance, avoiding economic costs, and overwork. On the contrary, with the decline in performance, more quality control rules, more different levels of quality controls, and higher quality control frequency are required.
As Cys C in this study, that had "marginal" performance, five multirules (1 3s /2 2s /R 4s /4 1s /6 X ) were needed and the QC frequency could be increased to one control per 45 clinical samples. As Cys C had 38 average daily measurements in our laboratory,one time QC frequency per day is needed.
Though the Westgard Sigma multi-rules provide a scientific and reasonable method for setting QC procedure, it could not fully reflect the precision and accuracy of the method. QGI is a tool to provide easy insights into the reasons for quality errors such as those caused by imprecision, inaccuracy, or both. 21,26 Cys C (QGI = 0.12) had low precision and some action must be taken. We make continu- such as temperature and humidity. In fact, Cys C used in this research is a third-party reagent, not Roche original, the reagent batch is updated so quickly, and the difference between batches is large, resulting in a large CV. Aware of this, our laboratory try our best to use the same batch reagents. We would not replace the new batch reagents until the expiration date. In fact, combined with the increase calibration frequency, our laboratory has reduced the average CV of Cys C to below 4% from January to July 2021, leading improvement of precision. Using Westgard Advisor subfunction of BIO-RAD Unity Real Time, Cys C has higher probability of error detection (Ped), from 0.5 using 1 3s /2 2s QC procedure increased to 0.995 using 1 3s /2 2s / R 4s /4 1s /6 X rules, but also higher probability of false rejection (Pfr), from 0.006 using 1 3s /2 2s QC procedure increased to 0.095 using 1 3s /2 2s /R 4s /4 1s /6 X rules. In future studies, the other aspects should be prioritized to generate more conclusive results.
Nevertheless, there were three aspects of limitations in this research as follows: (Ⅰ).
Because the target means in PT/EQA plans were derived from statistical results of peer groups without measurement traceability, 21,27 those using this approach to calculate bias should be aware of possible limitations, including statistical methods used to generate the data and the number of laboratories that participate. There may be increased imprecision due to the small number of laboratories. There also might be concerns about the commutability of PT samples, 28 because they are not the same as real patient specimens.
(Ⅱ) Another weakness is that the bias evaluation in our research (analyze PT samples at five different concentrations daily for 3 days to obtain 15 results) was not conducted strictly to the method described in the CLSI EP15-A2 document (analyze one run per day with 3 replicate samples at each of different concentrations daily for 5 days to obtain 15 results), which may have underestimated bias. 29 (Ⅲ) The performance specification used to benchmark the methods This research can provide support for laboratories to select detection systems for these 10 analytes and aid individual laboratories in their choice of proper TEa goals and in working out a detailed troubleshooting action plan as a part of their quality improvement tool. Laboratory staffs can use these tools to help them select highquality products, further contributing to the delivery of excellent quality healthcare for patients. The result is more efficient instrument operation, more optimized laboratory workflow, and more reliable test results, ultimately helping clinicians better diagnose and treat their patients.

CO N FLI C T O F I NTE R E S T
None declared.

AUTH O R CO NTR I B UTI O N S
YL and SO designed the study, drafted the work, analyzed and interpreted the data, and wrote this article; XY, QX, YL, and JP searched the literature, performed the experimental procedure; QL, YC, YC, and HZ supervised this study and reviewed this article; CC reviewed this article. All authors have read and approved the final manuscript.

DATA AVA I L A B I L I T Y S TAT E M E N T
Some or all data generated or used during the study are available on NCCLnet (https://www.nccl.org.cn/mainCn), the Login account and password are private that cannot be shared. The relevant data in the article are available from the corresponding author (Email: ousong-bang@126.com) by request.