A study of the moving rate of positive results for use in a patient‐based real‐time quality control program on a procalcitonin point‐of‐care testing analyzer

Abstract Objective To establish an applicable and highly sensitive patient‐based real‐time quality control (PBRTQC) program based on a data model constructed with patients’ results of a procalcitonin point‐of‐care testing (POCT) analyzer. Methods Patients’ results were retrospectively collected within one year. The Excel software was used to establish quality control (QC) programs of the moving average (MA) and the moving rate of positive results (MR). A Monte Carlo simulation was used to introduce positive and negative biases between 0.01 and 1 ng/ml at random points of the testing data set. Different parameters were used to detect the biases, and the detection efficiency was expressed using the median number of patient samples affected until error detection (MNPed). After comparing the MNPeds of different programs, MA and MR programs with appropriate parameters were selected, and validation plots were generated using MNPeds and maximum number of the patient samples affected (MAX). β curves were generated using the power function of the programs, the performances were compared with that of the conventional QC program. Results Neither the conventional QC nor MA program was sensitive to small bias, While MR program can detect the minimum positive bias of 0.06 ng/ml and negative of 0.4 ng/ml at an average daily run size of 10 specimens, with FRs < 1.0%, βs < 1%. Conclusion The MR program, which is more sensitive to small biases than conventional QC and MA programs, with low FR and β. As such, it can be used as a PBRTQC program with high performance.

as internal QC products, 6 the precision of the sample concentration and the extremely high requirements for laboratory storage render this approach infeasible in most primary laboratories. For this reason, a patient-based real-time quality control (PBRTQC) method has been proposed for the QC analysis of population data by calculating the mean, median, or standard deviation of real-time results. 7,8 Among the PBRTQC methods, the moving average (MA) program is the most widely used. However, the MA program cannot rapidly detect all types of biases such as small biases. 9 As one of the early clinical diagnostic indicators of infection, procalcitonin (PCT) is a highly specific and sensitive biomarker. 10 However, it associates with systematic errors due to factors such as instrument failure, poor operator habits, or changes in reagents and environment, all of which can affect clinical judgment. The sensitivity of the QC program determines whether small system errors can be rapidly detected so that appropriate measures can be quickly implemented. Liu et al. 11 demonstrated that the moving sum of the number of positive patient results for prostate-specific antigen, as the QC procedure, can rapidly detect a positive bias of 0.03 mg/L, which is impossible with conventional QC and MA programs.
However, in previously published studies, [12][13][14] the project clinical decision point was taken as the parameter of the PBRTQC program. In this study, we identify optimal parameters to detect small errors for the programs through simulations based on the data model, strive to shorten the time required for the QC programs to detect systematic errors under the conditions that both type I and II errors are within acceptable limits, and ensure the clinical accuracy of the POCT programs by monitoring the quality of PCT detection.

| General information
The results of 2434 PCT samples tested in a laboratory at the Fifth People's Hospital of Panyu District, Guangzhou from July 2019 to June 2020 were retrospectively collected. The results of 20 proficiency samples and 146 internal QC (IQC) samples were excluded. The rule of 4 1S /1 3S /2 2S was followed by the laboratory, and no out-of-control points caused by PCT analyzers, methods, or reagents were observed in the IQC chart during this period. The PCT results were verified by SPSS 22 software (IBM, Armonk, NY, USA), and the data showed a skewed distribution (p < 0.05). Data simulation analysis was performed by Excel 2007 software (Microsoft, Redmond, WA, USA). After considering the small specimen size on weekends and holidays, the daily run size for weekdays was set to approximately 10 specimens/day.

| Instruments and reagents
The TZ-301 analyzer (ReLIA, Shenzhen, China), the PCT detection kit (ReLIA) with a minimum detection limit of 0.02 ng/ml, and IQC analytes (Acusera series, RANDOX, city, UK) were used in this study.

| Efficacy of traditional IQC to detect biases
The Westgard 4 1S /1 3S /2 2S multi-rule method was used weekly to analyze the IQC analytes of two concentrations. The mean values of the 12-month QC results were 1.50 ng/ml and 19.88 ng/ml, respectively, and the analytical coefficients of variation (calculated as standard deviation/mean, SD/mean, CV) were 4.40% and 3.98%, respectively. We assumed that the IQC concentrations obeyed a normal distribution. Thus, to obtain the probability of a critical bias triggering a QC rule, we calculated the standard z value for the probability of a QC result greater than χ SD (χ = 1, 2, or 3) in presence of a critical bias as follows 12 : which could be expressed as: where Mean old and Mean new are the averages of the QC concentrations before and after a bias was introduced, respectively. This could be expressed as follows: Mean new = Mean old + critical bias. For example, the z value for the probability that the 1 2S rule was triggered in the presence of a 0.05 ng/ml bias via a QC analyte with low concentration of 1.50 ng/ml was: After consulting the z-table, we obtained p = 0.885, and the QC result greater than 2SD was (1-p) ×100% = 11.5%, and the probability of two consecutive QC results greater than 2SD (i.e., 2 2S rule) was (1-p) 2 × 100% = 1.32%. Similarly, when N = 1, the z value was 0.2346, p = 0.593, and (1-p) × 100% = 40.7%, so the probability of obtaining four consecutive QC results greater than 1SD (i.e., 4 1S rule) was (1-p) 4 × 100% = 2.75%.

| Determination of MA program parameters
The MA program was set up in three parts: (1) the exclusion of values above or below a certain threshold by applying a truncation limit (TL), (2) the MA calculation method, which included the MA algorithm and block size (N), was defined as the number of patient results to be averaged in the algorithm; and (3) the control limit (CL). The TL was used to minimize the impact of the extreme results on the dataset, which could reduce the false rejection rate (FR). 15 The MA program was expressed as follows: where Z (α) is the calculated average value of the PCT result, and X (α) is the result of sample α. According to the time series data, the MA program continuously operated on a term-by-term basis and calculated the sequential N average, including a certain block size. Each time a new result was merged into the block, the oldest result was discarded, and the average value of the block was recalculated for comparison with the predefined CL. The CLs were set using the mean and SD of the MA as follows:

| Determination of MR program parameters
The MR program also consisted of three parts, but instead of using the TL to smooth the dataset, the MR program used a cut-off value (COV) for the binary conversion of the data set. The returned state was "0" when the original value was ≤COV; otherwise, the returned state was "1." The program was as follows: where MR (α) is the operating moving rate within a block, and T (α) is the  MNPeds were plotted as bar graphs, and MAX values were plotted as error lines. Small values of MNPed, FR, and β were expected for the PBRTQC program, and ideally, errors were detected in the daily run size, that is, MAX error lines did not exceed the daily run size.

| Efficacy of conventional QC procedures to detect biases
The probability of triggering different QC programs by introducing biases between 0.01 and 1.0 was calculated, and data are shown in

| Determination of MA and MR program parameters
As the programs monitored the layout of the data sequence rather than the patient's clinical background, 16 instead of using the clinical decision points, in the MA program, we expanded the selection Note: The conventional QC program uses 1.50 ng/ml quality control product to trigger the 4 1 S rule first, which requires a bias of more than 0.24 ng/ml, with 95.5% probability.
Abbreviation: QC, quality control. range of the TL to an interval, that is, the population mean ± i × SD, and measured the most sensitive parameters within the selected interval. In addition, due to the large overall patient SD (4.21 ng/ ml), a smaller i value was required to converge the discrete degree. When i = 0.1, the data rejection rate was 85.57%, which was too high, rendering the monitoring program ineffective. When i = 0.2-0.8, the data rejection rate was 6.0%-2.59%, which rejected approximately 5% of the extreme values and maximized the utilization of the patient data. In the MR program, we first set the random block size to N = 50 and introduced a target bias of 0.05 ng/ ml. Pre-analysis found that minimum MNPeds (22-46) appeared in an interval of COV = 0.03-0.07 ng/ml, and that the MNPeds increased with the increase of the COV; therefore, the COV was set to 0.02-0.08 ng/ml before simulated in detail. The above programs used N = 10-100 as the block size to simulate the performance of the bias detection, and the CL and FR rates are shown in Table 2.

| MA or MR bias detection simulation
The minimum of the MNPeds for MA and MR programs with different parameters are shown in Figure 1, parameters with a FR greater than 1% were eliminated. For the MA program, the mini-

| Validation of MA and MR programs
The MA and MR validation plots drawn with the optimal parameters are shown in Figure 3, The MR program is more sensitive to small biases, with a positive bias of 0.06 ng/ml or above and a negative bias of 0.4 ng/ml or above detected at a rate of 100% in a day, but only positive or negative biases could be detected by the MR program with the two different combinations of parameters, separately. Note: The i was used in the MA procedure to eliminate a certain number of outliers, for example, i = 0.2 means that the population mean ± 0.2 SD was the truncated limit, this eliminated data accounting for 6% of the population data. When N = 30, the lower and upper control limits were 0.03 and 0.24, respectively, and the false rejection rate was 0.18%. The cut-off value was used in the MR method to assign binary values to patient data, for example, COV = 0.03, if the patient result was greater than the COV, then it was regarded as "1"; otherwise, it was regarded as "0." When N = 10, the lower control limit was 46.73%, and the upper control limit was invalid. The false rejection rate was 0.37%. Abbreviations: COV, cut-off value; FR, false rejection rate; LCL, lower control limit; MA, moving average; MR, moving rate; UCL, upper control limit.  conventional QC nor MA program was suitable for detecting small systematic biases, which is consistent with the reported results. 18 The MR program with simulated optimal parameters could consistently detect positive of 0.06 ng/ml and above, or negative of 0.4 ng/ml and above biases, at an average daily run size of 10. A comparison of the two PBRTQC programs revealed that the MA program rejected as many outliers as possible to narrow the CL and to improve the detection sensitivity, but the increased rejection rate of the data reduced the frequency of calculating patient results, leading to possible delays in rejection and the possibility of not detecting particularly large biases. 19,20 After converting the results into the binary state, the MR program had no TLs and excessive concentrations for judgment, but had higher data utilization than the MA program, so it is very suitable for analyses with small volumes of data such as that in this study. Furthermore, the PCT results of the population showed a skewed distribution, leading to a reduction in the applicability of the MA method. 21 On the one hand, if the moving median method is used, then it is more difficult to interpret the results 22 ; on the other hand, it is difficult to estimate the standard deviation of the median, and the mathematical relationship between it and the mean standard deviation complicates the program. 23 Therefore, the MR method with relatively simple operations is more applicable to similar distribution models.
Contrary to the previous studies, this study was based on the data model itself, and the parameter interval with the highest sensitivity to the target error (0.05 ng/ml) was first estimated by presimulation. Detailed simulation was subsequently performed to determine the optimal parameter combination, which was free from   25 The GB/T 29790-2020 Point-of-Care Testing (POCT)-Requirements for Quality and Competence, which was published in China, establishes the requirements for the quality assurance capability of POCT products, but there are no specific provisions for the practices of POCT operators. Therefore, there are many issues that need to be resolved, and technicians need to develop more sensitive and intelligent QC programs to address issues such as requiring additional manual operation steps when processing data, failing to judge true or false rejection signals, and traceability when result is outside the control limits. As such, expected results would be obtained during the QC process and be continuously improved, thereby allowing laboratory personnel to focus more on solving clinical problems.

CO N FLI C T O F I NTE R E S T
The authors confirm that they have no conflicts of interest.