Establishing and validating of an laboratory information system‐based auto‐verification system for biochemical test results in cancer patients

Background To establish and validate an laboratory information system (LIS)‐based auto‐verification (AV) system by using large amounts of biochemical test results in cancer patients. Methods An algorithm of the AV process was designed for pre‐analysis, analysis, and post‐analysis. The limit range check was adjusted three times, while the delta check criteria were first replaced by the same patients’ historical extremum results. AV rules of 51 biochemical test items were tested by using data of 121 123 samples (6 177 273 tests) in 2016 that were manually reviewed through the simulative i‐Vertification software of Roche. The improved and optimal AV rules were programed into our LIS and validated by using 140 113 clinical specimens in 2018. Results The AV passing rate for samples tested in our laboratory increased from 15.57% to the current overall passing rate of 49.70%. The passing rate of each item for rule 3 was between 71.16% and 99.91%. Different cancer groups had different passing rate, while the disease group of liver, gallbladder, and pancreas always had the lowest passing rate. A total of 9420 reports (6.72%) were not verified by AV but could be verified by MV in 2018, while there were no reports that were verified by AV but not by MV. The TAT of March 2018 decreased with increase in sample size compared with the same time in 2017. Conclusion We have firstly established an LIS‐based AV system and implemented it in actual clinical care for cancer patients.


| INTRODUC TI ON
With the increasing annual incidence of cancer, 1 the number of patients admitted to cancer specialist hospitals has risen considerably, along with the demands for shorter report release turnaround time (TAT) from both the clinicians and patients. In addition to the complex pathophysiological changes induced by cancer, the common treatment modalities such as radiotherapy, chemotherapy, and surgery frequently result in impaired liver and kidney functions. 2,3 Therefore, biochemical tests for liver and kidney function are routinely performed for cancer patients, accounting for more than 50% of the total laboratory workload. The verification of these tests is a major post-analytical process, 4 and the accuracy and timely release of the results are crucial for medical decision-making. In recent years, various auto-verification (AV) systems have been incorporated into standard diagnosis. However, most of them need an intermediate software to be installed between the laboratory information system (LIS) and the specific instruments, [5][6][7] and the data that are not transferred to this software cannot be automatically verified. We developed an LIS system to achieve real-time AV and decrease the TAT. This is the first study to give a detailed report of the AV process according to the characteristics of cancer patients and evaluate the clinical benefits of the AV system.

| Construction of the AV algorithm
The AV algorithm was designed according to International organization for Standardization (ISO) 15189, 8 College of American Pathologists (CAP) Checklist 9 and Clinical and Laboratory Standards Institute (CLSI) AUTO-10A 10 (Figure 1), which covered the entire analytical process, that is, pre-analysis (eg, patient diagnosis), analysis (eg, sample information, quality control, instrument status flags), and post-analysis (eg, previous results). The limit range check was adjusted three times, first by using conservative reference range, and then using 95% confidence interval for each test item from the historical results in 2016, and finally adjusted by three technicians and three clinicians according to the individual cancer patients. The delta check was first replaced by the same patients' historical extremum, and the critical value check and consistency check were also selected for the AV process. Qualified samples, quality control (QC) results, and properly performing instruments are the pre-requisites for using the AV system. The order of validation using this system was critical value check, followed by limit range check, historical extremum, and finally consistency check for each single test item, and then for all test items. A sample passed the AV only if each single test item passed the process, while the results that failed any of the above rules were manually verified (MV).

| Establishment of AV rules
The principles for establishing the AV rules were as followed:

| QC
Our laboratory routinely uses internal QC (IQC) and takes part in

| Instrument error flags
The instruments were programed to give alerts for any problems with the reagents, barcodes, samples, or mechanical failure, for example, in the event of reagent crystallization or blood clotting. In addition, the results that were out of the analytical range also generated a warning flag and required sample dilution prior to re-analysis.

| Critical value
The critical values used in our hospital were determined locally with clinicians and are listed in Table 1. Results that were outside the range of these values required verification by a technician, and those within the range were subjected to the limit range check.

| Limit range check
We first used conventional reference intervals as the limit range to verify the results (rule 1), then calculated the 95% confidence interval for each test item from the historical results of 2016 (rule 2), and at last adjusted each test item limit range with three technicians and three clinicians (rule 3; Table 1).

| Historical extremum
To the best of our knowledge, this is the first study to compare current test results with the same patients' historical extremum results.
Cancer patients need regular reviews and follow-ups after treatment, and therefore, each patient has multiple biochemical test results which

| Validation method
Since the AV rules were based on historical data in 2016 and were written to our LIS system in 2017, they had to be validated with actual patient results before uploading, according to the CLSI Auto10-A Guidelines.
Therefore, we reanalyzed 140 113 clinical specimens in 2018 to verify whether rule 3 was able to meet our requirements. Since our laboratory does not have a pre-processing system, the samples were verified manually to be without visible hemolysis, jaundice, lipidemia etc The system and rules ran well and did not demonstrate any error flags. TAT was defined as the time from the receipt of specimens in our laboratory to the time when the report was released to clinicians or patients.

| The optimized rule 3 had the highest AV passing rate
To acquire the best AV rule, a total of 121 123 samples and 6 177 273 tests were collected and imported into the AV simulation analysis platform i-Vertification. Rule 3 showed the highest AV passing rate of 49.70%, while rules 1 and 2 had respective passing rates of 15.57% and 35.55% (Figure 2). The passing rate of each item for the three rules is summarized in

| Different cancer groups had different passing rate
The passing rates for each cancer group were 7.06%-25.20%, 20.99%-48.03%, and 29.97%-65.50%, for rules 1, 2, and 3, respectively ( Figure S1). Our results showed that with each adjustment, the passing rate was higher than before, while the disease group of liver, gallbladder, and pancreas (group B) always had the lowest passing rate.

| Work efficiency analysis in actual patient results
The AV program based on our LIS could be directly applied to routine clinical work. The verified rule 3 showed a passing rate of 58.46%

Verification of clinical laboratory reports involves different techni-
cians in the pre-analytical, analytical, and post-analytical phases, particularly in specialized cancer hospitals which run thousands of biochemical tests every day. AV of biochemical tests is a significant part of decision-making and has benefitted from the development and application of artificial intelligence to the medical field in the last 20 years. 4,11 Although CLSI AUTO-10A guideline has provided a general framework for AV, but it also advised each laboratory should design, implement, validate, and customize rules based on the needs of its own patient population. We previously found these rules were not applicable to cancer patients. Therefore, we summarized the manually validated results of our hospital, transformed it into computer language, and established the AV rules for the biochemical tests of cancer patients. Our LIS is programed to distinguish in real time whether the results need any further manual intervention for as long as the tests are carried out. This is the first study to give a detailed report of the LIS-based AV process according to the clinical characteristics of cancer patients.
Since the limit range-based criteria have the greatest impact on AV, 5 (Table S1).
In addition, the different cancer groups had different passing rates, and with each adjustment, the liver, gallbladder, and pancreas group always had the lowest passing rate. This could be due to the fact that tumors in these areas directly influence the biochemical tests. A qualified AV process should cover the entire analytical process, including patient diagnosis. However, accurate patient diagnosis is difficult, especially for the first time patients, and strongly dependent on information sharing between HIS and LIS. Therefore, we suggest that the best AV rules should be established by disease groups. Furthermore, since we wrote the algorithm of the AV process directly into LIS without an intermediate software, 12,13 the test results could be released directly to the physicians as soon as they passed the AV process.
Prior to uploading, our AV process was validated with 140 113 clinical specimens in 2018, and 6.72% of the reports which could be released directly were still intercepted. There is no analytical error yield based on our work. Due to the immediate release of AV results, the TAT of patient reports was shorter than those manually verified in the same period last year. In addition, the time and labor expended by the laboratory staff were unaffected despite the increasing number of samples, as well as the need for manual validation of some samples. Taken together, the ability to auto-verify even a small percentage of the results can increase productivity and save labor.
Compared to the conventional manual verification which is tedious, time-consuming, and (human) error-prone, the biggest benefit of AV is the consistency of the test results. However, despite the advantages of AV, there are potential negatives. For example, the validation of AV was time-consuming and required high attention to details. Furthermore, even the most thorough AV can miss unexpected instrument error flags or other rare events. It is also not possible to test every conceivable combination of rules. In conclusion, we have established and implemented an LIS-based AV for the biochemical tests of cancer patients, although our AV passing rate of the rules still need to be improved compared to the commercially available AV software systems. With the development of information technology, report review will become more and more convenient and personalized.

ACK N OWLED G EM ENT
The authors thank Roche and Beijing Hai-Hui for their dedicational technical supports.

CO N FLI C T O F I NTE R E S T
The authors have no conflicts of interests.