Rapid diagnostic testing for SARS‐CoV‐2: Validation and comparison of three point‐of‐care antibody tests

Abstract With the emergence of severe acute respiratory syndrome coronavirus 2 (SARS‐CoV‐2), a need for diagnostic tests has surfaced. Point‐of‐care (POC) antibody tests can detect immunoglobulin (Ig) G and M against SARS‐CoV‐2 in serum, plasma, or whole blood and give results within 15 min. Validation of the performance of such tests is needed if they are to be used in clinical practice. In this study, we evaluated three POC antibody tests. Convalescent serum samples from 47 reverse transcription‐polymerase chain reaction (RT‐PCR) verified patients with coronavirus disease 2019 (COVID‐19) collected at least 28 days post RT‐PCR diagnosis as well as 50 negative pre‐COVID‐19 controls were tested. The three tests (denoted the J‐, N‐, and Z‐tests) displayed the sensitivities of 87%, 96%, and 85%, respectively, for the detection of IgG. All tests had the same specificity for IgG (98%). The tests did not differ significantly for the detection of IgG. The sensitivities for IgM were lower (15%, 67%, and 70%) and the specificities were 90%, 98%, and 90%, respectively. The positive and negative predictive values were similar among the tests. Our results indicate that these POC antibody tests might be accurate enough to use in routine clinical practice.

which are often of the lateral flow immunoassay type and work by detecting IgG and IgM against SARS-CoV-2 in serum, plasma, or whole blood. Results are often available as soon as 10 min after initiation of testing. Prior studies have shown that most patients have developed antibodies two weeks after symptom onset. 2,3 While these types of rapid diagnostic tests (RDT) may not be used to identify patients with an active infection, they can potentially be used to confirm whether or not the patient has undergone infection and developed antibodies. This can be especially valuable for finding infected individuals who did not get tested with RT-PCR during the acute phase of infection due to exhibiting few or no symptoms. A rapid influx of POC antibody tests has hit the market. The reported sensitivity of these tests varies significantly, ranging from 39% to 100%, but the specificity is generally high, ranging from 90% to 99%. [4][5][6] In this study, we have compared and validated three different POC antibody tests.

| Data collection
The study was conducted at the Department for Infectious Diseases, Skåne University Hospital, Lund, Sweden. Convalescent blood samples from patients with RT-PCR verified COVID-19 (n = 47) were collected at least 28 days after the RT-PCR verified COVID-19 diagnosis. Twenty-three (49%) of the patients were hospitalized, of whom 11 required treatment with oxygen. The blood was allowed to coagulate for 1 h and centrifuged at 570×g for 10 min. Serum was frozen at −80°C until analysis. RT-PCR for SARS-CoV-2 was performed on nasopharyngeal swab samples with a modified in-house method in line with World Health Organization (WHO) guidelines as described by Corman et al. 7 In brief, primer design and assay sequence are identical to the referred method. Our modifications constitutes changes in the thermal cycling for the E-and RdRP genes, utilizing 48°C for 10 min followed by 95°C for 10 min. We used an annealing temperature for the E gene of 55°C, and we used an amplification phase of 45 s for both genes. Finally, for the RdRP probe we used a concentration of 0.2 µM. The negative control group (n = 50) was comprised of serum samples obtained from patients 4-6 weeks after discharge from in-hospital treatment for respiratory tract infections. Serum was collected as above between 1997 and 2007. The samples had been kept frozen at −80°C since collection.

| Antibody testing
The three tests evaluated in this study were the SARS-CoV-

| Statistics
Categorical data were expressed as numbers and differences between data were analyzed using the χ 2 test. Fisher's exact test was used when comparing the values of two tests to each other. The specificities and sensitivities were calculated and presented with 95% confidence intervals within parentheses. The confidence intervals were estimated with the Clopper-Pearson method. Statistical analysis was performed with, and graphs were created using, GraphPad Prism version 8.3.1. A p value < .05 was considered statistically significant.

| RESULTS
The N-test displayed the highest sensitivity for detecting IgG (96% [85%-99%]). The observed sensitivities for the J-and Z-tests were 87% (74%-95%) and 85% (72%-94%), respectively. There was no statistically significant difference in the ability to detect IgG between the three tests (p = .4). All three tests displayed the same specificity (98%). Sensitivity and specificity for each test to detect IgG can be seen in Figure 1A and Table 1.
Upon comparing the effect on the positive predictive value (PPV) when the prevalence of positive patients ranged between 0% and STRAND ET AL. | 4593 25%, there was no noteworthy difference found between the tests.
The graphical illustration of this is presented in Figure 1C.

| DISCUSSION
In this study, we have evaluated and compared three POC antibody tests designed to detect specific IgG and IgM antibodies against SARS-CoV-2. Due to the high contagiousness of SARS-CoV-2, 8 being able to identify patients who have undergone COVID-19 infection can be valuable during the process of differential diagnosis and for studying routes of transmission. A plethora of antibody tests have been developed for this purpose, but their sensitivity and specificity need to be validated in order for them to be used in routine clinical practice. Of the three evaluated tests, we found no significant differences between the tests regarding the sensitivity to detect IgG antibodies. Furthermore, there was no noteworthy difference between the tests when looking at the effect of prevalence on the PPV.  As for our results, there were some discrepancies between the tests that need to be discussed. There were three samples that evoked a positive IgG response in the N-test but that were negative in the J-and Z-test. Similarly, there was one sample that was negative in the J-test but that was positive in the two other tests and there were two samples that were negative when tested with the Z-test but that were positive when tested with the two other tests. For the specificities, each test detected IgG in one pre-COVID-19 sample but interestingly these false positives were discordant across the three tests.
Using RT-PCR as a reference method for determining the sensitivity and specificity to detect antibodies is not without problems. This approach assumes that all RT-PCR positive patients develops antibodies, which is not necessarily true. A potential explanation as to why some RT-PCR positive patients tested negative for IgG antibodies might be the novel finding of T cell-mediated immunity against SARS-CoV-2. 9 If immunity can be acquired through the means of T-cells, it is possible that the patients testing negative for IgG-antibodies are truly seronegative. There were however still discrepancies between the three tests which can also be explained by low antibody titers. An alternative to the use of sensitivity and specificity is instead reporting the results in terms of positive percent agreement (PPA) and negative percent agreement (NPA). These are calculated in the same way that sensitivity and specificity are, but more properly illustrate that we are comparing the agreement between two diagnostic methods of uncertain true sensitivity and specificity. 10 However, as RT-PCR has become an established gold standard for many authors, for the sake of consistency with the reported findings from similar studies, 4-6 we chose to also use sensitivity and specificity for the observed results in this study.
Another factor potentially affecting seroconversion is disease severity. It has been shown that a more severe course of COVID-19 correlates to higher levels of antibody production. 11 Of the 47 included RT-PCR positive patients, 23 were admitted.
About 77% (n = 36) of the included patients with COVID-19 in this study had a mild disease course. Of the six discordant samples mentioned above, five came from patients with mild symptoms. The observed differences in sensitivity was thus likely due to relatively low antibody levels in some individuals with a mild course of COVID-19.
Since the sensitivity for detecting IgG ranged between 85% and 96% between the three tests, we wanted to examine how this would impact the PPV. As seen in Figure 1C, there was no relevant difference in PPV even when plotting against a low prevalence.