Cepheid Xpert® Flu/RSV and Seegene Allplex™ RP1 show high diagnostic agreement for the detection of influenza A/B and respiratory syncytial viruses in clinical practice

Abstract Background Molecular assays based on reverse transcription‐polymerase chain reaction (RT‐PCR) provide reliable results for the detection of respiratory pathogens, although diagnostic agreement varies. This study determined the agreement between the RT‐PCR assays (Xpert® Flu/RSV vs Allplex™ RP1) in detecting influenza A, influenza B, and respiratory syncytial viruses (RSVs) in clinical practice. Methods We retrospectively identified 914 patient encounters where testing with both Xpert® Flu/RSV and Allplex™ RP1 was undertaken between October 2015 and September 2019 in seven hospitals across New South Wales, Australia. The diagnostic agreement of the two assays was evaluated using positive percent agreement, negative percent agreement, and prevalence and bias‐adjusted kappa. Results The positive percent agreement was 95.1% for influenza A, 87.5% for influenza B, and 77.8% for RSV. The negative percent agreement was 99.4% for influenza A, 99.9% for influenza B, and 100% for RSV. The prevalence and bias‐adjusted kappa was 0.98 for influenza A, 0.99 for influenza B, and 0.97 for RSV. In a sensitivity analysis, the positive percent agreement values were significantly higher during the non‐influenza season than the influenza season for influenza B and RSV. Conclusions The Xpert® Flu/RSV and Allplex™ RP1 demonstrated a high diagnostic agreement for all three viruses assessed. The seasonal variation in the positive percent agreement of the two assays for influenza B and RSV may have been due to lower numbers assessed, variability in the virology of infections outside the peak season, or changes in the physiology of the infected host in different seasons.


| INTRODUC TI ON
Acute respiratory infections due to influenza and respiratory syncytial viruses (RSVs) have a significant health and economic burden in Australia 1,2 and internationally. 3,4 Worldwide, influenza is implicated in approximately two percent of all respiratory deaths, 5 with an official World Health Organization estimate of 290 000-650 000 seasonal influenza-associated deaths globally each year. 6 In 2017, the influenza virus was implicated in an estimated 11.5% of episodes of care for lower respiratory tract infection, including 9.5 million hospitalizations and 81.5 million hospital days worldwide. 4  viruses and provides real-time influenza A subtyping. 7 It comprises three viral panels and one bacterial panel. The first of these viral panels (the Allplex ™ RP1) detects influenza A and subtypes (H1, H1pdm09, and H3), influenza B, and RSV (types A and B). Although the Allplex ™ RP offers comprehensive testing for respiratory viruses, it is undertaken by a central laboratory and specimens are tested in batches (processing multiple specimens simultaneously) rather than through continuous needs-based testing. Therefore, it has a test turnaround time of 1-4 days depending upon the frequency of batch testing and location of the hospital. 8 The Xpert Flu/RSV XC is an automated, multiplex real-time, RT-PCR assay for the detection of influenza A, influenza B, and RSV. 9 It has a faster turnaround time of 1-4 hours, 8,10 but unlike the Allplex ™ RP1, Xpert ® Flu/RSV cannot discriminate among influenza A virus subtypes. Nevertheless, the rapid identification of respiratory viruses has the potential to improve patient outcomes by supporting clinical decision-making around antimicrobial use. 11 More rapid identification could also improve clinical processes, including optimizing bed management and infection control 12 as well as minimizing unnecessary ancillary test utilizations, 8,10,13 with subsequent reductions to healthcare costs. 14 All molecular assays based on RT-PCR provide reliable results for the detection of respiratory viruses but are not always in diagnostic agreement. Previous studies reported the performance of Xpert ® Flu/RSV and Allplex ™ RP1 against other reference assays. 7,9,[15][16][17][18][19][20] However, to the best of our knowledge, no prior published studies have compared the Xpert ® Flu/RSV and Allplex ™ RP1 in clinical practice. The objective of the study was to determine the diagnostic agreement between Xpert ® Flu/RSV vs Allplex ™ RP1 for the detection of influenza A, influenza B, and RSV in the hospital emergency department (ED) or inpatient settings.

| Study design and setting
We conducted a retrospective observational study utilizing 4 years of data from October 2015 to September 2019 extracted from seven public hospitals (Hospitals A-G) in New South Wales, Australia (six general hospitals and one children's hospital [Hospital D]). Xpert ® Flu/RSV was introduced to four of the study hospitals (Hospitals A-D) in July 2017, while Hospitals E-G have used the test since October 2015. Therefore, the data before July 2017 were only from the other three hospitals (Hospitals E-G).

| Participants and data sources
Eligibility criteria included patients for whom both Xpert ® Flu/RSV and Allplex ™ RP1 tests were ordered at the same time for the same episode of care. Exclusion criteria specified patients for whom: (a) only one of the assays was ordered, (b) both assays were ordered but at different times (eg, Xpert ® Flu/RSV while the patient was in the ED and Allplex ™ RP1 while the patient was in the inpatient ward), and (c) both assays were ordered but the result of one of the assays was not reported or missing due to unacceptable specimens ( Figure 1).
The data used for this study were sourced from the Laboratory Information System (LIS) of each hospital. The LIS contains laboratory test order information including, but not limited to, patient age and sex, patient Medical Reference Number, test order episode identification, type of tests ordered, location of the order, test results and specimen types, as well as the date and time a specimen is collected, received at the laboratory and a verified result available.
All study hospitals used one LIS and thus had similar test catalogs and configurations.

| Test methods
Detailed information regarding the use of the two assays has been reported in our previous studies. 8,10,21 Briefly, Allplex ™ RP has been used as a referral test at a large central laboratory located at Hospital B and all other hospitals sent samples to this laboratory for analysis.
Testing was performed in batches once or twice depending upon demand. 8 As Allplex ™ RP was offered as three viral panels, in addition to panel 1, clinicians had the option of ordering panel 2 (which detects adenovirus, metapneumovirus, enterovirus, and parainfluenza viruses) and panel 3 (which detects bocaviruses, coronaviruses, and rhinoviruses) depending on the clinical scenario. Unlike AllplexTM RP, testing using the Xpert ® Flu/RSV was performed at local hospital laboratories avoiding the need for sending the specimen to the central laboratory. While both assays were available to clinicians during the study period, the use of Xpert ® Flu/RSV was recommended to be used by the laboratory service for patients at high risk of influenza, where a result was required more urgently. This included intensive care patients with influenza-like illness, immunocompromised patients with influenza-like illness, ED patients with a significant respiratory infection and isolation requirement.
Nasopharyngeal swabs and nasopharyngeal aspirates were the two main types of specimens that have been used for both Xpert ® Flu/ RSV and AllplexTM RP1. Sputum or bronchial lavages were also used in very few cases.

| Statistical analysis
A two-by-two contingency table comparing the results of Xpert ® Flu/RSV against Allplex ™ RP1 was created. The diagnostic agreement of the two assays was evaluated using positive percent agreement (PPA), negative percent agreement (NPA), kappa, and prevalence and bias-adjusted kappa (PABAK) along with their 95% confidence intervals (CI). PPA (which is analogous to sensitivity) was calculated by dividing the number of Allplex ™ RP1 + and Xpert ® + cases by total Allplex ™ RP1 + cases. NPA (which was analogous to specificity) was calculated by dividing the number of Allplex ™ RP1− and Xpert ® − cases by total Allplex ™ RP1− cases. The values of kappa can range from −1 to 1 and can be interpreted as <0.2 as none to slight; 0.21-0.4 as fair; 0.41-0.6 as moderate; 0.61-0.8 as substantial; and 0.81-1.0 as almost perfect agreement. 22 Because kappa can be affected by disease prevalence and potential bias between the assays, PABAK was reported to account for these influences. 23 This is particularly important given the low prevalence of respiratory viruses in our sample. Bias in the context of this study is said to occur when the two assays differ in the frequency of the detection of a given virus in the study sample. 23 Baseline factors associated with discordant results between the two assays were determined using penalized logistic regression as it reduces the small-sample bias and is thus suitable for modeling low prevalence binary outcomes. 24 A discordant result (yes/no) was defined as discrepancies between the results of the two assays in at least one of the three viruses (ie, Allplex ™ RP1 + and Xpert ® − and vice versa).

| Ethical approval
Ethical approval for the study was granted by the Human Research Ethics Committee of the South Eastern Sydney Local Health District (reference, HREC/16/POWH/412).

| Participants
A total of 914 patient encounters fulfilled the inclusion criteria ( Figure 1).
The median age was 28 years, and 55.2% (n = 505) were male. A nasopharyngeal swab was the most common specimen type (Table 1).

| Test results
The median turnaround time from specimen receipt to authorized result was 3.3 hours (ranged from 1. 5

| Diagnostic agreement
The diagnostic agreement measures are shown in

| Factors associated with diagnostic disagreement
Overall, 23 specimens showed a discordant result. One of these specimens showed discrepancies in results for more than one virus (Table S1). Gender, age category, setting where tests were ordered or the type of specimen was not associated with a discordant result. In other words, there were no significant differences in the results of the two tests across these characteristics. However, influenza season status was significantly associated with a discordant result. The difference between the two assays was higher during influenza season compared to the non-influenza season.
Xpert ® Flu/RSV and Allplex ™ RP1 were 3.38 times more likely to have discordant results during the influenza season than the noninfluenza season (Table 3).

| Sensitivity analysis
Sensitivity analysis by influenza season status was conducted as it was associated with the discrepancy between the results of the

| Key findings
We retrospectively evaluated the agreement of two PCR-based assays (

| Interpretation and comparison with existing literature
No previous studies comparing the Xpert ® Flu/RSV and Allplex ™ RP, against which to compare our findings, were identified. However, the performance of each assay has previously been evaluated against other molecular assays. Xpert ® Flu/RSV has been compared to quality control samples, 15 single-plex real-time RT-PCR, 19 a laboratory-developed assay, 18 BioFire FilmArray (USA), 20 and GenMark Diagnostics eSensor RVP (USA), 9

| Implications for practice
The main clinical implication of our finding is that there is no need to use both assays at the same time in clinical practice, given the Abbreviations: ED, emergency department; NP, nasopharyngeal. a NP swab (Allplex) and NP aspirate (Xpert) or NP aspirate (Allplex) and NP swab (Xpert) or the use of sputum or bronchial lavages in one of the assays.

| Strengths and limitations
To the best of our knowledge, this is the first study to compare the Xpert ® Flu/RSV and Allplex ™ RP1. The inclusion of diverse study populations from multiple sites including six general and one children's hospitals can be considered as the main strength of this study.
The key limitation of this study was that, in the case of discrepant results between the two assays, a three-way comparison using another confirmatory method has not been conducted to verify the results. This study utilized retrospective data from hospitals where Xpert ® Flu/RSV and Allplex ™ RP1 were the main laboratory tests for the diagnosis of influenza and RSV. It was not possible to determine which of the two assays was correct in the case of discrepancies between the assays. The Allplex ™ RP1 assay flexibility in assessing other non-respiratory viruses at a lower unit cost, also means assessment of other causes of a patient's symptoms are potentially assessed. According to the local guideline for the use of Xpert ® Flu/ RSV, its use was mainly reserved for patients at high risk of influenza where an urgent result was needed. The study population selected for this study may, therefore, differ in disease severity from other patients presenting to the hospitals with influenza-like illnesses but were not eligible to receive the test, potentially introducing bias into the study. As this study was not designed to assess test cost or health economic outcomes, no conclusions around this can be drawn.

| CON CLUS ION
In conclusion, Xpert ® Flu/RSV XC and Allplex ™ RP1 demonstrated a high diagnostic agreement for all three viruses assessed. The seasonal variation in the PPA of the two assays for influenza B and RSV may have been due to lower numbers assessed, variability in the virology of infections outside the peak season, or changes in infected host physiology in different seasons.

ACK N OWLED G EM ENTS
The study is part of a partnership project funded by a National

CO N FLI C T O F I NTE R E S T S
The authors declare that they have no conflict of interest.