A European multicentre evaluation of detection and typing methods for human enteroviruses and parechoviruses using RNA transcripts

Abstract Polymerase chain reaction (PCR) detection has become the gold standard for diagnosis and typing of enterovirus (EV) and human parechovirus (HPeV) infections. Its effectiveness depends critically on using the appropriate sample types and high assay sensitivity as viral loads in cerebrospinal fluid samples from meningitis and sepsis clinical presentation can be extremely low. This study evaluated the sensitivity and specificity of currently used commercial and in‐house diagnostic and typing assays. Accurately quantified RNA transcript controls were distributed to 27 diagnostic and 12 reference laboratories in 17 European countries for blinded testing. Transcripts represented the four human EV species (EV‐A71, echovirus 30, coxsackie A virus 21, and EV‐D68), HPeV3, and specificity controls. Reported results from 48 in‐house and 15 commercial assays showed 98% detection frequencies of high copy (1000 RNA copies/5 µL) transcripts. In‐house assays showed significantly greater detection frequencies of the low copy (10 copies/5 µL) EV and HPeV transcripts (81% and 86%, respectively) compared with commercial assays (56%, 50%; P = 7 × 10−5). EV‐specific PCRs showed low cross‐reactivity with human rhinovirus C (3 of 42 tests) and infrequent positivity in the negative control (2 of 63 tests). Most or all high copy EV and HPeV controls were successfully typed (88%, 100%) by reference laboratories, but showed reduced effectiveness for low copy controls (41%, 67%). Stabilized RNA transcripts provide an effective, logistically simple and inexpensive reagent for evaluation of diagnostic assay performance. The study provides reassurance of the performance of the many in‐house assay formats used across Europe. However, it identified often substantially reduced sensitivities of commercial assays often used as point‐of‐care tests.

Its effectiveness depends critically on using the appropriate sample types and high assay sensitivity as viral loads in cerebrospinal fluid samples from meningitis and sepsis clinical presentation can be extremely low. This study evaluated the sensitivity and specificity of currently used commercial and in-house diagnostic and typing assays. Accurately quantified RNA transcript controls were distributed to 27 diagnostic and 12 reference laboratories in 17 European countries for blinded testing. Transcripts represented the four human EV species (EV-A71, echovirus 30, coxsackie A virus 21, and EV-D68), HPeV3, and specificity controls. Reported results from 48 in-house and 15 commercial assays showed 98% detection frequencies of high copy (1000 RNA copies/5 µL) transcripts. In-house assays showed significantly greater detection frequencies of the low copy (10 copies/5 µL) EV and HPeV transcripts (81% and 86%, respectively) compared with commercial assays (56%, 50%; P = 7 × 10 −5 ). EV-specific PCRs showed low cross-reactivity with human rhinovirus C (3 of 42 tests) and infrequent positivity in the negative control (2 of 63 tests). Most or all high copy EV and HPeV controls were successfully typed (88%, 100%) by reference laboratories, but showed reduced effectiveness for low copy controls (41%, 67%). Stabilized RNA transcripts provide an effective, logistically simple and inexpensive reagent for evaluation of diagnostic assay performance.
The study provides reassurance of the performance of the many in-house assay formats used across Europe. However, it identified often substantially reduced sensitivities of commercial assays often used as point-of-care tests. recently emerged as a respiratory pathogen occasionally leading to acute flaccid myelitis (AFM). 3 Infections with human parechoviruses (HPeVs) in the genus Parechovirus are enteric, usually asymptomatic apart from those of HPeV type 3, which is associated with sepsis-like illness, meningitis, and encephalitis in young children. [4][5][6] Although there is no effective antiviral treatment available for EV infections, detection and identification of EV and HPeV infections are vital for informing other treatment options, supportive care and prognosis of affected individuals. The reverse-transcriptase polymerase chain reaction (RT-PCR) is now the "gold standard" for diagnosing EV and HPeV infections due to its advantages of fast turn-around time and high sensitivity over virus isolation. 7 Even in severe cases, viral loads are relatively low in cerebrospinal fluid (CSF) samples that are typically tested in patients presenting with meningitis or encephalitis, and may be missed by less sensitive methods.
To detect all EV types, RT-PCR assays for the detection of EV RNA usually target the highly conserved 5′ non-translated region (5′NTR). Depending on the primer and probe design, some molecular detection methods may fail to detect certain EV types such as EV-D68, whereas some assays may also detect HRVs (reviewed in Holm-Hansen 3 ). EV and HPeV serotypes are defined serologically and genetically by their capsid region sequences; virus typing, therefore, requires amplification and sequencing of regions within this structural gene block, typically VP1. 8,9 Evaluation of sensitivity and specificity of the diagnostic assays used for EV and HPeV detection and typing is essential. We have previously evaluated the use of RNA transcripts of several EV and HRV serotypes and HPeV1 for quality control purposes in six expert clinical virology laboratories in Europe. 10 Following this study, we have now produced a further set of RNA transcript standards for selected representative serotypes from EV species A-D and HPeV3.
The RNA standards were distributed via the European non-polio enterovirus network (ENPEN) to members in diagnostic and reference laboratories for evaluation of the sensitivity of their routinely used assays for detection and typing of enteroviruses.

| RNA transcript synthesis
Available full-length cDNA clones of EV species A (EV-A71 genogroup B4 strain, accession number AF316321) 11  Plasmids were linearized at the 3′ end, and RNA transcripts were produced using the MEGAscript T7 Transcription Kit (Ambion), followed by DNase treatment to remove template DNA. RNA was purified using the RNAEasy Mini Kit (Qiagen), according to the manufacturer's instructions.

| RNA quantification and stability assessment
Quantification of RNA transcripts was carried out using the NanoDrop ND-1000 UV-Vis Spectrophotometer (Thermo Fisher Scientific) and the Qubit 4 Fluorometer (Thermo Fisher Scientific).
The concentrations obtained were used to calculate copy numbers of transcripts produced, assuming a mean molecular mass for each base of 330 g/mol. A serial dilution of RNA transcripts (10 5 to 10 −1 copies/µL) was prepared using the RNA storage solution (Thermo Fisher Scientific; 1 mM sodium citrate, pH 6.4) containing herring sperm carrier RNA (50 µg/mL) and RNasin (New England BioLabs UK, 100 U/mL). Dilutions were aliquoted and stored at −80°C before testing and distribution to the participating laboratories.
EV species A (EV-A71) and C (CVA21) transcripts were investigated for stability at different temperatures. Transcripts were incubated in storage solution for up to 30 days at ambient temperature, 4°C and 37°C. A further aliquot of each was freeze-thawed three times. The amount of RNA was quantified by RT-PCR and values compared to those of the original preparations.

| Transcript amplification by real-time RT-PCR
For quantification of RNA sequences before distribution, In-house quantitative real-time RT-PCR was carried out using the StepOne-Plus Real-Time PCR System (Thermo Fisher Scientific), according to the manufacturer's instructions. The following reaction conditions were used: 50°C for 30 minutes, 95°C for 15 minutes followed by 45 cycles of 95°C for 10 seconds then 60°C for 1 minute. A total of 20 μL reaction volume containing 2 μL of the diluted transcript was used. PCRs used primers and probes as previously described for EV, 13 HRV, 14 and HPeV 15 (Table S1).

| Participating laboratories
The RNA transcripts were distributed via ENPEN to member diagnostic and reference laboratories for evaluation of the sensitivity of their routinely used assays for detection and typing of EV and HPeV.
Laboratories were identified by a standardized code (L1, L3….). Coded transcripts panels were shipped by standard registered UK post in  were classified as national reference laboratories and the remaining 27 as primary diagnostic laboratories (Table S2).

| Participating laboratories
Most laboratories (n = 36) participated in the evaluation of detection assays, some evaluating multiple assays; this produced a total of 63 sets of results for detection assays. Reference Laboratories showed greater investment in in-house detection methods for EV and HPeV detection; in-house assays were used A total of 37 sets of in-house typing results were provided, from these 18 sets (14 for EV typing, 3 for HPeV, and 1 for both) were reported by reference laboratories and 12 sets by diagnostic laboratories from 7 different countries including Spain, Sweden, Greece, Italy, Germany, Norway, and UK. Most of these results were reported for EV typing (n = 29) but HPeV typing was also performed (n = 10).

| Sensitivity and specificity of screening methods
The sensitivity and specificity of detection assay results were calculated, and totals adjusted for the declared target range of the tests. Intended assay targets included combined EV and HPeV detection (n = 23), EV detection only (n = 17), HPeV detection only (n = 10) and combined EV and HRV detection (n = 7) as well as monospecific assays for EV-D68 (n = 1) and EV-A71 (n = 1). Two HRV-only assays were evaluated for specificity only.
In general, laboratories reported high rates of detection (98% for CVA21; 100% for EV-A71, E30, and EV-D68) of the EV transcripts at the higher concentration (10 3 RNA copies in 5 µL) (Figure 3). More variable detection of the low concentration transcripts (10 RNA copies in 5 µL) was reported, ranging from 62% (EV-D68) to 90% (EV-A71). Detection frequencies of the HPeV3 transcripts were comparable; 97% for higher concentration and 75% for lower concentration. For assays reporting C t values for the higher and lower concentration transcripts, values were compared to evaluate viral load ratios ( Figure S1). Although no assay produced quantitative results, reported results showed a 58 to 106 fold differences in geometric mean viral loads, close to the expected 100-fold difference. Assays were therefore reasonably quantitative in relative terms in this concentration range.
Assays were also generally highly specific, with only 2 high C t value (weak positive) results reported falsely positive from the 62 tests performed. A larger number of tests specific for EVs reported HRV detection, with five tests designed for the detection of EV (n = 2), EV and HPeV (n = 2) and EV-D68 (n = 1) reporting positive results with the HRV-C49 RNA transcript.
Methodology differences contributed substantially to the sensitivity of the screening assays ( Figure 3B). In particular, commercial assays, often highly multiplexed for other viral targets in CSF, showed significantly reduced sensitivity for the detection of RNA transcripts at a lower concentration compared to in-house methods F I G U R E 2 Stability of RNA transcripts on incubation at different temperatures and freeze/thawing. Fold changes in RNA detection of two representative RNA transcripts preparations of CAV21 (EV species C) at low copy number and EV-A71 (species A, high copy number) used for laboratory distribution. Transcripts were incubated for various durations at different temperatures. Detected viral loads were compared to those of RNA transcripts stored at −80°C. RNA transcripts were additionally subjected to three freeze/thaw cycles (rapid cooling and thawing; right hand panel). Bar heights show fold reductions of RNA relative to the starting amount; error bars show SEMS of three assay repeats (56% detection rate compared to 81%; P = .0009; Figure 3B). There was a comparable difference in HPeV detection rates, with 50% of the lower concentration transcript detected by commercial and 86% by in-house assays. There were significant differences in assay sensitivity in results reported by diagnostic and by reference laboratories ( Figure 3C). This is largely accounted for by the greater use of in-house assays by reference laboratories.
Commercial assays included a variety of platforms and assay specificities ( Table 1); several including Biofire, AusDiagnostics, Progenie and some assays from Seegene and bioMérieux were unable to detect the lower concentration RNA control (10 RNA copies in 5 µL), and in some cases, even the higher concentration RNA transcript (1000 copies in 5 µL) (Biofire, AusDiagnostics). We further investigated the sensitivity of the Biofire assay with intermediate RNA concentrations; assay sensitivity lay between 400 and 1000 RNA copies for most of the EV transcripts and between 1000 and 40 000 copies for HPeV RNA (Table 2).   Detection frequencies in the 10 and 1000 copies/5 µL transcript dilutions. Insensitive results-low detection rate of the 10 copy/5 µL control-are underlined, unexpected results-detection failure of 1000 copy/5 µL controls are indicated in bold. 2 The 1000 copy/5 µL E30 (EV species B) transcript was undetected in both assays. 3 The 1000 copy/5 µL HPeV3 transcript was undetected.

| Sensitivity and accuracy of EV and HPeV typing methods
regions of the genome between EV species suitable for amplification by PCR are largely confined to short motifs in the 5′-UTR. 13   all reference laboratories achieved this sensitivity using in-house PCR methods, but the 10 copy control was frequently negative in testing with commercial assays. In the case of the biofire film array assay, the mean limit of detection for the 4 EV species was greater than 2000-4000 RNA copies/mL, with an even lower sensitivity for HPeV3 (Table 2). These findings are consistent with previous evaluations of the Biofire film assay, with reported limits of detection for EV detection of >500 RNA copies/mL, 18 and some reduction in rates of detection of (unquantified) EV-positive clinical samples compared to conventional diagnostic assays. [19][20][21] Viral loads in CSF are low, often at the limit of assay sensitivity of PCR, so variability in assay sensitivity could substantially influence diagnostic target detection rates in diagnostic samples.

| Virus typing
In the current study, EV and HPeV typing were performed in both reference and diagnostic laboratories (Figure 4). Although all results reported the correct EV-type identification and most laboratories successfully typed the high concentration (1000 copies in 5 µL) transcripts, fewer than 50% of either reference or diagnostic laboratories could successfully amplify and sequence the low concentration controls. The observed restriction in assay sensitivity underpins the importance of obtaining multiple samples including blood, respiratory samples and feces for EV diagnostic and typing assays where viral loads are higher during acute infections. [22][23][24] It is also time to consider how EV and HPeV typing data can be centrally collected and analyzed at the time when increasing numbers of diagnostic laboratories are starting to introduce typing within the hospital premises. 7,16 In conclusion, effective EV and HPeV detection and type identification are integral to clinical management, public health surveillance and outbreak preparedness for emerging strains.
However, their genetic diversity, and often low viral loads in diagnostic specimens places stringent demands on the analytical sensitivity and breadth of detection and typing assays. RNA transcripts provide the means to independently evaluate these aspects of their performance. In the future, they can provide objective and fixed standards needed for a more critical assessment of the effectiveness of the numerous, newly developed and currently largely unevaluated testing platforms for syndromic testing. We would be delighted to provide EV, HPeV and further RNA transcript controls for a wider range of viruses to laboratories for QA purposes in the future.