Assessment of reproducibility of a VP7 Blocking ELISA diagnostic test for African horse sickness

Abstract The laboratory diagnosis of African horse sickness (AHS) is important for: (a) demonstrating freedom from infection in a population, animals or products for trade (b) assessing the efficiency of eradication policies; (c) laboratory confirmation of clinical diagnosis; (d) estimating the prevalence of AHS infection; and (e) assessing postvaccination immune status of individual animals or populations. Although serological techniques play a secondary role in the confirmation of clinical cases, their use is very important for all the other purposes due to their high throughput, ease of use and good cost‐benefit ratio. The main objective of this study was to support the validation of AHS VP7 Blocking ELISA up to the Stage 3 of the World Animal Health Organization (OIE) assay validation pathway. To achieve this, a collaborative ring trial, which included all OIE Reference Laboratories and other AHS‐specialist diagnostic centres, was conducted in order to assess the diagnostic performance characteristics of the VP7 Blocking ELISA. In this trial, a panel of sera of different epidemiological origin and infection status was used. Through this comprehensive evaluation we can conclude that the VP7 Blocking ELISA satisfies the OIE requirements of reproducibility. The VP7 Blocking ELISA, in its commercial version is ready to enter Stage 4 of the validation pathway (Programme Implementation). Specifically, this will require testing the diagnostic performance of the assay using contemporary serum samples collected during control campaigns in endemic countries.

Corrections added on 24 December 2018, after first online publication: a spelling error in author Potgieter's name has been corrected and an affiliation added; authors Cécile Beck, Sylvie Lecollinet and Corinne Sailleau have been added.

| INTRODUCTION
African horse sickness (AHS) is an infectious but noncontagious viral disease affecting all species of Equidae and is caused by an Orbivirus of the family Reoviridae and characterized by respiratory and circulatory syndromes. Nine different serotypes of AHS virus (AHSV) have been identified. AHS is transmitted by species of Culicoides spp biting midges. All serotypes of AHS occur in eastern and southern Africa, from where they occasionally spread into countries surrounding the Mediterranean and on occasions reaching India and Pakistan (Mellor & Hamblin, 2004;Zientara, Weyer, & Lecollinet, 2015). Due to its high mortality rate, which can exceed 90% in susceptible populations, the huge economic losses that it causes in endemic countries and when it spills over into nonendemic territories, and its negative impact on international trade of equids, AHS is an OIE (World Organization for Animal Health)-listed disease. Furthermore, its control is considered a priority and AHS is one of the six animal diseases currently included by the OIE in the procedure for official recognition of a country's disease-free status (OIE, 2017a).
Clinical signs are characteristic but not pathognomonic of the disease, especially in endemic countries where the infection can cause a milder clinical form of the disease. Confirmatory diagnosis in the laboratory is most often achieved through virus detection techniques in blood samples or in postmortem tissue samples when AHSV-infected horses die before specific antibodies can become detectable in clinical serum specimens (Mellor & Hamblin, 2004;Zientara et al., 2015). Currently, polymerase chain reaction (PCR) methods are the first choice for diagnosis (Agüero et al., 2008;Guthrie et al., 2013). Once the disease has been confirmed, virus characterization can be attempted by PCR typing techniques  or by the classical pathway of virus isolation in cell cultures followed by virus neutralization testing (OIE, 2017b).
Horses that survive natural infection develop antibodies against the infecting serotype of AHSV within 8-12 days postinfection and, consequently, serology is the most practical approach for surveillance in nonendemic countries, determining disease freedom in a population or for import-export testing prior to international trade.
The VP7 protein AHSV is the outer core protein of the capsid and a group-specific antigen (Zientara et al., 2015;Maree & Paweska, 2005

| Diagnostic test kits
The test kits used in the study were obtained directly from the commercial manufacturer (INGENASA, Madrid, Spain). All partners used the same serum panel (Table 1)  The tests were conducted as indicated by the manufacturer's instructions. Briefly, serum samples and positive and negative con-

| Panel of serum samples
All participant laboratories contributed to the preparation of the sera panel by providing available samples. The panel was assembled at the LCV-Algete, and it was made out of 186 serum samples representing all possible AHS sero-epidemiological status that can be found in the field (Table 1)

| Assay of samples, interpretation of results and expected values
In each laboratory, the VP7 Blocking ELISA test was conducted in duplicate for each serum sample. Samples were classified, according to the manufacturer's instructions, as positive, doubtful or negative. The criterion of maximum sensitivity was selected for the final test evaluation. Thus, doubtful results were considered as positive.
To determine the diagnostic characteristics of tests (sensitivity and specificity), a reference value was assigned to each sample tested. The reference value assigned was the result previously obtained by the VP7 Blocking ELISA in the laboratory that donated the serum.

| Statistical methods
The assessment of reproducibility of the assays was the main objective of the study by estimating test's sensitivity (Se) and specificity (Sp) and their associated 95% confidence level interval (95% CI) calculated over the data generated by laboratories as a whole (total observations), after rejecting outlier laboratories.
For identifying outliers, two statistical approaches were complementarily used: (a) Cohen's Kappa statistic, used to compare the level of agreement of the results for each pair of laboratories (Thrusfield, 1995); and (b) The Z-Score or standard score, which determines the level of standard deviation by which the value of an observation is below/above the mean value (ISO, 2005).

Tests were completed successfully by all participants and results
were sent to LCV-Algete who compiled the data and distributed it to all partners for sharing information and discussion.
The performance of laboratories was assessed by computing kappa statistics (Figure 1). This revealed that Laboratory F produced highly discordant results in comparison with the rest of the laborato-  Table 2).

| DISCUSSION
The clinical outcome of AHS in horses is characterized by a rapid evolution of the disease inducing syndromes that range from subacute to per-acute form. Mortality rates typically exceed 50%, but can reach 90% depending on clinical form of the disease and the immunological status of the animals (Mellor & Hamblin, 2004 (Maree & Paweska, 2005).
ELISA methods based on the competitive-blocking principle are useful for the diagnosis of viral and bacterial diseases (Gallardo et al., 2015;Praud et al., 2016) due to their good diagnostic performance and their potential to be used with sera independent of the animal species of origin. Currently, the VP7 Blocking ELISA for the diagnosis of AHS is available as a commercial kit at National Reference Laboratories of the EU, and its performance is checked annually through ring trials organized by the European Reference Laboratory for AHS (Agüero, personal communications). In many laboratories, the test is currently the basis of preexport testing for AHS, a procedure that is routinely used to support the intense and increasing movement of horses across international borders. Both indirect and blocking VP7 ELISA methods are tests recommended by OIE for serological diagnosis of AHS (OIE, 2017b) and are prescribed tests by European legislation for movement checks and importation of Equidae (European Union, 2009).
We conducted a thorough analysis of performance of the AHS-VP7 Blocking ELISA with the view of generating enough data to support the validation of this assay up to Stage 3 (Reproducibility) of the OIE validation pathway for diagnostic tests (OIE, 2012(OIE, , 2017c. Data from one laboratory clearly performed as an outlier and, after investigation, was excluded from the assessment due to unidentified operational errors, after which, reproducibility was estimated by assessing the overall sensitivity and specificity and their associated uncertainty. The good test sensitivity (98.4% [95% CI: 95.3-99.7]) of the VP7 Blocking ELISA in naturally or experimentally infected animals confirms its ability to detect nonvaccinated individuals previously exposed to virus and would support its use for trade purposes. The ability to detect antibodies in infected horses previously immunized with vaccines containing complete AHSV virions was also very good either in animals vaccinated with attenuated (100% [95% CI: 95.9-100]) or with inactivated vaccines (100% [95% CI: 97.3-100]). This reveals the potential usefulness of the test to assess the sero-prevalence of AHS in a population of Equidae. In the case of a population where vaccination was performed, with either inactivated or live attenuated vaccines, it would be difficult to determine whether the seropositive animals had been exposed to field virus. In these cases, Some potential DIVA vaccines are based on the immunogenic antigen VP2 or VP2 and VP5 (Alberca et al., 2014;Castillo-Olivares et al., 2011;Chiam et al., 2009;Guthrie et al., 2009) break (1987-1992), when AHSV was not circulating in the field, long time after the animals were last vaccinated (as indicated in Table 1).
Therefore, under these circumstances, it is not surprising that the antibody titres had waned to undetectable levels by the time the horses were sampled.
The surveillance of AHSV free territories is of great importance in containing the disease within endemic areas. For this reason, serological diagnostic tests need to be highly sensitive to accurately detect possible incursions of AHSV in nonendemic areas or to certify with confidence that a previously infected population is free of AHS after the implementation of an eradication programme. In addition, they need to be highly specific to avoid the unnecessary costs associated with the animal movement restrictions that would be imposed in the region until investigation of any false positive results is concluded. The results on test specificity obtained with VP7 Blocking ELISA in our study (100% [95% CI: 98.9-100]) combined with its high sensitivity (reviewed above) would suggest its feasibility for the surveillance of the AHS in free areas. Again, the values obtained in our study are in line with those obtained by previous work when evaluating the VP7-based indirect ELISA (Maree & Paweska, 2005).
In conclusion, through this comprehensive evaluation we can conclude that the VP7 Blocking ELISA is reproducible and satisfies the OIE requirements of reproducibility. However, it is recommended (OIE, 2012, 2017c) that monitoring of diagnostic performance of the assay is periodically performed through international ring trials, such as those that have been previously conducted by the OIE/EU reference laboratories over the years.
The VP7 Blocking ELISA, in its commercial version, is ready to enter Stage 4 of the validation pathway (Programme Implementation).
Specifically, future studies should aim to test the diagnostic performance of the assay using contemporary serum samples collected during control campaigns in endemic countries. This type of analysis will help to determine positive and negative predictive values of the tests in the various epidemiological scenarios in endemic territories. Interpretation and analysis of the data, collected from nine different laboratories, including INGENASA, was conducted by Drs Duran, Aguero, Gubbins and Castillo-Olivares. We therefore conclude that the objectivity of the study was not affected. Otherwise there are no competing interests.