Rapid tests for the diagnosis of visceral leishmaniasis in patients with suspected disease

Target condition being diagnosed

Visceral leishmaniasis (VL), also known as kala-azar, is a life-threatening systemic disease caused by the obligate intracellular protozoan, Leishmania, and transmitted through the bites of phlebotomine sand flies (Herwaldt 1999). Leishmanial infection can cause a diverse spectrum of diseases, among which VL is the most severe form and which is almost always fatal without adequate, timely treatment. The species that causes VL isLeishmania donovani in Asia and eastern Africa, and L. infantum in Europe, North Africa and Latin America (Boelaert 2000). The geographical distribution of VL is often limited to well-identified endemic foci, but it has also emerged as epidemics (Seaman 1996) and as one of the opportunistic infections in human immunodeficiency virus (HIV)-positive patients (Alvar 2008; Pasquau 2005). VL is a neglected disease that affects the poorest and most vulnerable people in rural, remote settings where there is limited access to health care. The estimated incidence is 200,000 to 400,000 cases per year, of which more than 90% are reported from India, Bangladesh, Ethiopia, Sudan, South Sudan, and Brazil (Alvar 2012). The two main transmission modes of VL are anthroponotic and zoonotic. The human-to-human transmission is predominant in the Indian subcontinent and in eastern Africa, while the zoonotic transmission mode is found in the Mediterranean region and the Americas. The zoonotic transmission mode involves dogs as the main parasite reservoir (Chappuis 2007).

The pathogenesis of VL entails a complex interaction between the characteristics of the parasite and the host (Rittig 2000). The parasite life cycle involves the replication of the promastigote form in the gut of the female sand fly and is completed when she takes a blood meal on a non-immune host. In the human body, the Leishmania promastigote evolves into an amastigote form (without flagellum), which colonizes the macrophages in the liver, spleen and bone marrow. The control of the infection depends on the characteristics of the host, especially an intact cell-mediated immunity in the form of a T-helper type 1 response, involving the secretion of interleukin 12 and interferon-gamma. The failure of this cell-mediated immunity in some, but not all, of the infected individuals ultimately leads to the clinical manifestations of VL disease (Murray 2005).

The main clinical manifestations of VL are fever of insidious onset, weakness, loss of appetite, weight loss, abdominal distension due to enlargement of the spleen or the liver or both, lymph node enlargement, and a low blood cell count (pancytopenia). In children, other symptoms include diarrhoea, coughing, abdominal pain, and growth retardation. In any age group, if left untreated, the disease will progress with time, causing debilitation, bleeding, susceptibility to secondary infection and, eventually, death. Anti-leishmanial treatment is life-saving but non-response or relapse can occur. The pentavalent antimonials, sodium stibogluconate and meglumine antimoniate, despite their toxicity, have been the mainstay of VL treatment in many areas for decades. Alternatives include (liposomal) amphotericin B, paromomycin, and miltefosine. Combination therapy is the suggested way forward to increase treatment efficacy, prevent resistance, and reduce treatment duration and cost (van Griensven 2010). It is important that diagnostic tests for VL do not give false-negative results because VL may be fatal. Neither should they give false-positive results in order to prevent people without VL from receiving the toxic VL treatment.

As stated above, asymptomatic or subclinical infections also occur, and in cross-sectional surveys in endemic areas, a significant proportion of the healthy individuals present with anti-leishmanial antibodies. In longitudinal studies, the ratio between incident clinical cases and incident asymptomatic infections ranged from 1:5 in Kenya (Ho 1982) to 1:8 in Brazil (Evans 1992), 1:9 in Ethiopia (Hailu 2009), 1:8.9 in India and Nepal (Ostyn 2011), and 1:11 and 2.6:1 in Sudan (Zijlstra 1994). Furthermore, antibodies (seropositivity) in VL patients tend to persist for many years after cure (De Almeida 2006; Hailu 1990).

Index test(s)

Rapid diagnostic tests (RDTs) are defined as equipment-free diagnostic devices that do not require highly skilled laboratory staff. The results of an RDT can be read easily within minutes, or at most an hour or two (PATH 2008). Most RDTs work by capturing either an antigen or an antibody on a solid surface and then attaching molecules to them that allow detection by the naked eye. The technology used is mainly immunochromatography with a dipstick or lateral flow format. These immunochromatographic tests (ICTs) are used for VL diagnosis in dipstick or cassette format, using protein isolated from Leishmania sp as the antigen. The recombinant form of the 39 amino-acid-sequence from L. chagasi is the most widely used and is known as rK39. Other recombinant antigens such as rK9, rK16, rK26 and rK28 have also been evaluated. Currently, there are several on-going projects to develop a VL test based on antigen detection, for example, the latex agglutination test detecting a urinary leishmanial antigen.

The rK39 antigen was first used in an enzyme-linked immunosorbent assay (ELISA) format, but the newer dipstick or strip formats are increasingly being used in the field or away from the main centres. These rK39 ICTs give an immediate result (typically between 10 and 20 minutes) and give a binary reading (positive or negative). The test procedure involves adding the patient’s blood or serum with diluent buffer on the strip. When present in the blood or serum, specific antibodies against rK39 antigens, which are bound to the strip, can be read with the naked eye in the test window. There is a limited number of commercially available RDTs for VL, such as the IT-LEISH® (DiaMed AG, Switzerland – now Biorad, France), Kalazar Detect® (InBios International, USA), Onsite Leishmania Ab Rapid Test (CTK Biotech, USA) as well as a number of prototypes under various formats (Cunningham 2012).

rK39 ICTs cannot be used to diagnose VL in people with a past history of VL due to the persistence of antibodies after cure. In addition, to avoid diagnosis of asymptomatic infections, these tests must be applied in patients suspected to have VL, ie patients with prolonged fever and splenomegaly (WHO 2010).

Clinical pathway

Rationale

Correctly diagnosing VL disease is crucial as the signs and symptoms of VL are not specific enough to differentiate the condition from chronic malaria, schistosomiasis or other systemic infections. VL should be suspected in patients presenting with fever and splenomegaly, but confirmation is needed. The classical method relying on microscopic smears from tissue aspirates (spleen, bone marrow, lymph node) are unsuitable for settings with limited resources. Other methods such as antibody detection are more feasible, most notably the DAT and the rK39 antigen-based ICT. Both have been specifically developed in the past two decades for use in such contexts and have shown high diagnostic accuracy in most endemic areas (Boelaert 2004; Chappuis 2006).

The major drawbacks of these serological methods are linked to the antibodies that remain detectable after a cure and those that are due to past or present asymptomatic infection present in a sizeable proportion of residents of endemic areas. The use of a clinical case definition and adequate medical history taking should help to avoid false positives, yet it underscores the need for better VL diagnosis tests that are specific to acute stage disease. Ideally, the test should be highly sensitive, as VL is potentially fatal, and it should also be specific, as presumptive treatment cannot be fully justified with the current treatment regimens (den Boer 2006).

The RDTs for VL, mainly but not exclusively the rK39-based ICTs, seem to be the current solution for field diagnosis in remote settings: their ease of use, convenience and cost make them potentially advantageous to increase patients' access to VL diagnosis and treatment. The current WHO recommendation for VL case management includes the use of the rK39-based ICT as the basis for initiating treatment and it has also been adopted as the diagnostic tool in first-line services in the VL elimination initiative in the Indian subcontinent (WHO 2005). In other areas, the RDTs have been used as the first test in diagnosis-treatment algorithms that often incorporate other test(s), such as the DAT or tissue aspiration (Raguenaud 2007; Veeken 2003;).

Despite its operational advantages, some regional variability in rK39-based ICT performance has been observed (Chappuis 2006). Other prototype tests have been proposed as RDTs, but their real value in clinical practice is unclear. Furthermore, in the published literature, many phase II studies (with a sub-optimal, case-control design) might overshadow the more limited information of better quality based on phase III studies (including consecutive patients suspected to have VL). Thus, it is of utmost importance to synthesize the available evidence in this rapidly evolving field. Knowing the diagnostic accuracy of these RDTs would help inform policy makers and stakeholders on how best to use them in VL control in diverse settings. The role of RDTs in the different epidemiological contexts could be better defined, giving clearer evidence on their accuracy, and this review aims to contribute to exactly this goal.

Secondary objectives

Criteria for considering studies for this review

Search methods for identification of studies

We attempted to identify all relevant studies regardless of their publication status (published, unpublished, in press, ongoing). No language or date limits were applied. We limited our searches to studies conducted in humans.

Data collection and analysis

Results of the search

The date of the search was 3 December 2013. The search identified 1758 records, each record corresponding to a published article (Figure 1). After screening titles and abstracts, 1648 irrelevant records were removed. We were unable to obtain the full-text article of one record and excluded one other article because it was written in Chinese language and no translators were available. We retrieved the full text of the remaining 108 records and excluded another 87 for the following reasons: publication of the same study in more than one record (we kept the record that was published most recently and excluded the others; n = 4); no original research (n = 6); index test not a rapid test (n = 1); target condition not clinical visceral leishmaniasis (n = 5); not a phase III diagnostic accuracy study (n = 66); and reference standard not according to criteria (n = 5; Figure 1 and Characteristics of excluded studies). The remaining 21 records were included in this review.

Several records reported multi-country, multicentre or stratified data. If a ‘study’ is defined as a phase III design leading to a sensitivity and specificity estimate of one or more RDTs in a specific patient population, then three of the 21 included records contained results of more than one unique study. One article described RDT performance in patient populations in five countries (Boelaert 2008, henceforth distinguished as Boelaert 2008 - Ethiopia; Boelaert 2008 - India; Boelaert 2008 - Kenya; Boelaert 2008 - Nepal; Boelaert 2008 - Sudan); Ter Horst et al. reported data separately for HIV-positive and HIV-negative patient populations (ter Horst 2009 - HIV neg; ter Horst 2009 - HIV pos); and Diro 2007 included data from clinically suspected patients presenting at the study site and from people identified through active case finding in the community. Furthermore, in one record, two different brands of the rK39 immunochromatographic test (ICT) were evaluated in the same population (Chappuis 2005 - DiaMed; Chappuis 2005 - InBios), and we treated this record as two different studies. On the other hand, three patient populations were presented in more than one record. Boelaert 2008 - Ethiopia re-analysed the same patient population data as Diro 2007, but using a different reference standard. Similarly, Boelaert 2008 - India described the same patient population as Sundar 2007 but again, analysed it with a different reference standard. This also occurred in Machado de Assis 2011 and Machado de Assis 2012. We have treated these records as one single study in each country with a primary and a secondary analysis (see below).  Altogether, the 21 included records contained information about 25 unique studies.  We excluded the study based on active case finding (part of Diro 2007) from the review as it did not comply with the eligibility criteria of our protocol and, therefore, we finally included 24 studies.

These 24 studies included 4271 participants of whom 2606 were classified as having VL. Four studies used a reference standard including direct smears or culture of splenic aspirate, or both, 11 used a composite reference standard, three used latent class analysis, and six presented two sets of accuracy estimates using two different reference standard categories (Table 1).

The 24 studies contained information about five index tests: the rK39 ICT, the latex agglutination test in urine, the FAST agglutination test, an rK26 ICT and an rKE16 ICT. Six studies assessed the accuracy of more than one type of index test in the same patient population: four studies evaluated the rK39 ICT and the latex agglutination test; one study evaluated the rK39 and the rKE16 ICT; and one study evaluated the rK39 ICT, the rK26 ICT, and the latex agglutination test (Sundar 2007). Overall, the rK39 ICT test was evaluated in 20 studies including a total of 3806 participants of whom 2370 had VL. The latex agglutination test was evaluated in seven studies, which corresponds to 1459 participants including 873 people with VL. The FAST agglutination screening test was evaluated in two studies with a total of 148 participants including 69 with VL. The rK26 ICT test was assessed in one study with 352 participants of whom 282 had VL, and the rKE16 test was evaluated in one study with 219 participants of whom 131 had VL.

For the meta-analysis of the accuracy of the rK39 ICT test, 18 out of 20 studies were included (Table 2). At this stage, we excluded two studies because they only described HIV-positive patients (ter Horst 2009 - HIV pos and Cota 2013). In all the other studies using the rK39 ICT test, the included patients were HIV negative or the HIV status was unknown but considered of no importance in the study population. Six out of the 18 included studies generated more than one set of sensitivity and specificity estimates by using different reference standards in a primary and a secondary analysis. We selected one of the two estimates for the primary meta-analysis and explored the effects of this choice in the sensitivity analysis (Table 1).

For the meta-analysis of the accuracy of the latex agglutination test, we included six studies (Table 2). We excluded one study because it described mostly HIV-positive patients (Vilaplana 2004).

Methodological quality of included studies

The QUADAS-2 tool was used to assess the quality of the 21 included records in terms of the risk of bias and of concerns about applicability. Figure 2 summarizes the overall methodological quality and Figure 3 gives the ratings for each of the included records. In the domain of patient selection, we had high concerns about the risk of bias and the applicability of one record (Veeken 2003 - composite) because the inclusion of participants was conditional on the availability of stored serum samples and of diagnostic information from another serological test that could have correlated with the index test. In another record (Cota 2013), we had high concerns about the applicability to the review question because a considerable proportion of the study population (21%) had a previous history of VL. In the domains of the index test and the reference standard, the risk of bias was unclear for 14 of the 21 included records. The most frequent underlying reason was that the publication did not report whether the index test results were interpreted without knowledge of the reference standard and vice versa. We considered that there was a high risk of bias in those studies that used bone marrow aspirate smear tests and culture with or without clinical information as a reference standard due to the low sensitivity of these techniques (Kilic 2008 and Cota 2013 - composite 2). In the domain of flow and timing, we considered that there was a high risk of bias in five records, because not all patients were included in the analysis (Rijal 2004; ter Horst 2009; Veeken 2003; and Peruhype-Magalhaes 2012) or because not all patients received the same reference standard (Sundar 1998).

Findings

rK39-based rapid diagnostic tests

Data summary

Twenty studies reported Se and Sp estimates for the rK39 ICT. Six of these studies gave two sets of Se and Sp estimates, based on alternative reference standards (Table 1). A forest plot of the 26 available Se/Sp estimates is given in Figure 4. Figure 5 presents the available Se/Sp pairs according to the reference standard used; the results referring to the same data analysed with different reference standards are connected with a line.

Meta-analysis

A formal meta-analysis was performed on 18 of the 26 available data points: studies including mainly HIV-infected patients were reported separately (ter Horst 2009 - HIV pos, Cota 2013 - composite 1 and Cota 2013 - composite 2) and for the five remaining studies with multiple Se and Sp estimates, we used one set of estimates for the primary analysis (Table 1). Combining data from all studies without accounting for possible covariates explaining heterogeneity and using the bivariate model (Reitsma 2005), the average (95% CI) Se was 91.9% (84.8 to 96.5) and the average Sp was 92.4% (85.6 to 96.8)

The 95% prediction interval (PI) for the diagnostic accuracy that would be observed in a new study was (52.3 to 100)% for the Se and (54.9 to 100)% for the Sp (Figure 6).

Of note is the fact that, in contrast to what is presumed in the HROC model, the estimated correlation between the transformed Se and Sp was positive: 0.16 (95% CI: -0.40 to 0.65). Given these observations, we only report results from the bivariate model using the complementary log-log, and not from the ROC-based analysis.

Assessment of heterogeneity

A summary of the heterogeneity assessment through meta-analysis can be found in Figure 7 and Table 3.

Table 3. Heterogeneity assessment of the rK39 ICT meta-analysis

* The categories "parasitology including spleen aspirate - no serology" and "parasitology not including spleen aspirate - no serology" were combined because the latter category contained only one study.
	Sensitivity		Specificity
	Estimate	(95% CI)	Estimate	(95% CI)
Geographic region
Indian subcontinent	97.0	(90.0 to 99.5)	90.2	(76.1 to 97.7)
East Africa	85.3	(74.5 to 93.2)	91.1	(80.3 to 97.3)
Commercial brand
DiaMed	86.4	(70.7 to 95.9)	94.4	(80.5 to 99.2)
InBios	91.3	(83.8 to 96.3)	91.2	(83.1 to 96.3)
Other	97.8	(88.7 to 99.9	92.5	(75.5 to 99.0)
Disease prevalence in sample
Low (< 65%)	91.0	(82.2 to 96.9)	94.4	(87.2 to 98.3)
High (≥ 65%)	92.4	(84.5 to 97.4)	89.8	(79.7 to 95.8)
Study size
Small (< 250)	91.5	(82.7 to 96.7)	89.8	(80.6 to 95.6)
Large (≥ 250)	92.3	(83.4 to 97.4)	94.5	(87.8 to 98.3)
QUADAS-2: risk of bias
Low	89.5	(69.5 to 98.3)	96.0	(82.8 to 99.7)
Unclear	91.9	(83.7 to 97.1)	92.0	(84.0 to 96.7)
High	93.0	(77.9 to 99.1)	87.8	(70.7 to 97.2)
Reference standard
Parasitology - no serology *	95.6	(84.3 to 99.6)	89.1	(72.4 to 97.4)
Parasitology and serology	89.6	(78.6 to 96.7)	94.6	(85.2 to 98.7)
Latent class analysis	91.0	(79.8 to 97.2)	91.5	(80.7 to 97.3)

Geographic region

We assessed the difference in diagnostic accuracy effect between East Africa and the Indian subcontinent. There were nine data points from East Africa (two from Ethiopia, two from Kenya, three from Sudan, and two from Uganda), and six from the Indian subcontinent (two from India and four from Nepal). Three studies from other regions (two from Brazil and one from Turkey) were not considered in this analysis. The Se was significantly lower in East Africa (85.3%; 95% CI: 74.5 to 93.2) than in the Indian subcontinent (97.0%; 95% CI: 90.0 to 99.5). There was no significant difference in Sp: the Sp (95% CI) in east Africa was 91.1% (80.3 to 97.3) and in the Indian subcontinent 90.2% (76.1 to 97.7) (Summary of findings 1 and Summary of findings 2). Confidence and prediction regions by geographic region are given in Figure 8.

Commercial brand or type of test

The majority of studies used the InBios rK39 ICT test (11 studies); four assessed the DiaMed rK39 ICT test; and three evaluated other tests (DiaMed DUAL-IT L/M ICT test, Amrad ICT rK39 ICT test, and Arista Biologicals rK39 ICT test). There were no significant differences in diagnostic accuracy between commercial brands.

Disease prevalence

We separated studies by estimated prevalence of VL in the study sample: nine studies reported prevalence rates < 65% and nine reported prevalence rates ≥ 65%. There were no significant differences in diagnostic accuracy between studies with low and high prior probability of disease in the patient sample.

Quality assessment

We used study size and risk of bias according to QUADAS-2 as indicators of the quality of the primary studies.

Study size

We categorized studies according to the total number of study participants (true cases + non-cases) in small (< 250) and large (≥ 250) studies. There were 10 small and eight large studies. There were no significant differences in diagnostic accuracy between small and large studies, but there was a trend for larger studies to show a higher Sp estimate.

QUADAS-2: risk of bias

Studies were categorized according to the risk of bias assessed with QUADAS-2: low risk (three studies), high risk (four studies), and unclear risk of bias (11 studies). There were no significant differences among these categories, but there was a trend for studies with high risk of bias to show a low Sp and studies with low risk of bias to show a higher Sp.

Type of reference standard

Studies were categorized according to the reference standard used in the primary publication. There were no significant differences in accuracy among these categories. Further exploration of the influence of the reference standard is assessed in the sensitivity analysis below.

Diagnostic accuracy in HIV-positive participants

Two studies evaluated the rK39 ICT in HIV-positive participants and were not included in the meta-analysis.

The first study reported on the accuracy of the DiaMed rK39 ICT in 71 HIV-positive participants in Ethiopia (ter Horst 2009 - HIV pos). Based on a composite reference standard that combined direct parasitological examination of spleen aspirate with DAT serology, 65 participants had a diagnosis of VL. In this setting, the sensitivity of the rK39 ICT was 64.6% and the specificity 66.7%.

The second study evaluated the InBios rK39 ICT in 113 HIV-infected people in Brazil. Data were extracted according to two different reference standards: (1) the decision of an adjudication committee after clinical follow-up and taking into account all available information (including parasitology and serology) (Cota 2013 - composite 1; VL prevalence: 40.7%); and (2) direct smear and culture of bone marrow aspirate (Cota 2013 - composite 2; VL prevalence: 36.9%). The choice of the reference standard was of little influence. The Se of the rK39 ICT was low: 45.7% with the first and 46.3% with the second reference standard; and the Sp was high: 97.0% with the first and 97.1% with the second reference standard.

Sensitivity analyses

We performed sensitivity analyses of the influence of the choice of reference standard and possible biases resulting from imperfect reference standards. Given the impact of geographic region on the diagnostic accuracy of rK39 ICT, we corrected the sensitivity analyses for region effects and limited the analyses to 15 studies performed in the Indian subcontinent and East Africa. An overview of the results of the sensitivity analyses is given in Table 4.

Table 4. Sensitivity analysis results of the rK39 ICT meta-analysis

	Sensitivity		Specificity
	Estimate	(95% CI)	Estimate	(95% CI)
Main Analysis
Indian subcontinent	97.0	(90.0 to 99.5)	90.2	(76.1 to 97.7)
East Africa	85.3	(74.5 to 93.2)	91.1	(80.3 to 97.3)
Alternative Analysis Set
Indian subcontinent	96.1	(88.9 to 99.2)	86.7	(69.2 to 99.2)
East Africa	85.2	(75.0 to 92.7)	90.1	(77.0 to 97.4)
Bayesian analysis allowing for imperfect reference standards: expert priors - using main analysis set
Indian subcontinent	97.3	(91.9 to 99.5)	93.7	(74.9 to 99.7)
East Africa	86.7	(77.5 to 94.0)	95.7	(84.3 to 99.8)
Bayesian analysis allowing for imperfect reference standards: vague priors - using main analysis set
Indian subcontinent	97.3	(91.9 to 99.5)	93.0	(77.8 to 99.3)
East Africa	86.1	(77.2 to 93.3)	94.3	(83.8 to 99.6)

Analysis using secondary reference standards

As a sensitivity analysis to the choice of the set of estimates of a given study, we made an alternative selection of the reference standard for those studies presenting results for two reference standards (Table 1). The estimated Sp was slightly lower compared to the main analysis set: 86.7% versus 90.2% for the Indian subcontinent and 90.1% versus 91.1% for east Africa. This may be related to a lower Se of the reference standards included in this analysis, which would result in an underestimation of the Sp of the index test. Estimated Se was similar in the sensitivity and main analysis.

Analysis allowing for imperfect reference standards

In the studies available for this sensitivity analysis, two types of reference standards were used: parasitology including spleen aspirate – no serology; (three studies) and a combined reference standard of parasitology and serology (six studies), in addition to the six studies analysed using latent class analysis. We elicited the opinion from seven VL experts on the diagnostic accuracy and possible variation across the two reference standards (Figure 9). We used these expert opinions as prior information in a Bayesian statistical model to estimate the diagnostic accuracy of the rK39 ICT. This analysis indicated that due to lack of sensitivity of the reference standard, the Sp of rK39 ICT may be somewhat underestimated. The estimated Sp correcting for this bias was 93.7% in the Indian subcontinent, and 95.7% in east Africa, compared to 90.2% and 91.1% assuming perfect reference standards. The estimates of Se did not change significantly when we corrected for imperfect reference standards. In this model, the Se of spleen aspirate was estimated to be 96.0% (95% CI: 92.5 to 98.8) and the Sp was estimated to be 99.4% (95% CI: 98.1 to 99.9). The estimated Se of the combined reference standard of spleen parasitology and a serological test was estimated to be 98.6% (95% CI: 97.1 to 99.4) and the estimated Sp was 96.5% (95% CI: 91.5 to 99.2).

Similar results were obtained when we used non-informative priors instead of expert opinion to estimate the Bayesian model.

Assessment of structure of hierarchical model

Using the more standard logit link, in comparison to the better fitting complementary log-log link increased the DIC from 171.3 to 199.0. The logit link resulted in similar estimates of Se and Sp: the average (95% CI) Se was 91.9% (83.6 to 96.2) and the average Sp was 92.4% (84.9 to 96.3). However, the remaining heterogeneity was considerably larger resulting in wider prediction regions: 28.3 to 99.7% for Se and 31.1 to 99.7% for Sp (Figure 10) compared to the primary analysis.

Using a bivariate t-distribution for the random-effects did not improve model fit compared to the normal model.

Assessment of reporting bias

Reporting bias was assessed through the robust funnel plot proposed by Deeks 2005. This robust funnel plot presents the log of the diagnostic odds ratio by a robust measure of the study size. No indication of reporting bias was observed in this plot (Figure 11).

Latex agglutination test

Data summary

Seven studies were identified that reported Se or Sp results of the latex agglutination test in urine (KAtex). For two of these studies, two sets of Se and Sp estimates were available. A forest plot of the nine available Se/Sp estimates is given in Figure 12.

Meta-analysis

One study was not pooled with the others in the formal meta-analysis because, contrary to the other studies, this study included a majority of HIV-positive participants (Vilaplana 2004). For the two studies that presented Se/Sp estimates using LCA and a reference standard on the same data (Boelaert 2008 - Ethiopia and Diro 2007, Boelaert 2008 - India and Sundar 2007), the results from the two analysis approaches were very similar. We used the results from LCA for the primary meta-analysis.

Combining data from six studies without accounting for possible covariates explaining heterogeneity, and using the bivariate normal model with complementary log-log link, the average (95% CI) Se was 63.6% (40.9 to 85.6) and the average Sp was 92.9% (76.7 to 99.2) (Figure 13 and Summary of findings 3). The 95% prediction intervals were very wide: (16.5 to 99.6)% for the Se and (41.3 to 100)% for the Sp (Figure 13). The estimated correlation between the transformed Se and Sp was -0.39 (95% CI: -0.90 to 0.70).

Assessment of heterogeneity

Of the predefined sources of heterogeneity, geographic region, disease prevalence, and study size showed sufficient variability across studies to warrant a meta-regression. All studies used the KAtex latex agglutination test manufactured by Kalon biological. Five of the six studies had an unclear risk of bias. All studies in east Africa reported prevalences of VL < 65% and all studies in the Indian subcontinent reported prevalences ≥ 65%. Consequently, the effects of region and disease prevalence in the sample could not be separated. There were no significant differences in diagnostic accuracy of the latex agglutination test among regions or study sizes (Figure 14 and Table 5).

Table 5. Heterogeneity assessment of the Latex agglutination test meta-analysis

	Sensitivity		Specificity
	Estimate	(95% CI)	Estimate	(95% CI)
Geographic region
Indian subcontinent	50.8	(34.1 to 69.3)	95.3	(73.6 to 99.8)
East Africa	77.9	(58.2 to 92.3)	88.6	(59.5 to 92.3)
Study size
Small (< 200)	52.9	(27.6 to 84.1)	86.2	(52.5 to 99.2)
Large (≥ 200)	68.5	(45.2 to 88.2)	94.5	(49.0 to 100.0)

Sensitivity analysis

As a sensitivity analysis to the choice of the set of estimates of a given study, we made an alternative selection of the reference standard for those studies presenting results for two reference standards. The results were very similar to the primary analysis: the average (95% CI) Se was 63.4% (40.8 to 85.4) and the average Sp was 92.8% (76.3 to 99.2) (Table 6).

Table 6. Sensitivity analysis results of the Latex agglutination test meta-analysis

	Sensitivity		Specificity
	Estimate	(95% CI)	Estimate	(95% CI)
Main Analysis	63.6	(40.9 to 85.6)	92.9	(76.7 to 99.2)
Alternative Analysis Set	63.4	(40.8 to 85.4)	92.8	(76.3 to 99.2)

Diagnostic accuracy in HIV-positive participants

One study assessed the accuracy of the latex agglutination test in a population of 85 mostly HIV-positive participants (93%) in Spain (Vilaplana 2004). According to a reference standard based on culture or direct parasitological examination of bone marrow aspirate, 12 participants were classified as having VL. The estimated Se and Sp of the KAtex test were very high: the Se was 100% (12/12; 95% CI: 74 to 100) and the Sp was 96% (70/73; 95% CI: 88 to 99).

Summary of findings

Summary of main results

The rK39 ICT, developed in 1998, is the rapid diagnostic test for VL that has been most thoroughly evaluated so far. We retrieved 21 publications that corresponded to our inclusion criteria for the review, and from these, ultimately 18 independent sets of sensitivity (Se) and specificity (Sp) estimates could be included in a formal meta-analysis of diagnostic accuracy of the rK39 ICT. In patients with febrile splenomegaly and no previous history of VL, the overall sensitivity of the rK39 ICT was 91.9% (95% CI 84.8 to 96.5) and the specificity 92.4% (95% CI 85.6 to 96.8). Sensitivity was significantly lower in East Africa (84.3%; 95% CI 74.5 to 93.2) than in the Indian subcontinent (97.0%; 95% CI 90.0 to 99.5), but there was no significant difference in specificity between geographic regions (Summary of findings 1, Summary of findings 2, and Appendix 3). This assessment of heterogeneity did not include the Latin American and Mediterranean region as the number of studies from those regions was too low. There was no significant difference according to prevalence of disease in the sample or manufacturer of the rK39 ICT, but the number of estimates in some categories of the latter was low (11/18 InBios, 4/18 DiaMed, and 3/18 other brands). Generally, care should be taken when interpreting this heterogeneity assessment due to the limited number of studies included in the meta-analysis. Apart from the lack of power, several risk factors may be correlated inducing confounding between different covariates.

For the KAtex test, a latex agglutination test in urine, when used in a clinical setting in patients with febrile splenomegaly, overall sensitivity was 63.6% (95% CI 40.9 to 85.6) and specificity 92.9 % (95% CI 76.7 to 99.2) (Summary of findings 3). Seven studies were included in the review and six in the meta-analysis. Because of the limited number of estimates, no complete heterogeneity assessment could be made. There were no significant differences in diagnostic accuracy of the KAtex test among regions or study sizes, but the number of included studies was too small to allow for definite conclusions.

The number of studies addressing three other rapid diagnostic tests (FAST, rK26 ICT, and rKE16 ICT) was too low to allow for a meta-analysis. Our conclusions do not apply to HIV-positive patients. In summary, completeness of evidence was most satisfactory for the rK39 ICT, only partially complete for the latex agglutination test in urine, and too incomplete for the other three tests.

The methodological quality of the evidence on the rK39 ICT and the latex agglutination test was acceptable, although we had concerns about the risk of bias in some studies (see below).

Strengths and weaknesses of the review

Our review protocol explicitly set out to select clinical evaluations in a phase III design, based on consecutively recruited patients all suspect for VL, as this type of evidence level is required for making policy recommendations for clinical practice. Therefore, we had to exclude many records using a phase II (case control) design. However, in some records, the study design was not clearly reported, which made it difficult to distinguish between a phase III and a case control design. Because of this poor reporting, we may have missed some true phase III studies.

We believe that the procedures used for study selection were quite exhaustive, as we did not impose any a priori language barriers (excluding only one study in Chinese ex-post), and we searched several databases including regional ones such as LILACS. Therefore, we believe that the information presented here reflects the body of evidence accumulating over the past 15 years rather well. In addition, assessment of possible publication bias did not indicate that smaller studies without favourable results for the RDTs were less likely to be reported. 

Most important was the striking heterogeneity in methods across the studies. Although the included studies all complied with the criteria for an acceptable reference standard set out in our protocol, there was a remarkable variety in the reference standards that were used. These reference standards were based on different tests, done in different orders, with different cut-off values, and making different use of clinical information. We formally assessed the risk of bias in each record using the revised QUADAS-2 instrument. In five out of the 21 included records, we had clear concerns related to the patient and sample flow and the timing. For 14 records, the risk of bias was difficult to assess in the domains of the index test and the reference standard mainly because authors failed to report explicitly if and how blinding was applied.  When we categorized records into low (n = 3), unclear (n = 11), and high (n = 7) risk of bias, there was no significant difference in Se and Sp of the rK39 ICT, but high-risk studies tended to have a higher sensitivity and a lower specificity. One possible explanation is that the reference standards in studies classified as at high risk of bias were of sub-optimal sensitivity. This could result in a selection of cases being primarily severe cases which may be easy to diagnose with rapid diagnostic tests, while the control group may contain some undiagnosed true VL cases (falsely negative in the reference standard).

We tried to assess the influence of this quality in study design on the estimates of accuracy in a sensitivity analysis in the case of the rK39 ICT. Neither the use of a secondary analysis set that was based on a different reference standard, nor an analysis allowing for an imperfect reference standard affected the main conclusions of the meta-analysis on the rK39 ICT, although the secondary analysis using an alternative reference standard in five studies tended to give a lower specificity estimate. This may be related to a lower Se of the reference standards included in this analysis, which would result in an underestimation of the Sp of the index test.

Finally, the findings seemed sufficiently homogeneous to allow for a meaningful summary estimate for the specificity of rK39 ICT as well as the Se and Sp of the latex agglutination test in urine. However, the sensitivity of rK39 ICT was different according to geographic region, and seems substantially lower in East Africa than in the Indian subcontinent. This finding is in line with an earlier meta-analysis conducted by our team (Chappuis 2006) and with a multi-country study conducted by WHO/TDR (data not included) (Cunningham 2012). Several hypotheses have been raised to explain this lower sensitivity in east Africa: a lower level of circulating antibodies, different age group, or parasitological differences (Bhattacharyya 2013; Harhay 2011).

Applicability of findings to the review question

Our review focused on clinical studies (phase III) of diagnostic accuracy, recruiting participants representative for the spectrum of patients encountered in clinical practice in whom a rapid diagnostic test for VL would be warranted. In summary, those patients would be in line with the WHO case definition and defined as “patients with a clinical suspicion of VL, that is, those who are febrile for more than two weeks and have splenomegaly, presenting at health services in endemic areas.” (Source: protocol). Patients who were previously treated for VL (non-responders or relapsed cases), and those who had signs and symptoms of other forms of leishmaniasis, such as post kala-azar dermal leishmaniasis (PKDL) were excluded. All clinical studies included in the review clearly corresponded with these inclusion criteria. Nevertheless, the settings of these clinical studies varied: some were conducted in tertiary care centres, and some in smaller hospitals of an intermediary level. Few studies recruited patients at the first-contact point, the most peripheral level of the health service, ie health centres or health posts. Although the level of health service leads to variable prior probability of disease in the study sample, and as such is of influence in diagnostic accuracy studies, we believe that our findings are applicable across the several levels of the health system as an analysis of heterogeneity according to disease prevalence did not reveal any difference in the Se or Sp estimates.

Our review does not apply to HIV-positive patients, and caution should be taken in areas with high HIV-prevalence. Because HIV-status is known to influence diagnostic accuracy (ter Horst 2009 - HIV neg; ter Horst 2009 - HIV pos; Alvar 2008), we originally intended to analyse the accuracy of the RDTs in HIV-positive and HIV-negative patients separately. Unfortunately, the number of included studies of HIV-positive people was too low (n = 3) for a separate analysis or a comparison. Furthermore, the information from the Mediterranean region and from Latin America was too limited to draw robust conclusions. Finally, it was not possible to precisely evaluate the importance of the type of sample (serum, plasma or blood; fresh or frozen urine) or the manufacturer or version of a certain test, because these parameters did not vary enough among the included studies. In particular, Cunningham et al. pointed to important variations of sensitivity in a head-to-head comparison of five brands of RDT where two tests based on rK16 antigen performed substantially less well in east Africa and Brazil (Cunningham 2012).

In summary, we conclude that for current practice of VL control, there is one rapid diagnostic test, the rK39 ICT, which has been sufficiently evaluated, showing high sensitivity and specificity on average, but with a notably lower sensitivity in east Africa compared to the Indian subcontinent. In the Indian subcontinent, the rK39 ICT clearly complies with the normative requirements put forward before, of a sensitivity above 95% and a specificity above 90% (Boelaert 2007). In east Africa, the sensitivity of the rK39 ICT does not comply with these criteria, and can therefore not be used as a stand-alone test to reliably rule out VL in a patient who is suspected to have VL. This does not mean that the use of an rK39 ICT is precluded, but it should be embedded in a test algorithm with a clear instruction on what to do in case of a negative rK39 ICT (second test, referral, come back after two weeks for repeat testing or other).

For the latex agglutination test, although there were only six studies included in the meta-analysis, we can conclude that the low sensitivity makes it unsuitable for use in current clinical practice, though it does not preclude that its performance could be further improved in the future.

No recommendations can be made concerning the FAST test, the rK26 ICT or the rKE16 ICT because of paucity of evidence. 

Appendix 1. Statistical methods

A short summary of some less common statistical methods used in this review is given below. A more extensive description of the model used to correct for imperfect reference standards can be found in Menten 2013.

1) Latent class analysis (LCA)

LCA is a modelling technique that can be used in situations in which there is no good reference standard. It assumes that the true disease status in a study population is unknown (or latent). The LCA model estimates the sensitivity and specificity of a set of diagnostic tests (A, B, C, …) on the basis of observed frequencies in test patterns (ABC+++, ABC++-, ABC+-+,…). As such, the LCA model provides a model-based estimate of the gold standard classification; ie the best way to group study participants in diseased or non-diseased.

The basic latent class model assumes that the observed variables are conditionally independent. This means that there should be no associations between the results of the diagnostic tests within each category of the latent variable (disease status). If this assumption does not hold, more advanced techniques (eg based on Bayesian statistical methodology) can be used. To be selected for this review, studies using LCA had to assess the conditional independence assumption between the diagnostic tests, and if conditional dependence was expected, they had to use appropriate statistical methods to take this into account. If a study was selected, the sensitivity and specificity estimates derived from the final LCA model were included in this review.

More information can be found in Hui 1980, Black 2002, Branscum 2005, and Baughman 2008 among other references.

2) The complementary Log-Log function

In the bivariate model a "link" function g(y) is used to allow the use of the Normal (Gaussian) distribution to model the underlying study-specific sensitivity (Se) and specificity (Sp) of each study included in the meta-analysis. The standard link function used in the bivariate model is the logit link:

g(y) = log (y/(1-y))

An alternative link function is the complementary log log (cloglog) link:

g(y) = log(-log(1-y))

Both link functions transform Se and Sp, which are in the interval [0,1], to any real number between minus infinity and plus infinity.

The advantage of the cloglog link with our data is that it approaches infinity less quickly when y approaches 1 and consequently it mitigates the influence of studies that report 100% Se or Sp. This also reduces the inflation of the random-effects standard deviations as is apparent from comparison of Figure 6 and Figure 10. With the logit link, the prediction region extends to below the line of no diagnostic value (Se + Sp < 1), while the study which reports the lowest diagnostic value has Se = 0.75 and Sp = 0.70 (Figure 10). On the other hand, some observed data with high Se and Sp are not contained within the prediction region. The prediction region of the model with the cloglog link contains all observed data points, while not extending far beyond the studies with lowest observed Se and Sp (Figure 6). This is reflected in a lower deviance information criterion (DIC), a measure of model fit, for the cloglog model formulation compared to the logit formulation. The model with the lowest DIC shows the best fit to the data.

3) WinBUGS code for the primary model

WinBUGS is a statistical software for Bayesian analysis using Markov chain Monte Carlo methods. WinBUGS (or its recent open-source version OpenBUGS) provides a flexible Bayesian framework for model fitting. It can be used to fit both the bivariate and HSROC models.

Below is the code to fit the basic bivariate model, allowing for data from studies that use a reference standard and for studies that use latent class analysis.

Assuming there are N=N1+N2 studies:

• for the N1 studies that use a reference standard, the data extracted is:

• • NDiseased[i] and NNotDiseased[i]: the (true) number of diseased and non-diseased subjects in study i

• • TP[i], TN[i]: the number of true positives and true negatives in study i

• for the N2 studies that use latent class analysis, the data extracted is:

• • Y1[i] and Y2[i]: the estimates of g(Se) and g(Sp) obtained from the results of the LCA reported in the publication for study i

• • W1[i] and W2[i]: the estimates of the standard error of Y1[i] and Y2[i] for study i

• • with g() the link function (logit or cloglog)

The code is as follows:

model{

# Binomials for Studies that Use a Reference Standard

# (where the reference standard is presumed to be perfect)

for(i in 1:N1){

TP[i] ˜ dbin(SE[i],NDiseased[i])

TN[i] ˜ dbin(SP[i],NNotDiseased[i])

}

# Normals for Studies that Use LCA

for(i in 1:N2){

Y1[i] ˜ dnorm(alpha[i+N1,1],W1[i])

Y2[i] ˜ dnorm(alpha[i+N1,2],W2[i])

}

# Bivariate normals for g(Se) and g(Sp)

for(i in 1:N){

# Implement the link function g()

# If logit is used

logit(SE[i]) <- alpha[i,1]

logit(SP[i]) <- alpha[i,2]

OR

# If cloglog is used

SE[i] <- 1-exp(-exp(alpha[i,1]))

SP[i] <- 1-exp(-exp(alpha[i,2]))

# Specify the bivariate normal

alpha[i,1:2] ˜ dmnorm(mu[],R[,])

}

# Specify Non-Informative Priors

# for means:

mu[1] ˜ dnorm(0.0,.37)

mu[2] ˜ dnorm(0.0,.37)

# for variance-covariance matrix

R[1:2,1:2] ˜ dwish(Omega[,], 2) }

Note: Prior for Omega are provided as data:

Omega = matrix(c(.001,0,0,.001),nrow=2,byrow=T)

Appendix 2. Search strategy

Detailed search strategy

Appendix 3. Interpretation of the clinical relevance of the findings of this review - predictive values and likelihood ratios

What follows is an interpretation of the clinical relevance of the findings of this review regarding the rK39 immunochromatographic test (ICT) when it is used to diagnose visceral leishmaniasis (VL) among patients with febrile splenomegaly and no previous history of VL.

Internal sources

• Institute of Tropical Medicine, Antwerp, Belgium.

External sources

• Belgian Development Cooperation, Belgium.

  Grant to Temmy Sunyoto

• Department of Economy, Science and Innovation of the Flemish Government, Belgium.

  Funding for the Clinical Trials Unit at the Institute of Tropical Medicine, Antwerp