SEARCH

SEARCH BY CITATION

Keywords:

  • water microbiology;
  • water supply;
  • environmental monitoring;
  • meta-analysis;
  • Escherichia coli ;
  • coliform bacilli
  • microbiologie de l’eau;
  • approvisionnement en eau;
  • surveillance environnementale;
  • méta-analyse;
  • Escherichia coli ;
  • bacilles coliformes
  • Microbiología del Agua;
  • Fuentes de agua;
  • Monitorización Ambiental;
  • Meta-análisis;
  • Escherichia coli ;
  • bacilos coliformes

Abstract

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Objectives  To assess the diagnostic accuracy of the H2S test for microbiological contamination of domestic water across different settings, as a basis for providing guidance on its use.

Methods  We searched a range of bibliographic and ‘grey’ literature databases to identify studies that had processed domestic water samples using both the H2S test and recognized tests for thermotolerant coliforms or Escherichia coli. We screened 661 study abstracts and identified 51 relevant studies based on 13 853 water samples. For each relevant study, we recorded the level of correspondence between the H2S and recognized tests, microbial testing procedures, details of the samples processed and study quality indicators. We conducted a meta-analysis to investigate the impact of testing procedures, study quality and sample characteristics on the diagnostic accuracy of the H2S test.

Results  H2S test implementation varied between studies, and the test’s diagnostic accuracy varied significantly and substantially between studies. Little of this variation was explained by testing procedures, study quality or the nature of the samples processed.

Conclusions  Although in widespread use, our findings suggest that the diagnostic accuracy, particularly specificity, of the H2S test is variable. Optimal conditions for conducting the test remain unclear. As H2S test accuracy is low in a minority of these studies, we recommend that its performance be evaluated relative to standard methods, prior to its operational deployment in a new setting.

Objectifs:  Evaluer la précision diagnostique du test H2S pour la contamination microbiologique de l’eau domestique dans différents contextes, afin de pouvoir conseiller sur son utilisation.

Méthodes:  Nous avons recherché dans une gamme de bases de données bibliographiques et la littérature «grise» pour identifier les études ayant testé des échantillons d’eau domestique en utilisant à la fois le test H2S et des tests reconnus, basés sur les coliformes thermo tolérants ou E. coli. Nous avons parcouru 661 résumés d’études et identifié 51 études pertinentes menées sur 13853 échantillons d’eau. Pour chaque étude pertinente, nous avons noté le niveau de concordance entre le test H2S et les tests reconnus, les procédures des tests microbiens, les informations sur les échantillons traités et les indicateurs de qualité de l’étude. Nous avons effectué une méta-analyse pour évaluer l’impact des procédures des tests, la qualité de l’étude et les caractéristiques des échantillons sur la précision diagnostique du test H2S.

Résultats:  L’implémentation du test H2S variaient selon l’étude et la précision diagnostique du test variait de manière significative et substantielle selon l’étude. Cette variation s’expliquait quelque peu par les procédures de tests, la qualité de l’étude ou la nature des échantillons traités.

Conclusions:  Nos résultats suggèrent que bien que d’utilisation répandue, la précision diagnostique du test H2S, en particulier la spécificité, est variable. Les conditions optimales pour effectuer le test restent floues. Comme la précision du test H2S est faible dans certaines de ces études, nous recommandons que sa performance soit évaluée par rapport aux méthodes standard, préalablement à son déploiement opérationnel dans un nouveau cadre.

Objetivos:  Evaluar la precisión diagnóstica de la prueba de H2S en la contaminación microbiológica de aguas domésticas a lo largo de diferentes emplazamientos, como base para dar información sobre su uso.

Métodos:  Hemos realizado una búsqueda en un rango de bases de datos bibliográficas y de literatura ‘gris’ para identificar estudios en los que se han procesado muestras de agua domésticas utilizando tanto la prueba de H2S como pruebas reconocidas para coliformes termotolerantes o E. coli. Hemos evaluado 661 resúmenes de estudios e identificado 51 estudios relevantes de 13,853 muestras de aguas. Para cada estudio relevante, hemos anotado el nivel de correspondencia entre el H2S y las pruebas reconocidas, procedimientos de testaje microbiano, detalles de las muestras procesadas e indicadores de calidad del estudio. Realizamos un meta-análisis para investigar el impacto de los procedimientos de prueba, la calidad del estudio y las características de precisión diagnóstica de la prueba del H2S.

Resultados:  La implementación de la prueba de H2S varió entre estudios y la precisión diagnóstica de la prueba varió significativamente y sustancialmente entre estudios. Poca variación puede explicarse por los procedimientos de las pruebas, la calidad del estudio o la naturaleza de las muestras procesadas.

Conclusiones:  Aunque ampliamente utilizada, nuestros hallazgos sugieren que la precisión diagnóstica, particularmente la especificidad de la prueba H2S, es variable. Las condiciones óptimas para realizar la prueba siguen sin estar claras. Puesto que la precisión de la prueba H2S es baja en una minoría de estos estudios, recomendamos que su desempeño se evalúe frente a métodos estándar, antes de su despliegue operativo a nuevos emplazamientos.


Introduction

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Diarrhoeal diseases from inadequate water, sanitation and hygiene annually cause an estimated 2.2 million deaths globally and 76.3 million disability adjusted life years (DALYs), 3.9% and 5.3% of the respective global totals (Prüss et al. 2002). Faecal contamination, the most widespread health risk associated with drinking water, is identified through microbiological testing. Standard tests require the collection of a sample and processing at well-resourced laboratories. Many developing countries lack such facilities, with resources, budgets and poorly trained staff further hampering the ability to deliver accurate assessments.

A simple, low-cost presence/absence test for bacteria that produce hydrogen sulphide (H2S) has been proposed to assess faecal contamination (Manja et al. 1982). As originally proposed, H2S test medium contains 20 g of peptone, 1.5 g of dipotassium hydrogen phosphate, 0.75 g of ferric ammonium citrate, 1 g of sodium thiosulphate, 1 ml of teepol and 50 ml of water. After incubation for at least half a day, often under ambient conditions, a change to a black or grey colour indicates the presence of H2S-producing bacteria and presumed faecal contamination. H2S test kits are relatively easy to manufacture, so are often made locally, at lower cost than recognized methods (Genthe & Franck 1999). The test is used in many developing countries, in emergencies (Mosley & Sharp 2005) and in remote areas of developed countries (UNICEF 2007).

Previous studies have compared H2S test results with those for recognized methods to enumerate thermotolerant coliforms (TTC) or Escherichia coli (EC). An earlier non-systematic review examined this literature (Sobsey & Pfaender 2003), identifying false-positive results from groundwater samples as a particular concern and recommending its use with caution, only where other alternative tests are infeasible. This study aims to assess variation in diagnostic accuracy of the H2S test in drinking or domestic water across different settings based on a systematic review and meta-analysis.

Methods

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Search strategy and selection criteria

We conducted a systematic review and meta-analysis, the protocol for which is available from the corresponding author on request. We identified microbiological studies that simultaneously test the same set of domestic water samples for faecal contamination using two different methods, the H2S test and a standard, recognized test for TTC or EC. The target condition being diagnosed was thus faecal contamination of domestic water, with the associated risk of water-borne disease.

We included studies of the H2S test where its implementation was low-cost (i.e. less than US$2/test) and capable of use onsite (i.e. not requiring permanent laboratory facilities). Included comparator tests for the TTC or EC indicator bacteria groups were recognized as an International Organisation for Standardisation standard, in the UK Blue Book methods (http://www.environment-agency.gov.uk/research/commercial/32874.aspx), in the American Water Works Association Standard Methods (Eaton et al. 2005), USEPA (http://www.epa.gov/fem/methcollectns.htm) or in equivalent country-specific standards at the time of the study. We included samples from stored water and supply systems for both households and public places such as schools and clinics. We also included surface water samples from streams, rivers and lakes where used for drinking.

The search strategy for bibliographic databases combined terms for the H2S test (e.g. ‘H2S’, or ‘hydrogen sulphide’, or ‘hydrogen sulphide’, or ‘pathoscreen’, or ‘Manja’) with terms for domestic water samples (e.g. ‘water’, or ‘environmental samples’) and terms for comparator test indicator bacteria groups (e.g. ‘thermotolerant’, or ‘faecal’, or ‘faecal’, or ‘coliform’, or ‘E. coli’). These terms were used in English and Chinese with 21 bibliographic databases and 12 sources of grey literature (online-only supplementary materials, Table S1).

Bibliographic and grey literature searches covered the period until 15 February and 2 March 2010, depending on the source. References to and from included studies, a relevant review (Sobsey & Pfaender 2003), and to Manja's original study (Manja et al. 1982) were also reviewed. Where the full text of a potentially relevant paper was unavailable, we contacted one or more of the authors. Studies identified by searches were independently assessed for inclusion by two members of the research team (JAW, HY), who then independently recorded characteristics of included studies. They also assessed study quality against 14 criteria commonly used for clinical diagnostics (Whiting et al. 2003). Arithmetic errors in each included study report were also recorded. Disagreements were resolved by consensus or referred to other team members (SP, JE).

We used the 1 cfu/100 ml threshold to define contamination with TTC or EC wherever possible, as this reflects World Health Organization guideline values (WHO 2008). Where necessary, we recorded individual sample data and calculated numbers of concordant and discordant samples.

Statistical analysis

Meta-analyses were based on sensitivity and specificity, determined using numbers of concordant and discordant samples. Univariable meta-regression was applied within the bivariate random effects framework using the Stata midas routine (Arends et al. 2008), to identify characteristics of studies, samples and test procedures associated with test performance.

For 21 included studies that implemented multiple H2S test procedures (e.g. evaluating the test under different incubation conditions or media) and compared these with standard methods, we selected the most widely used H2S procedure across all included studies. We undertook a sensitivity analysis to test the robustness of our findings to this strategy for choosing comparison groups. To investigate publication bias, we looked for asymmetry on a funnel plot of the diagnostic odds ratio. We also performed a linear regression of log odds ratios on the inverse root of the effective sample size to test for funnel plot asymmetry (Deeks et al. 2005).

Results

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

As the search strategy and study selection results given in Figure 1 show, 51 studies were included in the systematic review from 58 reports, whose characteristics are shown in Table 1. These studies reported a total of 13 853 water samples tested using both the H2S method and standard assays for TTC or EC. The 51 studies comprised 3140 groundwater samples; 845 surface or rainwater samples; 4404 distribution system, tap or standpipe samples; and 4111 samples of water stored in the home or public places. The source of a further 1353 samples was unclear, and 4106 samples had been treated. Twenty-nine studies used TTC, of which 11 used 1 cfu/100 ml to define contamination. Twenty-three studies used EC as the reference test, of which 12 defined water contamination using a threshold of 1 cfu/100 ml.

image

Figure 1.  Flow diagram, showing how studies were selected for inclusion in the review.

Download figure to PowerPoint

Table 1. Characteristics of the 51 studies included in the systematic review (see Appendix S1 for references)
StudySample characteristicsH2S test proceduresStandard methodStudy quality
CountryTotal samplesDominant sample type%Media formulationIncubation period (h)Incubation temperature (°C)VolumeIndicator groupGeometric mean bacteria density (cfu/100 ml)Threshold density (cfu/100 ml)No of reporting errors
  1. Dominant water source abbreviations: DST, distribution system/tap; GW, ground water; T, treated; POU, point-of-use; SRW, surface/rain water; abbreviation for the H2S test media formulation: M1982, Manja et al. (1982); M2001, Manja et al. (2001); M1982_1, Manja et al. (1982) + ‘6-fold liquid’ + lauryl sulphate, sodium salt; M1982_2, Manja et al. (1982) + l-cystine; M1982_3, Manja et al. (1982), replacing teepol with bile salt; M1982_4, modified Manja et al. (1982), but without media details; V1994, Venkobachar et al. (1994); Hiselective, Hiselective H2S kit; TTC, thermotolerant coliforms; Threshold, threshold density used to define contaminated sample.

  2. †Hewison et al. (1988) reported on two separate studies.

  3. ‡Raka et al. (1999) report a main study and subsequent follow-up work using different H2S test media, separated here as two studies.

Anwar et al. (1999)Pakistan 210DST77.14M1982243720TTC  1
Castillo et al. (1988), Castillo et al. (1990)Chile 200T91.5M198212020, 3520TTC 22
Cervantes (2003)Nepal  21GW100Pathoscreen24, 483520 Escherichia coli 0.3610
Chaudhary et al. (2003)India  51  Hiselective    E. coli   0
Coulbert (2005)Peru  17T94.12Pathoscreen4835100 E. coli 3.710
Desmarchelier et al. (1992) Malaysia 388POU49.48M198218, 3037∼3820 E. coli 0.881, 10, 501
Genthe and Franck (1999) South Africa 415  M1982 & V19944822, 3520TTC8.591, 10, 100, 10000
Grant and Ziel (1996)USA 104SRW86.54M1982_12430100TTC 10
Gupta et al. (2008) Bangladesh 382GW62.83M1982_324, 48, 722520 E. coli 17.651, 10, 100, 1000, 100000
Hewison et al. (1988a)†Thailand 703GW84.21M19822428∼3320TTC 102
Hewison et al. (1988b)†Thailand 128POU91.41M198224, 30Ambient temperature TTC2.052.20
Johnson (2007)Ghana  68POU100PathoScreen4825∼3520 E. coli 6.4110
Kaspar et al. (1992)Paraguay  48GW81.25M19824822, 28, 3720TTC37.3 0
Khush et al. (2009) India3710T86.63 24Ambient temperature30 E. coli 0.5210
Kromoredjo and Fujioka (1991a), Kromoredjo and Fujioka (1991b)Indonesia  50T96M198212∼15, 18∼2426∼30  E. coli 4.282.20
Lantagne and Environmental (2001)Nicaragua  48POU100PathoScreen12, 24, 36, 48   E. coli  10
Lee (2001)Nepal  78  PathoScreen  20 E. coli  10
Lira et al. (2008)Nicaragua 215DST51.16PathoScreen1425∼35100TTC2611
Lukacs (2002)Nepal  75GW100Hach & IDRC723710, 20TTC & E. coli1.34 & 3.0310
Manja et al. (2001) India 686DST100M1982_4 25∼44 TTC  0
Mattelet (2006)Ghana  46T71.74PathoScreen483520 E. coli 6.03 0
Monjour et al. (1986a), Monjour et al. (1986b)Burkina Faso  58GW100M198218∼243720TTC0.2510
Mosley et al. (2004)Fiji Island  59T74.58M19823625∼3010TTC2.8310
Mosley et al. (2004), Mosley and Sharp (2005)Fiji   3SRW100M198292Ambient temperature30, 50TTC551.1310
Nair et al. (2001a)Australia 101SRW100Pillai 199924, 4837100TTC 11
Nair et al. (2001b)Australia 253SRW64.03Pillai et al. 199924, 4837  E. coli  11
Nikaeen et al. (2010)Iran  35  M1982483760, 100, 120TTC & E. coli5.7 & 2.1512
Oates et al. (2003)Haiti  24  PathoScreen    E. coli   0
Okioga (2007)Ghana  30POU100PathoScreen24, 4824∼3520 E. coli 0.1310
Pant et al. (2002)India 200GW97.5M1982_22435 ± 120TTC 20
Pathak and Gopal (2005)India  90T44.44M1982_27235100TTC 2.21
Peletz (2006)Ghana  25POU100PathoScreen24, 483520 E. coli 5.7110
Pradhan (2004), Pradhan et al. (2005)Nepal  25DST80M1982    E. coli 29.88 0
Raka et al. (1999)‡India1535  M198218, 243720TTC  7
Raka et al. (1999)‡India 100  M1982 & M1982_3243720TTC  0
Ramteke (1995)India 194GW68.04M198218∼243020TTC9.59 2
Ratto et al. (1988), Ratto et al. (1990)Peru 160T≥65.45M1982 22, 35 TTC0.8620
Ratto et al. (1989)Peru  20T100M198212∼1822, 3520TTC0.7320
Rijal et al. (2000)Unclear   25  M19822425∼30  E. coli   0
Roser et al. (2005)Australia  12GW100M1982_2722510, 20, 100 E. coli 0 (<0.1)0.10
Singh et al. (1985) India  35  M198212∼183720TTC38.741, 10, 1000
Smith (2001)Nepal  33GW100Hach & IDRC4830, 3510, 100 E. coli   1
Stauber and Sobsey (2007)Dominica 446       E. coli 7.95101
Stauber and Sobsey (2007)Dominica 690    Ambient temperature100 E. coli  10
Striebig et al. (2006)Benin   PathScreen   TTC  0
Hirulkar and Tambekar (2006), Tambekar et al. (2007), Tambekar and Hirulkar (2007)India1050GW67.62M1982_418, 24, 48Room temp & 3720TTC  3
Tambekar et al. (2008a)India 260GW65.77M200124, 48  TTC  0
Tambekar et al. (2008b), Tambekar et al. (2008c)India 525T32.95M2001483720TTC  3
Tewari et al. (2003)India  67GW52.24M198224∼4844.520TTC  2
Vasudevan et al. (2001), Vasudevan and Tandon (2008)India  54SRW100M198248Room temp20TTC6.461, 100
Venkobachar et al. (1994)India 101GW64.34M198217, 223520TTC 2.20

The implementation of the H2S test varied markedly between studies, with a wide range of incubation temperatures and periods used (Table 1). Most studies used locally manufactured media, rather than using commercially available test kits. Water sample volumes were largely 20 ml, although four studies used 10 ml and nine studies used 50 or 100 ml. Thirty-two studies performed the H2S test procedure in the laboratory with skilled staff, although the test is often intended for field use by staff with only basic training. H2S tests were performed in a variety of ways. Twenty-three studies used the original media proposed by Manja et al. (1982), three modified the original media by adding l-cystine, two modified the original media by replacing Teepol with bile salt or lauryl sulphate and sodium salt, whilst two used modified media suggested in Manja et al. (2001).

We were unable to determine numbers of concordant and discordant test results in 9 of these 51 studies. This was because we lacked full text reports for two studies, numbers of concordant and discordant samples were not provided in the full text for six studies, and one further study was still ongoing. Three studies contained no false positives and true negatives or no false negatives and true positives, so we were unable to calculate sensitivity and specificity. There were a further three studies for which the numbers of concordant and discordant samples were described with a small degree of ambiguity in the original text. After excluding studies where we lacked numbers of concordant and discordant samples, this left 8279 samples available for meta-analysis.

We identified one study for which a preliminary report was available, but from which full results were not yet published (Khush et al. 2009). There were also 23 potentially relevant studies for which we were unable to obtain full texts, of which 4 were characterized based on abstracts and citations. Of the remainder, we did not obtain full texts because either these were oral presentations with no accompanying manuscript (12 studies) or the original report was unobtainable (7 studies).

The scores for the 14 study quality criteria for each study are summarized in Table 2. There were some generic strengths (e.g. simultaneous sample collection for both tests) and weaknesses (e.g. possible influence of one test result on the reading of the other) common to almost all studies. Scores for the remaining study quality items varied between studies. The test for funnel plot asymmetry suggested a low likelihood of publication bias (P = 0.29 for TTC; P = 0.13 for EC).

Table 2. Percentage of the 51 included studies meeting 14 study quality items, derived from a protocol used for studies of clinical diagnostics (Whiting et al. 2003)
ItemContentsYes (%)No (%)Unclear (%)
 1Were water samples drawn from areas where the H2S test would typically be deployed?78.4219.6
 2Do the authors describe how water sampling points were chosen?31.43.964.7
 3Were quality control procedures described for thermotolerant coliforms (TTC) or Escherichia coli testing?51.0049.0
 4Were water samples processed simultaneously using the H2S test and the TTC or E. coli test?10000
 5Were all water samples (or a random subset) tested for both H2S and TTC or E. coli?60.835.33.9
 6Did the TTC or E. coli test remain the same, regardless of the H2S test result?10000
 7Did the H2S test not form any part of the TTC or E. coli test procedure?10000
 8Were H2S test procedures described in sufficient detail to permit test replication?60.829.49.8
 9Were TTC or E. coli test procedures described in sufficient detail to permit test replication?25.564.79.8
10Were the H2S test results interpreted without knowledge of the TTC or E. coli test results?2098
11Were the TTC or E. coli samples interpreted without knowledge of the H2S test results?2098
12Were the same sample details available during test interpretation as would be available in practice?00100
13Were H2S samples that changed colour slightly (e.g. to light grey not black) reported?11.878.49.8
14Were water samples for which no results were presented explained?78.417.63.9

Forest plots of H2S test sensitivity and specificity for TTC and EC are shown in Figure 2. There was substantial heterogeneity in sensitivity for both TTC (I2 = 89.1% [85.7–92.5]) and EC (I2 = 93.6% [91.6–95.7]). Similarly, there was substantial heterogeneity in specificity for both TTC (I2 = 96.4% [95.6–97.2]) and EC (I2 = 95.4% [94.1–96.8]), with TTC specificity being more heterogenous than TTC sensitivity.

image

Figure 2.  Forest plots of the sensitivities and specificities of the H2S method compared to standard methods for the measurement of thermotolerant coliforms (TTC) and Escherichia coli. (a) sensitivity of TTC; (b) specificity of TTC; (c) sensitivity of E. coli; (d) specificity of E. coli. Hewison et al. (1988) reported two separate studies, presented as Hewison et al. (1988a) and Hewison et al. (1988b).

Download figure to PowerPoint

Figure 3 shows the test’s positive predictive value (proportion of all H2S positive samples that were true positives) relative to the percentage of samples with detectable TTC or EC. Unsurprisingly, positive predictive value was lower when few samples had detectable TTC or EC.

image

Figure 3.  Positive predictive value (true positives as a proportion of all H2S positive samples) relative to the percentage of samples with detectable thermotolerant coliforms or Escherichia coli for 41 included studies.

Download figure to PowerPoint

Meta-regression findings were inconsistent between TTC and EC and fewer explanatory variables were associated with specificity than sensitivity (Figure 4). Although some study quality items were associated with sensitivity for both TTC and EC, it was difficult to discern any consistent effect of study quality on diagnostic accuracy. Focussing on findings significant at the 1% level, lower sensitivity with TTC was associated with smaller H2S sample volumes, shorter incubation periods and higher incubation temperatures (Figure 4a) and with sampling strategies that focussed on remote settings and were well documented. Variation in specificity with TTC was not related to the variables presented in Figure 4b. Sensitivity with EC (Figure 4c) was lower for studies conducted on the Indian subcontinent, in studies predominantly based on groundwater samples and where microbiological procedures were described in detail. Specificity with EC was higher for studies from the Indian subcontinent (Figure 4d).

image

Figure 4.  Univariate meta-regression results, showing the influence of different H2S procedures, sample and study characteristics on H2S accuracy compared to standard methods for enumerating indicator bacteria. (a) sensitivity of thermotolerant coliforms (TTC); (b) specificity of TTC; (c) sensitivity of Escherichia coli; (d) specificity of E. coli.

Download figure to PowerPoint

Discussion

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

In contrast to a previous literature review (Sobsey & Pfaender 2003) of 13 studies, this quantitative analysis systematically synthesises evidence on H2S test diagnostic accuracy from 51 studies. Our study used meta-analysis to estimate heterogeneity in diagnostic accuracy and its causes. To our knowledge, meta-analysis of diagnostic accuracy has to date been applied to clinical diagnostics, rather than to environmental diagnostics as in this study.

Our review suggests that there are now many studies of H2S test diagnostic accuracy, although implementation varies between studies. Many of the studies analysed took place in India and Nepal, perhaps, reflecting widespread H2S test use there. There was, however, substantial heterogeneity in both sensitivity and specificity relative to both TTC and EC. As little of this heterogeneity was explained by the H2S procedures used, study quality or the types of sample processed, this unexplained variation could result from unmeasured differences in test procedures (e.g. the experience of test operators) or sample characteristics (e.g. variations in microbial ecology).

In most studies reviewed here, H2S test performance would be considered as moderate to high accuracy based on area under the receiver operating characteristic (ROC) curve (Swets 1988), although performance varied widely between studies and was sometimes poor. In almost all studies, its performance is poorer than that of recognized microbiological tests. For example, Colilert achieved 0.94 for both sensitivity and specificity compared with those of existing standard methods of the time (Edberg et al. 1988), whilst Colitag and Colisure achieved sensitivities of 0.96 and 0.98 and specificities of 0.93 and 0.96, respectively (NEMI 2009; McFeters et al. 1995). Its current status as a non-standard test for use in remote, resource-poor settings and for educational purposes seems consistent with its performance.

Meta-regression findings were inconsistent between TTC and EC, and fewer explanatory variables were associated with specificity than sensitivity. The greater sensitivity with TTC observed with higher H2S sample volumes and longer incubation times is consistent with that of previous research (Gupta et al. 2008). However, lower sensitivity with EC was observed in studies based on predominantly groundwater samples. This runs counter to an anticipated decrease in specificity in groundwater, owing to sulphide formation by non-faecal bacteria in anaerobic aquifers (Sobsey & Pfaender 2003). Sensitivity with EC was lower and specificity higher in studies from the Indian subcontinent, where H2S use is widespread. The lack of any strong association between test performance and bacterial contamination levels is surprising, given some previous findings (Gupta et al. 2008).

As is well recognized in clinical diagnostics (Grimes & Schulz 2002), the use of even a relatively accurate test may be problematic in settings where water quality is generally good. The proportion of times a positive H2S test result proves to be a ‘false alarm’ (its positive predictive value) depends on how often water source contamination occurs as well as test accuracy. Where contaminated sources are commonplace, positive H2S results frequently reflect genuine contamination (Figure 3). Where contaminated sources are rare, the small number of truly positive H2S results is often dominated by false positives, producing many more ‘false alarms’. A false-positive test result can potentially have undesirable consequences, such as causing people to boil their drinking water unnecessarily or switch to more distant water sources. The benefit of identifying a small group of contaminated sources through field testing may therefore be offset by unnecessary concern and action as a result of false positives from a larger group of water sources. There is thus a weaker rationale for using a moderately accurate test like H2S where water contamination is seldom encountered.

As H2S test accuracy is low in a minority of these studies, we recommend its performance be evaluated relative to standard methods, prior to its operational deployment in a new setting.

Limitations

Four issues could have affected our conclusions: We assumed that recognized microbiological methods are equivalent to one another. In reality, such methods may vary both in their ability to recover indicator bacteria injured through water treatment and in their precision. More generally, TTC and EC have their weaknesses as indicator bacteria (Gleeson & Gray 1997), and there is no definitive means of identifying faecally contaminated drinking water (Sobsey & Pfaender 2003).

We searched for relevant literature in English and Chinese, but may have missed reports in other languages. We were unable to obtain the full text of some potentially relevant reports. This may have influenced our findings if test accuracy was systematically different in these reports.

Included studies typically conducted H2S testing in the laboratory through skilled individuals. Whilst we found no systematic differences in accuracy between field- and laboratory-based studies, these studies may collectively overestimate diagnostic accuracy as a consequence. In particular, they do not evaluate the effectiveness of any associated training in water testing.

In studies with multiple comparison groups based on differing H2S procedures, we included only one group in our analysis as described earlier. This may have reduced our ability to discern an effect of H2S procedures on accuracy.

Aside from analysing groups of water samples, it may be possible to gain further insights by analysing individual water sample results, particularly in relation to differing indicator bacteria concentrations. More generally, we used indirect comparison to examine different H2S procedures between studies, but ignored any direct comparison of H2S procedures within studies. In clinical meta-analysis, techniques are developing to synthesize evidence from both direct comparisons within studies and indirect evidence across studies (Caldwell et al. 2005). Were these techniques to be extended to handle studies of diagnostic accuracy, there would be value in applying them to the H2S test literature.

The H2S test remains substantially cheaper than other alternatives. Consumables costs for a 100 ml presence/absence H2S test have been estimated as $0.35 per test, compared with over $1.50 per test for commercially manufactured alternatives (Chuang et al. 2011). A Chilean study estimated annual monitoring costs as $84.8 using H2S and field incubation, but $211.2 using laboratory-based membrane filtration (Castillo 1997), with savings on both transportation and testing procedures. This lower cost makes the test affordable where conventional methods are too expensive, but also means a larger number of H2S tests could be conducted for the same cost as a smaller number of conventional tests. Repeat presence/absence tests using H2S may be particularly valuable in supply systems where intermittent contamination events occur, such as those documented following pressure drops in piped systems.

Although not equivalent to laboratory testing, the H2S test remains an alternative in remote and resource-poor settings because of its low cost and ease of use. The lower cost of H2S also makes more frequent testing feasible, which may be particularly valuable in supply systems where intermittent contamination occurs. However, given the variable accuracy observed in studies reviewed here, we recommend benchmarking the test against standard methods, prior to its operational deployment in a new setting. We also recommend exercising caution when using the test where contaminated supplies are seldom encountered because false positives may outnumber true positives in this situation.

Acknowledgements

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

This research has been funded by the Bill & Melinda Gates Foundation, under a grant to the University of Bristol, whose purpose is to develop a low-cost test of water quality for use in developing countries. We thank Dr Patricia Lucas of the School of Policy Studies, University of Bristol for reviewing an early version of the protocol for this study and Prof. Paul Hunter of the University of East Anglia for providing overall comments on a version of this paper.

References

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information
  • Arends LR, Hamza TH, Van Houwelingen JC, Heijenbrok-Kal MH, Hunink MGM & Stijnen T (2008) Bivariate random effects meta-analysis of ROC curves. Medical Decision Making28, 621628.
  • Caldwell DM, Ades AE & Higgins JPT (2005) Simultaneous comparison of multiple treatments: combining direct and indirect evidence. British Medical Journal331, 897900.
  • Castillo M (1997) Water Quality Control Chile – Phase II. IDRC Project 3-P-90-0100-02 final report. International Development Research Centre, Ottawa, Canada.
  • Chuang P, Trottier S & Murcott S (2011) Comparison and verification of four field-based microbiological tests: H2S test, Easygel, Colilert, PetrifilmTM. Journal of Water, Sanitation and Hygiene for Development1, 6885.
  • Deeks JJ, Macaskill P & Irwig L (2005) The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. Journal of Clinical Epidemiology58, 882893.
  • Desmarchelier P, Lew A, Caique W et al. (1992) An evaluation of the hydrogen sulfide water screening-test and coliform counts for water-quality assessment in rural Malaysia. Transactions of the Royal Society of Tropical Medicine and Hygiene86, 448450.
  • Eaton AD, Glesceri LS, Rice EW & Greenberg AE (2005) Standard Methods for the Examination of Water & Wastewater, 21st edn. American Publich Health Association, American Water Works Association and Water Environment Federation, Washington, DC.
  • Edberg SC, Allen MJ & Smith DB (1988) Rapid, specific, defined substrate technology for the simultaneous detection of total coliforms and Escherichia coli. Toxicity Assessment3, 565580.
  • Genthe B & Franck M (1999) A Tool for Addressing Microbial Water Quality in Small Community Water Supplies: An H2S Strip Test. Water Research Commission Report No. 961/1/99. Water Research Commission, Pretoria, South Africa.
  • Gleeson C & Gray NF (1997) The Coliform Index and Waterborne Disease: Problems of Microbial Drinking Water Assessment. E & FN Spon, London.
  • Grimes DA & Schulz KF (2002) Uses and abuses of screening tests. The Lancet359, 881884.
  • Gupta SK, Sheikh MA, Islam MS et al. (2008) Usefulness of the hydrogen sulfide test for assessment of water quality in Bangladesh. Journal of Applied Microbiology104, 388395.
  • Khush R, London A, Colford J, Arnold B, Balakrishnan K & Ramaswamy P (2009) Evaluating the Sustainability and Impacts of Water, Sanitation & Hygiene Interventions. Aquaya, San Francisco, CA.
  • Manja KS, Maurya MS & Rao KM (1982) A simple field test for the detection of faecal pollution in drinking water. Bulletin of the World Health Organization60, 797801.
  • Manja KS, Sambasiva R, Chandrashekhara KV et al. (2001) Report of Study on H2S Test for Drinking Water. UNICEF, New Delhi.
  • McFeters GA, Broadaway SC, Pyle BH, Pickett M & Egozy Y (1995) Comparative performance of ColisureTM and accepted methods in the detection of chlorine-injured total coliforms and E. coli. Water Science & Technology31, 259261.
  • Mosley LM & Sharp DS (2005) The Hydrogen Sulphide (H2S) Paper-Strip Test: A Simple Test for Monitoring Drinking Water Quality in the Pacific Islands. SOPAC Technical Report 373. Suva, Fiji Islands.
  • NEMI (2009) Modified Colitag Test Method for the Simultaneous Detection of E. coli and other Total Coliforms in Water (ATP D05-0035). Available at: http://www.nemi.gov/apex/f?p=237:38:3713534863451572::::P38_METHOD_ID:10619 (accessed 31 August 2010).
  • Prüss A, Kay D, Fewtrell L & Bartram J (2002) Estimating the burden of disease from water, sanitation, and hygiene at a global level. Environmental Health Perspectives110, 537542.
  • Singh MP, Srivastata RK, Bachani D & Khare KK (1985) Detection of faecal pollution of drinking water in rural areas: some observations. Indian Journal of Preventive and Social Medicine16, 1215.
  • Sobsey MD & Pfaender FK (2003) Evaluation of the H2S Method for Detection of Fecal Contamination of Drinking Water. World Health Organisation, Geneve.
  • Swets JA (1988) Measuring the accuracy of diagnostic systems. Science240, 12851293.
  • UNICEF (2007) UNICEF Handbook on Water Quality. UNICEF, New York.
  • Vasudevan P & Tandon M (2008) Microbial quality of rainwater from roof surfaces. Journal of Scientific & Industrial Research67, 432435.
  • Whiting P, Rutjes AWS, Reitsma JB, Bossuyt PMM & Kleijnen J (2003) The development of QUADAS: a tool for the quality assessment of studies of diagnostic accuracy included in systematic reviews. BMC Medical Research Methodology3, 25.
  • WHO (2008) Guidelines for Drinking-Water Quality, 3rd edn, Volume 1: Recommendations. World Health Organization, Geneva.

Supporting Information

  1. Top of page
  2. Abstract
  3. Introduction
  4. Methods
  5. Results
  6. Discussion
  7. Acknowledgements
  8. References
  9. Supporting Information

Appendix S1. Supporting References.

Table S1. List of bibliographic databases and grey literature sources searched as part of the systematic review.

FilenameFormatSizeDescription
tmi2887_sm_Appendix1.doc42KSupporting info item
tmi2887_sm_TableS1.doc28KSupporting info item

Please note: Wiley Blackwell is not responsible for the content or functionality of any supporting information supplied by the authors. Any queries (other than missing content) should be directed to the corresponding author for the article.