Unequal, Unreliable, and Unfixable?—The Need to Investigate Water Infrastructure Improvements in Intermittent Supply Systems

Infrastructure deterioration threatens urban water security and the pursuit of Sustainable Development Goal 6 (SDG 6, “water for all”). One billion urban residents worldwide already face intermittent piped water access, making their public water supply unreliable and unequal. Thus, effective investments in water infrastructure improvements are critical. In a recent WRR paper, Jeuland et al. (2023, https://doi.org/10.1029/2022WR033897) evaluate the impacts of a large infrastructure improvement project on various water security metrics and assess the challenges of such an investigation. They apply a rigorous difference‐in‐difference approach to data from a US$275 million investment in upgrades to water supply and sewage networks in the Zarqa governorate in arid Jordan, analyzing its effects on households, businesses, farms, and the water utility. The authors find a range of moderate improvements to water access and re‐use metrics, including a significant reduction in reported shortages. Key water security indicators related to access intermittency, however, such as the supply duration per day and the expenditure on water deliveries by tanker trucks, saw little to no improvement. The findings reveal that the efforts required to overcome the insecurity and inequity of intermittent public water supply systems might be considerably larger than expected. This suggests an urgent need to further enhance both the accuracy of impact evaluation methods and the effectiveness of infrastructure investments to support the pursuit of SDG 6.

particularly in the case of intermittent water supply systems, as complex causal chains link infrastructure upgrades to water security improvements, and confounding factors impede their measurement (Simukonda et al., 2018).How can successful investment projects be designed despite these obstacles?A recent WRR paper by Jeuland et al. (2023) provides rare insights into the challenges of implementing and evaluating a large-scale investment in water infrastructure improvements.The authors describe a rigorous impact evaluation (IE) for an extensive investment in water and sewage infrastructure upgrades in Jordan's Zarqa governorate.Despite the successful execution of this investment project, systematic data collection efforts, and a suitable difference-in-difference (DiD) design for the IE study, the magnitude of the statistically significant impacts identified by the authors is only moderate.
In the remainder of this short comment, I will discuss the study, its findings, and their implications in the context of the challenge to overcome water supply intermittency.The limited investment impacts found by the authors suggest that addressing these widespread public water infrastructure deficits could require considerably larger efforts than we anticipate.

Study Background
The study is motivated by the scarcity of secure knowledge about the effectiveness of water infrastructure investments.The authors identify a research gap regarding IE studies comprehensively assessing the effects of large-scale improvements to existing water and sanitation infrastructure.They aim to address this gap in a case study evaluating the impacts of a US$275 million investment project by the Millennium Challenge Corporation in improvements to the water supply network (US$76 million) and to the sewage system in Jordan's Zarqa governorate, which were implemented between 2013 and 2017.
As one of the most water-stressed nations worldwide, Jordan provides a striking example of the water infrastructure challenges many arid countries face, including scarce freshwater resources and deteriorating supply networks (Whitman, 2019).Public water providers in the country introduced intermittency in 1987 to ration water and reduce losses due to leaks (Potter & Darmame, 2010).The resulting supply interruptions make public water supply in the country unreliable and unequal, with major impacts on citizens' water security (Klassert et al., 2018).To cope with these public water supply deficits, households and firms in Jordan use storage tanks (Rosenberg et al., 2008) and rely heavily on expensive water deliveries from privately operated tanker trucks (Klassert et al., 2023).The governorate of Zarqa is particularly affected by intermittency (Jeuland et al., 2023).
Since the introduction of intermittency rationing in Jordan, leakage rates have grown, possibly as a consequence of the frequent supply interruptions.The increased losses make it increasingly difficult to switch back to a continuous supply regime (Klassert et al., 2018).Jordan's water security challenges are projected to grow dramatically as water scarcity becomes increasingly severe throughout the country (Yoon et al., 2021).In light of this situation, water infrastructure improvements seem particularly urgent in Jordan.The case study setting selected, therefore, appears conducive to measuring the impacts of investments in water infrastructure in general and intermittent public water supply systems in particular.

Research Objective and Design
The study highlights the methodological and practical challenges of designing a suitable IE approach for assessing large-scale water infrastructure investments, particularly in intermittent supply settings.The investment project studied aimed to improve Zarqa's public water supply network, wastewater network, and water treatment plant.The supply network improvement measures included repairing leaking pipes, upgrading a reservoir, and replacing meters.Key wastewater network improvements were connecting new customers and repairing and replacing pipes.The project enhanced the As-Samra wastewater treatment plant's capacity and effectiveness.
The stated goal for the improvements included in the investment project was to enhance water and sanitation access and thereby generate positive economic impacts for urban households and businesses in Zarqa and downstream farms in the Jordan Valley.A more frequent piped water supply with higher pressure was expected to reduce urban water users' demand for costly water deliveries from tanker vendors, substantially reducing their water expenditure.The authors adopted the project's objectives in their evaluation and added positive impacts on health and water utility performance as potential benefits to investigate.
The authors aimed to develop and implement an approach enabling a rigorous attribution of identified impacts to the investment efforts.The approach needed to contend with various challenges related to the complexity of the large-scale infrastructure improvements themselves and of their linkages to the intended outcomes.This is particularly difficult in the case of intermittent water supply systems, whose performance is strongly determined by water management decisions and where many of the evaluated outcomes depend on soft mediating factors.Reductions in the use of water from private vendors, for example, require trust in the reliability of public water supply (Galaitsi et al., 2016).
In the study, the authors compare a range of potential IE techniques, including randomized controlled trial (RCT) and DiD approaches, based on their validity, relevance, and practicality.The authors assess the RCT approach and other approaches requiring random assignment of the study treatment as impractical in the context of large water infrastructure improvements.Purely observational studies are judged as unsuitable in cases where the study site selection is not random.The authors opt for an approach combining DiD and observation matching, using survey and metering data from a treated population in Zarqa and non-treated comparison samples in both Zarqa and neighboring Amman (n = 3,359), collected over 5 years (2014)(2015)(2016)(2017)(2018).This approach has the advantage of controlling for trends in the evaluated variables affecting both treated and control populations without requiring random assignment.This design seems suitable to measure the positive impacts of the water infrastructure investments under investigation.

Significant Investment Impacts but Limited Intermittency Reduction
The DiD approach allows the study to demonstrate various significant impacts of the infrastructure investments, including increases in the use of public water by households and treated wastewater by farms.The magnitude of many of the benefits identified is moderate, however, and some anticipated impacts could not be confirmed, despite the suitable case study selection and IE design.The study provides evidence of a moderately successful implementation of the planned infrastructure upgrades based on utility performance indicators.Households reported significant improvements to water pressure and network quality on subjective scales, experienced fewer supply shortages, and raised their public water use by about one cubic meter per household per month on average.Positive impacts for businesses in Zarqa could not be confirmed.
Notably, the study finds only minor positive impacts on variables directly related to public supply intermittency.Improvements to the piped water supply duration of treated households in Zarqa are not statistically significant compared to untreated households in Zarqa.Compared to untreated households in Amman, the supply duration improvements were statistically significant but limited to a magnitude of 1-2 hr per day.At the same time, treated businesses in Zarqa experienced a significant supply duration reduction of 8 hr per day compared to non-treated businesses in Amman.Households also did not significantly reduce their expenditure on tanker water deliveries by private vendors or their overall water expenditures, thus failing to meet one of the stated objectives of the infrastructure improvement project.These limited improvements in intermittency-related variables are particularly relevant given the pronounced negative effects of public supply interruptions on households and businesses in Jordan.
The limited magnitude of various positive impacts of the infrastructure investment could indicate either that no further improvements were achieved, or that the statistical approach used did not detect them, or both.The reason is that many of the impacts evaluated are linked to infrastructure improvements by a complex chain of preconditions.As a consequence, the IE analysis faces multiple sets of obstacles with regard to both the implementation and the measurement of these impacts.Figure 1 illustrates this for increases in public supply durations and reductions in tanker water expenditure.Measuring reduced tanker water expenditures, for example, requires implementation steps on the utility level and the household level and a successful statistical measurement of the reduction.Each of these steps faces a set of obstacles (obstacles 1.a, 1.b, and 2.b in Figure 1).
On the implementation side, there is some evidence that the planned water infrastructure upgrades were executed successfully, such as a reduced number of pipe bursts after the investment and increased billing efficiency.The findings for other indicators are less clear: Utilities saw only minor reductions in water losses per subscriber after the intervention, for example, compared to the implementation period, and increased losses compared to the pre-intervention period.These mixed results leave some uncertainty about how effective the implementation of the planned infrastructure upgrades was.A less effective implementation could have contributed to smaller improvements regarding both supply durations and tanker water expenditures (obstacle 1.a).Further obstacles could have impeded the intended substitution of piped water for tanker water (1.b), such as a lack of trust in public supply or variability and heterogeneity in water demands (Zozmann et al., 2022a).
On the measurement side, the authors mention that spill-overs of investment benefits to non-treatment areas, a statistically underpowered design, and a simultaneous but unrelated utility corporatization posed obstacles to the detection of some positive impacts (obstacles 2.a and 2.b).The study managed to mitigate the effects of spill-overs on the IE results by using two comparison samples.Attenuating effects of the corporatization on some metrics and statistical power limitations are valid concerns.The comparatively large data set and sound design, however, combined with the fact that significant effects were found for several variables, make it unlikely that methodological limitations fully account for the absence of other anticipated impacts.The results, therefore, point to a combination of implementation and measurement obstacles impeding the successful confirmation of all foreseen benefits in the expected magnitude.

An Urgent Need to Improve Infrastructure Investments and Impact Evaluations
Increasing water scarcity and the prevalence of intermittent public water supply systems around the world create an acute need for large-scale water infrastructure investments.Jeuland et al.'s (2023) study shows that considerable methodological and practical challenges impede a reliable confirmation of the positive impacts of such investments.The study demonstrates the general suitability of a DiD approach to measure the benefits of water infrastructure improvements but also reveals limitations.Given the high investment volume and the IE study's systematic design, the failure to confirm various anticipated impacts to the expected extent also provides valuable insights.Public water supply durations saw only minor improvements and tanker water expenditures no reduction, despite a US$76 million investment dedicated to supply system upgrades.Related objectives that could foster reduced intermittency and tanker water demand were only achieved to a moderate degree.Taking these measured results at face value would suggest that overcoming the insecurity and inequity of Jordan's intermittent water supply system is prohibitively expensive and that reducing the country's heavy reliance on tanker water deliveries is unachievable.The limited impacts measured by the study might not reflect the full magnitude of the benefits attained, however, as the above discussion suggests.They are presumably also influenced by the shortcomings of current analysis methods.Methodological advances in both the implementation and evaluation of large water infrastructure improvement projects are, therefore, urgently needed.The scarce existing literature only provides limited guidance.Further research efforts should accompany future water infrastructure improvement projects to systematically evaluate their impacts.
The results of the present IE analysis already offer some insights into methodological improvements available to future studies.Addressing potential spill-overs and targeting issues through coordination between researchers, investment planners, and water authorities should be a high priority.In addition, the use of multiple comparison groups in the DiD approach was shown to be a promising strategy for reducing the consequences of spill-over effects for the evaluation.The authors suggest that a more extended evaluation period might have been required to detect all benefits of the investment efforts.Further investigations into the complex causal linkages between water infrastructure upgrades and water access improvements could contribute to developing early indicators for these long-term impacts.

Figure 1 .
Figure 1.Potential obstacles to measuring improvements in supply duration and water vendor expenditure.The presence of implementation (1.a., 1.b) or measurement obstacles (2.a, 2.b) or combinations of both could impede the detection of infrastructure improvement impacts.