Does supplemental irrigation enhance smallholder monsoon season rice yield? Evidence from Bangladesh

Irrigation is one pillar of the Green Revolution that drove dramatic agricultural productivity gains across Asia. In Bangladesh, irrigation uptake has been so significant that 97% of dry‐season rice is now irrigated. While most Bangladesh monsoon rice is completely rainfed, supplementary irrigation is sometimes employed where late monsoon onset is potentially yield‐limiting. Station‐controlled experiments provide a narrative of positive yield benefits from supplementary irrigation. In contrast, statistical evaluations of actual farm experience mostly show no yield benefit and lower profitability for supplementary irrigation adopters. To add evidence on this controversial practice, we evaluated data from 2012 and 2015 Bangladesh farm household surveys with causality econometric approaches that control for differences between supplementary irrigation adopter and non‐adopter groups. After controlling for self‐selection and endogeneity, we found no statistically significant yield benefit for supplementary irrigation. Our results support scepticism about the profitability of supplementary irrigation. As such, we recommend careful consideration of the mixed evidence on effectiveness in future supplementary irrigation project benefit cost analyses. Further evidence over a longer time and accounting for a broader range of crops is also important moving forward.


| INTRODUCTION
Irrigation is a well-established method for increasing agricultural yields in areas where rainfall is scarce (Carruthers et al., 1997;Domenech & Ringler, 2013;Evenson & Gollin, 2003;Hanjra et al., 2017;Huang et al., 2006;Hussain & Hanjra, 2003, 2004;Lipton et al., 2003).During the Green Revolution of the twentieth century, irrigation, in conjunction with improved seed and fertilizer, drove large productivity gains throughout Asia (Bhattarai et al., 2002).Bangladesh is the fourth largest rice-producing and rice-consuming country in the world and is an attractive case study to investigate the relationship between supplementary irrigation and rice yields today (Food and Agriculture Organization [FAO], 2018; Wailes, 2005).Following the country's independence from Pakistan, a devastating famine throughout Bangladesh (Dowlah, 2006) led to successive government interventions to increase food production and attain grain self-sufficiency (Ministry of Agriculture [MoA], 2018;Tarrant, 1982;Taslim, 2022).As a result, rice production has grown 284% over the past five decades (Bangladesh Bureau of Statistics [BBS], 2021).This was driven by high-yielding seed breeding and diffusion, more fertilizer, chemicals, machines and increased land use intensity with more plots cultivated in two and three seasons (Bhattarai et al., 2002).
Irrigation was also integral to the increased productivity, allowing crops to grow year-round, even in the dry season when rainfall would otherwise be limiting.By 2020, over 97% of rice grown in the dry season (i.e.boro rice) was irrigated (BBS, 2021).Aman (monsoon season) rice is traditionally cultivated in a rainfall-dependent manner.However, supplementary irrigation can improve yield when late-onset rainfall reduces yield potential (Satter & Parvin, 2009).While regular irrigation is typically applied during the dry season to provide a consistent supply of water to crops, supplementary irrigation is used to supplement natural rainfall during the monsoon season and provide additional water during unseasonably dry periods.As such, supplementary irrigation is defined as 'the addition of a limited amount of water to otherwise rainfed crops when rainfall fails to provide essential moisture for normal plant growth to improve and stabilize productivity' (Oweis & Hachum, 2005).
With most existing avenues for dry-season irrigation already utilized, the Bangladesh government continues to construct new dams and instigate irrigation projects for supplementary irrigation.Example projects include the 'Construction of rubber dams in small and medium rivers for increasing food production' and the 'Integrated agricultural productivity project'.Both endeavours provide supplementary irrigation to address the soil moisture deficit experienced with late monsoon onset (Bangladesh Agricultural Development Corporation [BADC], 2015).
A significant limitation of controlled environment experimental plot studies is that they are likely to overstate what can be achieved in the less controlled conditions expected on real-world farms (Cassman, 1999;Cassman et al., 2003).Stuart et al. (2016) showed that farmers realized 23-42% lower yields in field settings compared to research station experiments where water and nutrients were artificially manipulated.The limited evidence from actual farmers' field conditions is less convincing than the experimental plot evidence.For example, Rahman (2011) found no statistically significant effect of supplementary irrigation on aman rice yield gain in the Teesta irrigation project area in Bangladesh.Sarkar (2021) also found a statistically insignificant relationship between supplementary irrigation and wet-season rice yield on real farms in five districts in Bangladesh.Furthermore, an analysis by Ahmed (2004), using actual national-level farm data from Bangladesh, showed that farmers who adopted monsoon season rice supplementary irrigation achieved 12.8% less net return on average compared to farmers who grew monsoon rice without supplementary irrigation.While these studies used actual farm data, they did not use state-of-the-art techniques to control systematic differences between those who did and did not adopt supplementary irrigation.
This article is motivated by the need for more conclusive evidence from actual farm settings that control for potential bias that can arise in statistical comparison of adopter and non-adopter outcomes.This is significant given that the belief that supplementary irrigation can improve yields justifies many supplementary irrigation projects in Asia, including significant Bangladesh government spending (BADC, 2015;Banglapedia, 2012;Oweis & Hachum, 2009a;Oweis & Hachum, 2009b;Rahman, 2011).
Causality econometrics is an approach to account for treatment and control sample differences in establishing causality between the use of supplementary irrigation and variables such as yield and profit.Causality econometrics is becoming increasingly popular in social science, finance, medical research and development economics (Baker et al., 2022;Imbens, 2022;Wing et al., 2018).The differences-in-differences (DID) method is a set of popular causality econometric approaches sometimes referred to as workhorse tools of the creditability revolution (Baker et al., 2022).To date, only a few irrigation economics studies have used causality econometric and DID approaches (e.g.Pronti & Berbel, 2023;Smith et al., 2017).This is despite the obvious importance of properly attributing impact to irrigation in project evaluations and benefit cost analyses.
Here, we present the results of a study that aims to: 1. Add more conclusive evidence on supplementary irrigation and farmer yield impacts in countries with ongoing investment supporting supplementary irrigation.2. Investigate causality in monsoon rice supplementary irrigation and yield relationships accounting for cofounding differences that can lead to misattribution of supplementary irrigation treatment effects.

| METHODOLOGY AND DATA
Our approach to robustly isolate supplementary irrigation treatment effects involved three steps.First, we applied a propensity score matching (PSM) control treatment sample matching algorithm to match control and treatment samples based on observable attributes likely to influence outcome.This helps to eliminate observations from each group that are too dissimilar.Second, we tested the validity of the equal time trends assumption for explanatory variables for the control and treatment groups.Third, upon finding significant time-varying differences, we applied DID regression controlling timevarying variables, a standard approach when trends in outcome determinants change differently over time for treatment and control groups (Gertler et al., 2016).

| Matching the treatment and control samples
To obtain unbiased estimates, it is crucial that treatment and control groups are similar in observable attributes that might influence the outcome of interest.To address this, we applied PSM, a matching algorithm to assure greater similarity in control and treatment samples prior to regression (Abate et al., 2014;Ainembabazi et al., 2017;Heckman et al., 1997;Heckman et al., 1998;Rosenbaum & Rubin, 1983;Smith & Todd, 2005).PSM identifies a subpopulation including only nonadopters sufficiently close to the population of adopters to assume that the distribution of unobservable characteristics is the same, or at least not so different, for both groups that it possibly induces a bias (Becker & Ichino, 2002).The probability that a household has access to the treatment is used as the propensity score in this matching process (Dillon, 2011).
To estimate propensity scores, a probit model is used to regress a binary variable, P, with P = 1 indicating a household accessed supplementary irrigation, on a vector of household characteristics, X.The results are used to obtain predictions of household propensity to adopt supplementary irrigation.Then, the region of common support is established by eliminating the observations from the comparison group (i.e.non-adopters), with a p-score lower than the minimum p-score in the treatment group, and from the control group with a p-score higher than the maximum p-score of the treatment group (Van Rijsbergen et al., 2016), as shown in Figure 3 (see later in the article).
The exogenous variables included in the regression were selected based on findings that similar metrics were significant in previous irrigation adoption propensity studies (Bacha et al., 2011;Dillon, 2011;Nonvide, 2019).This included (i) land area, (ii) main occupationagriculture or not, (iii) gender of the head of household, (iv) access to credit, (v) have livestock, (vi) have fisheries and (vii) non-farm income.
The probit regression can be expressed as where PS = propensity score, L = land area, O = occupation, G = gender of the household head, C = access to credit, Li = have livestock, Fi = have fisheries, I = non-farm income.

| Basic DID method
After the probit regression and matching, we applied DID to identify the impact of the supplementary irrigation adoption binary variable, X, on the rice yield outcome, Y.This relationship is dependent on the participation status in time period t.Pre-and post-treatment periods are denoted by t 0 and t 1 , respectively.The difference in outcome of the treatment group (Y 1 ) before and after the treatment period is calculated as follows: The difference in outcome of the control group (Y 0 ) before and after the treatment is calculated as follows: The DID between the treatment group and the control group is calculated as In a regression format, the DID model can be written as (Abadie, 2005;Angrist & Pischke, 2008) where Y is the yield per hectare, d2 is a dummy with value 0 for the first time period and value 1 for the second time period and dT is equal to unity for the treatment group T in period 2. The treatment effect is the coefficient δ of the multiplication of d2 and dT.

| Controlling for time-varying treatment and control group differences
One of the central assumptions of the DID approach is that factors influencing outcome other than treatment follow similar trends over time for both treatment and control groups.However, the absence of time-varying differences between the control and treatment groups is a strong assumption (Gertler et al., 2016) given the significant change in key attributes likely to influence yield from 2012 to 2015, a time of rapid and variable economic development in Bangladesh.One solution is to identify and control for these time-varying attributes by including them in DID regression (Angrist & Pischke, 2008).Our data showed several time-varying trends that were statistically significantly different for the control and treatment groups, including trends in total labour hours, fertilizer cost, pesticide cost, compost cost, number of consecutive no-rainfall days in September and October, and improved crop variety or improved seed use (see Table 4).To account for these variables, we included them in our analysis.
Other attributes, including machinery cost, other income, land type, farmers' educational quality, age and other indicators of farm and human capital, were found to have similar trends over time for the control and treatment groups.Hence, we did not control for these variables.Denoting the time-varying variables included as σ i U i , our regression equation can be written in standard time-varying DID format (Abadie, 2005;Sant'Anna & Zhao, 2020;Wooldridge, 2016): where U i is a vector of time-varying variables: fertilizer cost, rice variety (improved or local), total labour hours, pesticide cost, compost cost and number of consecutive no-rainfall days, and σ i is a vector of regression coefficients for these variables.

| Data
We extracted data from the Bangladesh Integrated Household Survey (2012 and 2015) administered by the International Food Policy Research Institute (IFPRI).This survey is statistically representative of rural Bangladesh and includes all administrative divisions.The total sample size is 5500 households.The survey questionnaire consists of several modules on multiple aspects of living standards, including data on individual-level education, employment, household demographics, assets, and their values, income, expenditure, microcredit, disaster mitigation, migration and remittances.The survey's primary focus is agricultural production, including crop yield, input use levels for hired and family labour, seed, fertilizer, pesticide, irrigation and land characteristics.
Importantly, the survey collected information on irrigation sources.The farmer selected 'rainfed' as a source of water in instances where no other irrigation is applied.The farms in the treatment group listed rainfed as their only water source in 2012 and shallow or deep tube wells as a water source in 2015; the control group listed rainfed as the only source of water available to their aman season rice crop in both 2012 and 2015.The data set along with the questionnaire template is publicly available at Harvard Dataverse.Further information on the questionnaire template and questions included in it is available at the following URL: https://dataverse.harvard.edu/file.xhtml?persistentId=doi:10.7910/DVN/BXSYEL/ VVQXD9&version=4.1.
Aman rice is traditionally fully rainfall-dependent cultivation.However, supplementary irrigation can be advantageous when the late onset of rainfall reduces yield potential (Satter & Parvin, 2009).Total seasonal rainfall is a poor indicator of deficit in this critical time because rainfall always exceeds aman rice's full season irrigation water requirements, which vary from 80 to 350 mm in Bangladesh (Kirby et al., 2014).A better indicator of the potential benefit for yield benefit from supplementary irrigation is consecutive days with no rainfall in September and October.Greater than 15 days without rain in this period is indicative of seasonal drought likely sufficient to impact aman rice yield (Kirby et al., 2014).To test whether differences in aman drought conditions impacted yield differently for the control and treatment groups, we constructed a consecutive day with a no-rainfall indicator for the monsoon season start.It is calculated as the maximum number of consecutive no-rainfall days in September and October, with results by district shown in Figure 1.
The observable correlation between the maximum number of consecutive no-rainfall days and conversion to supplementary irrigation (Figure 2) was confirmed in 2015, and the correlation value was 0.34 and 0.2 in 2012.These results support the suggestion that the indicator for prolonged seasonal drought should be included in the regression on conversion from rainfed to supplementary irrigation.

| Summary statistics
Summary statistics for the variables included in the probit regression and DID regressions for the control and treatment groups in 2012 and 2015 are shown in Table 1.
The control group in our sample was 3978 plots that applied no supplementary irrigation to rice in both sampled years, and the treatment group was 297 plots where supplementary irrigation was applied to rice in 2015 but not in 2012.The treatment group and control group plots were similar in size, non-farm income, income from fisheries, access to credit and agricultural extension, household head age and proportions of households headed by women.The treatment plots had more livestock, used more improved variety seed and listed their main occupation as agriculture more often.On the other hand, they had less machinery cost and employed less labour effort.In 2015, the treatment group spent more time on their rice farms and experienced higher fertilizer costs.

| Matching the treatment group and control group
To investigate the impact of supplementary irrigation on rice yield gain, we matched the treatment group and comparison group based on observable characteristics of farmers before applying the DID approach.The estimated propensity score represents the probability of access to supplementary irrigation shown in Table 2 for the variables that have the potential to be different across adopter and non-adopter subsets in ways that should be accounted for to satisfy the balancing property of similarity between the treatment and control groups.
The PSM algorithm for selecting a subset of all potential control and treatment samples is designed to select samples with observable characteristics distributed equivalently across quintiles in both the treatment and control groups (Dehejia & Wahba, 2002;Smith & Todd, 2005).The results showed that if agriculture is the main occupation and if the household had more livestock, then they have a higher probability of having access to supplementary irrigation and being included in the treatment group.
Then, we matched the control group with the treatment group by eliminating the observations with a propensity score lower than the minimum p-score in the treatment group and with a p-score higher than the maximum p-score of the treatment group (Van Rijsbergen et al., 2016).Figure 3 shows the probability score of the treatment and control groups before and after matching.It is evident from the figure that the main impact of matching was to eliminate extreme sample points to the far right and left of the propensity distribution.

| DID without controlling the timevarying variables
After matching to improve the comparability of the treatment and control samples, we estimated the mean yield per hectare of the control and treatment groups in 2012 and 2015 without controlling for time-varying variables using the DID approach.The results are shown in Table 3.
The results showed a considerable 0.365 t ha À1 (13%) and statistically significant yield benefit of the treatment group compared to the control group.

| DID controlling time-varying variables
We deployed t-tests to investigate the time-varying attribute of relevant variables.The t-test results showed statistically significant variance between the two time periods for the following variables: maximum number of consecutive no-rainfall days; rate of uptake of high-yielding rice variety; fertilizer, pesticide and compost input expenditures; and labour input per hectare (Table 4).This indicates that these variables should be included in the DID specification to obtain an unbiased estimation.
Hence, we included the above-mentioned variables in the DID specification.The DID results after controlling for time-varying variables are shown in Table 5.
The statistically significant impact of supplementary irrigation that we observed in Table 4 disappeared when we controlled the time-varying variables in the DID specification.The results in Table 5 show a statistically insignificant supplementary irrigation yield effect in the treatment group compared to the comparison group.The results also suggest that higher fertilizer, compost and pesticide input levels and improved seed variety had a significant positive impact on rice yields, whereas the impact of improved variety was highly significant and had a high magnitude.

| DISCUSSION
This work highlights the importance of using observations from farm field sites.While previous research has resulted in findings consistent with this work, for example Ahmed (2004) showed that farmers who adopt supplementary irrigation for monsoon season rice do not necessarily improve yield and often earn less net profit than non-adopters, other field station-based local studies have shown the opposite to be true (Boonwichai et al., 2019;Mohanty et al., 2018;Panigrahi et al., 2015;Panigrahi & Panda, 2003;Singh et al., 2020).F G U R E 3 Graphical presentation of propensity score before and after matching.
Causality econometrics is a suite of approaches that help account for differences in control and treatment samples, which would otherwise lead to misattribution of a treatment's impact.In this study, we first applied a PSM algorithm.This process compares treatment and control samples and eliminates observations that have implausible differences in observable characteristics.Next, we tested for equal trends over time in explanatory variables in the treatment and control samples.Based on our finding that trends differed for several attributes, we included them in our final DID model, which included a time-varying effects econometric specification.
Importantly, this work illustrates that accounting for confounding time-varying effects can drastically change the outcome of modelling.We would have drawn the opposite conclusion had we not accounted for confounding time-varying effects in our ultimate DID specification.
The time-varying DID model results highlight how context influences the benefits of supplementary irrigation.For example, the positive and statistically significant coefficient of the maximum number of consecutive no-rainfall days variable indicates that supplementary irrigation would have been more beneficial in years with later monsoon onset.We also found that most supplementary irrigation adopters simultaneously adopted high yielding variety (HYV) seeds.Our time-varying DID specification illustrates that failing to attribute the impacts of this change would have led to misattribution of observed yield differences.
One limitation of this study is the use of only 2 years' worth of data.Further insight could be gained by extending time series observations.Conducting similar research to examine variations across variables, such as monsoon onset, will be an important next step in this area of research.Rice is a culturally and nutritionally significant crop in Bangladesh and other parts of the world.Nonetheless, consideration beyond just rice to understand shifts in the mix of crops and crop quality improvements could provide further insights.Additionally, incorporating alternate welfare metrics, such as income and nutritional outcomes, could uncover valuable insights.

| CONCLUSIONS
This work critically analyses the effect of supplementary irrigation on the yield of monsoon season rice.Using causality econometrics on farm field-level data, we found that adopters of supplementary irrigation did not achieve a statistically significant yield benefit compared to nonadopters.The two observational groups were farmers who adopted supplementary irrigation in monsoon rice between 2012 and 2015 and farmers who continued with rainfed production in both years.Our overall conclusion is that supplementary irrigation provided limited benefits in the year that we assessed, but it could in years with more protracted delay in monsoon onset.This aligns with findings from other real farm data-based studies in Asia.For example, research from Cambodia also found little benefit from wet-season irrigation on rice, other than in years of drought (Smith & Hornbuckle, 2013) and Oweis andHachum (2009a, 2009b) found that supplementary irrigation was mostly less profitable than rainfed production, except where the additional cost is small relative to production or price premium benefits.Factoring the mixed findings on benefit to date into realistic and evidencebased future benefit-cost analyses is advised.

F
I G U R E 1 Maximum number of consecutive no-rainfall days in September/ October in 2012 and 2015 by district.

F
Summary statistics of attributes of irrigated and non-irrigated plots in 2012 and 2015.
Impact of adopting supplementary irrigation on per hectare yield estimated with DID controlling for unequal time trends.p<0.1.Significant at the 10% level.**p<0.05.Significant at the 5% level.***p<0.01.Significant at the 1% level.T A B L E 4 t-Tests of control and treatment group equal time trend for explanatory variables.Note: BDT is Bangladeshi Taka (money).*Significant at the 10% level.