Assessing Cost‐Effectiveness When Environmental Benefits are Bundled: Agricultural Water Management in Great Barrier Reef Catchments

Using economic analysis to prioritise improvements in environmental conditions is particularly difficult when multiple benefits are involved. This includes &#8216;bundling&#8217; issues in agricultural pollution management, where a change in management action or farming systems generates multiple improvements, such as reductions in more than one pollutant. In this study, we conceptualise and compare two different approaches to analysing cost&#8208;effectiveness when varying bundles of benefits are generated for a single project investment. Each approach requires data to be transformed in some way to allow the analysis to proceed. The index approach requires the transformation on the benefits side so that the effects of multiple pollutant changes can be combined into a measure for each project which can then be compared to costs. By comparison, the disaggregation approach requires the transformation on the costs side where costs for each project have to be apportioned across the different pollutants involved. The paper provides novel insights with an application to agricultural water quality improvements into the Great Barrier Reef in Australia, demonstrating that while both approaches are effective in prioritising projects by cost&#8208;effectiveness, the disaggregation approach provides more insightful results and values that may be relevant for use as upper value guidelines in future project selection.


Introduction
Managing diffuse sources of pollutants in large catchments is challenging, not only because of the difficulties of pinpointing sources, but also because of multiple pollutants that may be involved (Bloodworth et al. 2015). Evaluating options for improved environmental management is difficult when there are multiple benefits involved, such as in catchment management (Gilvear et al. 2013), water quality (Zhang et al. 2017), marine areas (Oinonen et al. 2016) and ecosystem services (Wendland et al. 2010;Deal et al. 2012). Economic tools, such as cost-effectiveness analysis, are often required to help identify a least cost set of measures to reach multidimensional environmental improvements in these types of case studies (Pannell et al. 2012).Bundling commonly occurs in environmental protection strategies where different environmental goods, such as carbon, water and biodiversity services, are combined into packages with single payment streams (Wendland et al. 2010;Almeida and Garcia-Sanchez 2016). Benefits of bundling include better integration of ecosystem services and more efficiency in achieving broad environmental goals (Deal et al. 2012). However, analysts wishing to assess effectiveness of water quality programs where different pollutants may be bundled together need to disaggregate bundles of inputs and outputs (Doole et al. 2013) or compare inputs against some aggregation of different outputs (Gilvear et al. 2013;Bloodworth et al. 2015). Assessing multifunctionality across different options is best done with some form of benefit-cost or costeffectiveness analysis, where the costs of interventions in monetary terms can be compared to the benefits achieved (Balana et al. 2011;Pannell et al. 2012;Walsh and Wheeler 2013).
Index approaches have been long used to assess water quality (Horton 1965;Sutadian et al. 2016), with adaption by economists to assess benefits in a consistent manner (e.g. Carson and Mitchell 1993;Johnston et al. 2017) or select more effective management interventions (e.g. Bloodworth et al. 2015). However, indexes vary markedly according to the measure selection, aggregation and weighting steps (Walsh and Wheeler 2013), making it difficult to apply them consistently. While many strategies are framed around integrated catchment management targets, actual interventions tend to be focused on individual pollutants (Balana et al. 2011;Bloodworth et al. 2015). In these cases, bundling benefits together is not appropriate; what is needed is some form of disaggregation to more closely match management inputs with water quality changes.
In this study, we conceptualise and compare two different approaches to analysing cost-effectiveness 1 when varying bundles of benefits are generated. The first approach is to package all of the environmental benefits into a single index so that the cost-effectiveness of each project can then be evaluated by the ratio of project cost to the index of benefits. For example, Johnston et al. (2017) perform a meta-analysis of water quality improvement studies by standardising benefits to a 100-point water quality index. This is equivalent to the use of a benefits index in conservation auctions, where the biodiversity or ecological benefits of programs are summarised into a single score, such as a Biodiversity Benefits Index (e.g. Stoneham et al. 2003;Claassen et al. 2008). This approach is referred to as the index approach to cost-effectiveness.
The second approach is to disaggregate the costs of investments to align with the separate benefits that are outcomes of projects. This requires some mechanism to apportion the program incentives paid to landholders or broader project costs to the separate outputs, such as multiple pollutant reductions. Options to perform this disaggregation include standard regression analysis, data envelopment analysis (DEA) and stochastic frontier analysis (SFA), although all have challenges when there is random variation in the data sets. For convenience, we label this the disaggregation approach to cost-effectiveness.
We compare two approaches to dealing with the bundling issue in costeffectiveness analysis. They are utilised to evaluate mechanisms to improve the quality of water received into the Great Barrier Reef (GBR) in Australia. There has been significant public and private investment into improving water quality in GBR catchments but limited information is available about the cost-effectiveness of these investments (Rolfe and Windle 2016). There are two major aims of the analysis reported in this study: (a) to demonstrate the two alternative approaches to measuring cost-effectiveness when multiple benefits are involved and (b) to compare cost-effectiveness results of the two approaches. At the case study level, our results show that cost-effectiveness could be significantly improved if costs of individual pollutant reductions could be more carefully targeted. Our broader contribution to the literature is to show that disaggregation approaches to analysing issues of multiple pollutants provide more insights than the commonly applied index approaches and that random effects regression models can be applied for this purpose.

Case study
The health of the (GBR), the world's largest coral reef system, is under threat from a number of pressures, including land-based run-off, even though there has been concerted action and investment by the Queensland and Australian Governments (Waterhouse et al. 2017;DEHP 2016). Agricultural land uses are the main source of nitrogen, pesticides and sediments to the GBR, which flow out through the river systems to impact on corals, seagrasses and other ecosystems, particularly during major flood events (DEHP 2016;Kroon et al. 2016;Waterhouse et al. 2017). However, it is difficult to prioritise where costeffective changes can be made in agricultural water quality (Rolfe and Windle 2011; van Grieken et al. 2013;Star et al. 2015;Alluvium 2016;Beher et al. 2016;Pannell and Gibson 2016).
More than 80 per cent of land in the catchment area supports some form of agriculture. Cattle grazing is the most extensive land use, occurring in more than 74 per cent of the catchment. Although Intensive agricultural uses such as cropping (mostly sugarcane) occur in only five per cent of the catchment, they are located in the lower coastal floodplain and have a more direct impact on the GBR (GBRMPA 2014). Declining marine water quality influenced by land-based run-off is recognised as one of the most significant threats to the long-term health and resilience of the GBR (DEHP 2016). The 2017 Scientific Consensus Statement identified the largest contributors to elevated pollutant levels and greatest risks come from nutrients, sediments and pesticides leaving agricultural land (Waterhouse et al. 2017). Nitrogen discharge (predominantly from excess fertiliser applications in the sugar industry) is associated with crown-of-thorns starfish outbreaks (DEHP 2016). Sediment discharge (mainly from grazing land) increases turbidity and reduces the light available to seagrass ecosystems and inshore coral reefs Fabricius et al. 2014). Pesticides (largely from intensive cropping including sugarcane and horticulture) pose a risk to freshwater and some inshore and coastal habitats (Waterhouse et al. 2017).
The Australian and Queensland Governments have developed a number of plans to improve water quality entering the GBR lagoon, as well as specific targets to improve water quality. The latest of these is the Reef 2050 Long-Term Sustainability Plan (Australian Government 2015), with the end-ofcatchment targets (from a 2009 baseline) set for priority areas at: • Nitrogen: at least 50 per cent reduction by 2018, and 80 per cent reduction by 2025; • Sediment: at least 20 per cent reduction by 2018, and 50 per cent reduction by 2025; and • Pesticides: at least 60 per cent reduction by 2018.
There has been significant public investment to support these plans and to help achieve the water quality targets. The Water Science Taskforce (DEHP 2016) identified that the Australian Government committed $200M from 2009 to 2013 and $300M from 2014 to 2019, while the Queensland Government committed $175M from 2009 to 2013, and again from 2014 to 2018. Engagement with and funding to landholders mostly occur through the six Natural Resource Management (NRM) groups that cover the major regional catchments for the GBR (Figure 1).
In the 2014-15 year, a total of $29.8M was allocated to the NRM bodies ($23.2M from the Australian Government and $6.6M from the Queensland Government) (DEHP 2016). Most direct grant programs involving onground works have required at least matching funds from landholders, generating significant private funds and in-kind investment. However, the rate of change achieved with these programs has been slow (A&QG 2016; Kroon et al. 2016), and there is very limited data available on costeffectiveness of grant programs (Beher et al. 2016). Among the challenges of evaluating cost-effectiveness is the involvement of multiple pollutants, where various farm management changes may have varying impacts on sediment, nutrients and pesticides.
This study analysed the cost-effectiveness of grant funding to sugarcane growers in recent years (2013-14 and 2014-15), where changes in farming practices might generate individual or joint reductions in different pollutants. Growers undertook a wide range of initiatives to improve water quality though actions such as more precision application of fertilisers and pesticides, laser levelling of farm areas and establishing water recycle pits. The projects were funded through the Reef Rescue program, with co-investment from growers. Typically, projects were awarded on a fixed-grant basis, where uniform benefits per action were assumed, without taking into account biophysical differences between farms or landscape and climate factors such as distance to end-of-catchment and variations in rainfall and streamflow. The purpose of this study was to analyse some project data comparing costs with modelled pollutant changes and test different approaches to dealing with the multiple pollutants involved. Ideally, an economic analysis should focus on minimising the costs of achieving desired outcomes, such as improved reef health, and matching these to the benefits involved. However, this is difficult for two key reasons. First, the linkages between pollutant reductions and reef health are not straightforward, as the ecological and climatic processes are complex and stochastic (Waterhouse et al. 2017). Second, while some benefit estimates for protecting the GBR are available (e.g. Rolfe and Windle 2012), these are only available at a broad level and not matched to key pollutants. This explains why relevant government policies have tended to focus on outputs, such as reducing pollutant loads at end-of-catchment, with an implicit assumption that these goals will lead to improved outcomes in terms of improved reef health.

Methods
A cost-effectiveness measure involving multiple pollutants relevant to this study can be represented as: where CE = cost-effectiveness, C = project cost, B = project benefits, subscripts a, b and c refer to farm, grower and catchment, respectively, while subscripts SS, DIN and PSII refer to the three pollutant types, suspended sediment (SS), dissolved inorganic nitrogen (DIN) and pesticides (PSII).
The key difficulty is to find a systematic way of summarising the multiple benefits into a single measure. Other approaches to measuring costeffectiveness of water quality projects have been to select only one pollutant as being representative of the environmental benefits or use some form of an index approach to summarise the water quality improvements or reductions in risks (Bloodworth et al. 2015). However, index approaches usually do not represent quantitative changes very well, as most are summaries of a range of different features that are assessed in nonmetric ways (e.g. dummy codes to represent presence/absence or whether a guideline is above or below a threshold value) (Sutadian et al. 2016). This makes most water quality indexes not very suitable for inclusion in cost-effectiveness analysis (Walsh and Wheeler 2013;Bloodworth et al. 2015).
In this study, two different approaches were applied to compare the costs and benefits of these projects: an index approach and a disaggregation approach. The methods were selected to be relevant for cost-effectiveness analysis, avoiding the limitations of other water quality assessments. Each approach requires data to be transformed in some way to allow the analysis to proceed. In summary, the index approach requires the transformation on the benefits side so that the effects of multiple pollutant changes can be combined into a measure for each project which can then be compared to costs. By comparison, the disaggregation approach requires the transformation on the costs side where costs for each project have to be apportioned across the different pollutants involved.

The index approach
The index approach is to package all of the environmental benefits (reductions in pollutant loads) into a single index so that the cost-effectiveness of each project can be evaluated by the ratio of project cost to the index of benefits. Some indexes can be classified as aggregate approaches used to measure environmental performance at a national or regional level. For example, Fare et al. (2004) calculate a formal index of environmental performance at a country level using data envelope analysis, Almeida and Garcia-Sanchez (2016) compare the Composite Index of Environmental Performance and the Environmental Performance Index, and Azad and Ancev (2010) use ecological indices to assess environmental performance in irrigated agriculture. Other indexes can be classified as case study approaches, where the aim is to represent environmental condition or trend for a particular group of assets. For example, indexes can be used to provide a measure of biodiversity in a conservation auction (Stoneham et al. 2003), improvements to the environment in a water quality auction  or risks of pollutant mobilisation at field or catchment scales (Bloodworth et al. 2015).
There is no standard method for constructing a water quality index, although most include measures for physical, chemical and biological properties (Sutadian et al. 2016;Flint et al. 2017). Typically measure in water quality indices are evaluated against some benchmark or guideline value to report on the quality within a system, which is different to the focus of this case study on the discharge of pollutants to the GBR. To create an index for this study, the Government pollutant reduction targets for the GBR catchments were used as a benchmark so that the benefits of different pollutant reductions could be combined in a common unit. Targets are expressed as the proportional reductions that are desired from a 2009 baseline, with targets for 2018 set across GBR catchments at 20, 50 and 60 per cent reductions for SS, DIN and PSII, respectively, with more detailed target reductions set for each catchment (DEHP 2016).
Anthropogenic pollutant levels for each catchment in the GBR have been identified from Kroon et al. (2012), except for the Barron River which was taken from the Wet Tropics Region Water Quality Improvement Plan 2 . To calculate the index for each individual project, a two-step process was followed. First, the pollutant reductions for each project were divided by the relevant catchment targets. For example, a project in the Burdekin River saved 435 kg of dissolved inorganic nitrogen (DIN) against the DIN target for the catchment of 326,000 kg, so the project has contributed 0.1 per cent to the pollutant target. A larger project would have contributed more to the target and received a higher weighting. Second, the proportional reductions were summed across the three pollutants. For example, a reduction that delivered 0.5, 0.2 and 0 per cent of the catchment target reductions across the three pollutants would receive an index score of 0.007. The index calculations were also repeated with GBR-scale targets rather than individual catchment targets.
The index approach to estimating cost-effectiveness essentially involves replacing the denominator in Equation 1 with an index summing the change in environmental risks from the combined influence of reduced pollutants (environmental benefit): The index should be capable of assessing single pollutant changes as well as different combinations of multiple pollutants. Note that it is not appropriate to simply combine the quantities of pollutant change into an index. Instead what is required is to identify the reduced impacts on the environment for each pollutant reduction and to then aggregate the benefits. The challenges of estimating appropriate damage and benefit functions that are consistent across pollutants are substantial, but increasing scientific knowledge and modelling are improving the opportunities for this approach.
The use of catchment targets to convert a change in pollutant load into an index number that can then be summed across pollutants is a novel way of assessing multiple pollutants. Underpinning this approach is implicit assumptions that the catchment targets for each pollutant represent current scientific knowledge about the need for reductions and that the targets reflect the relative importance of the different pollutants. Note that pollutant reduction targets are available for each catchment in the GBR, as well as for the GBR as a whole, allowing two separate ways of calculating an index.

The disaggregation approach
The second approach is to disaggregate the costs of investments to align with the separate benefits that are outcomes of projects. This requires some mechanism to apportion the program incentives paid to landholders or broader project costs to the separate outputs, such as multiple pollutant reductions. The most common approach has been to use productivity or efficiency analysis to assess the effect of environmental inputs and outputs alongside of economic data, following Chung et al. (1997) among others. The approaches include nonparametric approaches such as DEA and parametric approaches such as SFA. Ramli and Munisamy (2013) provide a review of the various approaches to including undesirable outputs in a DEA framework, and Zhou et al. (2008) provides a survey of applications in energy and environmental studies, while examples of applications to water pollutants include Chung et al. (1997) and Hailu and Veeman (2001).
The approach taken in this study was to analyse the relationship between the costs of a management change (public funds) against the outputs (pollutant reductions) and other explanatory factors. The disaggregation approach involves separating out the costs of pollutant change by pollutant: The random effects regression model has been chosen for the analysis because of its ability to allow for random shocks and deal with 'nonparticipation' issues. Applying a random effects model helped to control for systematic differences when combinations of one, two or three pollutant reductions were achieved. In contrast to normal production data where variables are measured for each production unit in each time period, the pollutants in this study only aligned intermittently with each production unit. This made it problematic to apply a DEA. For example, of 288 valid projects used in the analysis: • 44 projects only made a change in sediment • 72 projects only made a change in DIN • 74 projects only made a change in PSII • 21 projects made a change in both sediment and DIN • 3 projects made a change in both sediment and PSII • 47 projects made a change in both DIN and PSII • 27 projects made reductions in sediment, DIN and PSII

Case study data
The NRM groups are required to submit details of awarded projects to Government each year, and estimates of pollutant reductions are subsequently made through the Paddock to Reef program. These estimates form the basis for the annual GBR report card (A&QG 2016). The Paddock to Reef program was established in 2009 to measure and report on progress towards the goals and targets of the Reef Water Quality Protection Plan 3 through annual Report Cards. The program includes a significant modelling component to assess pollutant movement from paddock level through to catchments and the reef lagoon. Project level information was subsequently been made available to the authors to analyse their cost-effectiveness. Information was provided for 530 individual farm-level sugarcane projects that have been funded from the 2013-14 and 2014-15 years. No sugarcane is grown in the Fitzroy catchment or Cape York Peninsula so there are only projects from four of the six NRM groups. Projects include: • Wet Tropics (238 projects); • Burdekin (168 projects); • Mackay Whitsunday (Reef Catchments) (74 projects); and • Burnett Mary Regional Group (50 projects).
Modelling had been undertaken through the Paddock to Reef program to estimate the net changes (end-of-farm) in pollutants for each project. Pollutants included SS, DIN and PSII. The different pollutant pathways, including through direct run-off or drainage through ground water, were taken into account. However, the time lags to achieve the pollutant reductions have not been taken into account. There is an underlying assumption that the project management actions are adopted and continue, as well as being effective in achieving the modelled reductions. It was estimated that the 530 projects resulted in 37,571 tonnes of sediment reduction, 242,150 kg of nitrogen reduction and 1,714 kg of pesticide reduction.
The public investment into the sugarcane projects summed to $6,879,481 over the two years, with a further $222,000 allocated for extension activities. These investments were augmented by $8,674,541 in direct funding by landholders, and a further in-kind investment of $3,101,318 by landholders. The costs of achieving change have been taken directly from the public component of project costs (the funding that landholders received), as well as the private investments from landholders (Table 1). However, the analysis has concentrated on a comparison of the public funds invested to the public benefits of the water quality improvements (reduced pollutant loads) that are generated. This essentially treats the co-contributions of landholders as private investments to increase farm productivity, rather than being used to generate public benefits.

Data for analysis
Not all projects were included in the cost-effectiveness analysis. Six projects receiving $55,621 in public funds were estimated to produce negative benefits (increased pollution emissions), and a further 187 projects receiving $2,110,156 in public funds were not modelled to produce a system change that would directly reduce pollutants. Perhaps these projects will generate longer term changes through extension and capacity-building mechanisms that could not be captured in the modelling approach. Some of these projects (22) were for extension projects where benefits were difficult to quantify. This means that 36 per cent of projects consuming 32 per cent of public funds were not modelled to generate any direct pollution reduction benefits, contrary to the purpose of the programs.
A total of 337 projects were identified as contributing to water quality improvements. Within this group, 7.7 per cent of projects made improvements across the three pollutants, 21.7 per cent made improvements across two of the three pollutants, and the remainder made improvements in only one pollutant. A little more than half of projects made DIN reductions (57.6 per cent) and PSII reductions (51.9 per cent), while 27.6 per cent of projects involved sediment reductions. There were limited correlations between the pollutant reductions across projects, with 18.9 per cent correlation between sediment and DIN, 5.6 per cent between DIN and pesticide projects, and 19.5 per cent between sediment and pesticide projects.

The index approach
In the index approach, the percentage that each pollutant made towards the target reduction was identified and then summed if more than one pollutant was involved. A total of 291 projects were identified that involved public funding and generated positive reductions in pollutants. The cost-effectiveness of projects is shown in Figure 2, together with a summary of average costs by quartiles. Each point on the graph represents a project, ordered from most to least cost-effective from left to right. Only public costs were included in the cost estimates. The analysis shows that average costs (for each per cent of catchment target met) increase rapidly by quartile and that the bulk of projects makes very little contribution to meeting the targets (noting that it was not possible to model the indirect effects of projects that generated longer term system changes). The first 50 per cent of projects generated 96 per cent of estimated benefits and cost $6.71 per 0.0001 per cent of catchment improvement across pollutants, whereas the second 50 per cent of projects generated 4 per cent of benefits at an average cost of $149 per 0.0001 per cent of catchment improvement. Cost-effectiveness could have been significantly improved with negligible impact on total benefits if the most expensive projects could have been excluded.
The analysis was also performed against the pollution reductions targets for the whole GBR, rather than by individual catchments. The correlation between project scores under the two prioritisation approaches is only 0.52; however, this was significant at the 1 per cent level. A paired samples t-test identified a significant difference between the scores calculated at the GBR and catchment level (t = 3.130 at 466 d.o.f., Sig. = 0.002). This confirms that there is potential for large variations in project appraisal depending on whether the pollutant reductions are assessed against the catchment targets or Figure 2 Project cost-effectiveness by cumulative contributions to pollutant targets broader GBR targets. However, the prioritisation order for projects was similar under either approach (Figure 3), with a correlation coefficient of 0.903.

The disaggregation approach
Estimating cost-effectiveness with the disaggregation approach involved three stages. First, the relative contribution of each pollutant to the project costs for each landholder was revealed through a random effects model, which also provided other insights into factors driving cost-effectiveness. Second, the relative contribution factors estimated from the random effects model were used to disaggregate the cost allocations across pollutants, while in the third step, patterns of cost-effectiveness were analysed for each pollutant in turn.
In the random effects model, public costs of each project were treated as the dependent variable, and the pollutant reductions as the independent variables (all converted to natural logs). Projects with zero public funding and zero or negative pollutant reductions were removed from the data set, together with some outliers, leaving 288 observations in the analysis (Table 2). Dummy variables for the regional areas (Wet Tropics/Burdekin is the base) and dummies for the number of pollutants involved (two pollutants is the base) were included as extra variables. Comparing the absolute values of the coefficients for the three pollutants implies that pesticide reductions account for approximately 39.1 per cent of total costs, followed by nitrogen reductions (37.3 per cent) and then sediment reductions (23.6 per cent). This provides a basis for allocating costs across projects generating multiple pollutant benefits. In the second stage of the analysis, the costs of multipollutant projects have been apportioned for each project between the pollutants involved. This has been performed using the ratios derived from the random effects regression model in Table 2 of 23.6, 37.3 and 39.1 for sediment, DIN and pesticides, respectively. For example, the costs for a project costing $1,000 across all three pollutants would be apportioned as $236 for sediment reductions, $373 for DIN reductions and $391 for pesticide reductions. For the third stage, the cost-effectiveness of each project has been calculated by finding the ratio of the pollutant reduction to the allocated cost of that reduction. The costeffectiveness estimates have then been sorted into ascending order, and the results graphed separately for each pollutant (below).
Results show substantial variation (heterogeneity) in cost estimates. This means that while some projects have been very cost-effective, many have not. At least one-quarter of Reef Rescue grants have not generated any significant pollutant reductions, while a further quarter of funding has generated very limited benefits.
For sediment (Figures 4 and 5), the best 50 per cent of projects cost $9.00/ tonne, while the worst 50 per cent of projects cost $177/tonne. Put another way, the first 25 per cent of projects (by cost-effectiveness) achieved 72 per cent of benefits, the second 25 per cent of projects achieved an additional 18 per cent of benefits, whereas the third and fourth quartiles of projects achieved only 9 and 1 per cent, respectively.
For DIN, the best 50 per cent of projects cost $2.92/kg, while the worst 50 per cent of projects cost $87/kg. Put another way, the first 25 per cent of projects (by cost-effectiveness) achieved 86 per cent of benefits, the second 25 per cent of projects achieved an additional 10 per cent of benefits, whereas the third and fourth quartiles of projects achieved only 4 and 1 per cent respectively (Figures 6 and 7).
For pesticides, the best 50 per cent of projects cost $365/kg, while the worst 50 per cent of projects cost $6,120/kg. Put another way, the first 25 per cent of Note: ***P < 0.01, **P < 0.05, *P < 0.1 projects (by cost-effectiveness) achieved 70 per cent of benefits, the second 25 per cent of projects achieved an additional 19 per cent of benefits, whereas the third and fourth quartiles of projects achieved only 9 and 1 per cent, respectively (Figures 8 and 9).

Discussion and recommendations
Applying economic frameworks to environmental management issues is challenging when benefits and costs are bundled, as the relationships between inputs and outputs are difficult to identify. Because of these limitations, most applications of cost-effectiveness analysis to environmental issues, such as water quality improvements, have simplified the analysis by focusing on a single dimension (e.g. Balana et al. 2011) or an index to represent overall improvements or risks of damage (e.g. Bloodworth et al. 2015).
In this paper, we demonstrate how the costs of bundled pollutant reductions can be disaggregated and compare that to the more standard approach of combining pollutant reductions into a single index measure. The disaggregation approach is more consistent with economic analysis than an index approach, providing more detailed information for the analyst. This approach also avoids or minimises the consolidation of different attributes and hidden weightings that underpin water quality indicators (Walsh and Wheeler 2013). Given that index approaches are widely applied in water quality assessment (Sutadian et al. 2016), the contribution of this research is  Figure 4 Cost-effectiveness of sediment reduction to demonstrate an alternative approach that is much more conducive to costeffectiveness analysis. We demonstrated these approaches in an important case study application regarding the assessment of farm management practices to improve water quality into the GBR. Two methods were applied to evaluate the relationship between program investments and multiple water quality benefits: an index    Figure 6 Cost-effectiveness of DIN reduction method to compare costs against a summary index of pollutant reductions; and a disaggregation approach which applied a random effects model to identify how investments have been apportioned over different pollutants. The index approach that was used focused on water quality targets rather than more standard water quality assessments, making it more suitable for  cost-effectiveness analysis. Even so, the disaggregation approach provided much more useful results for policymakers by generating separate estimates of costs for each pollutant. The analysis and results help to demonstrate the strengths and weaknesses of both approaches. The index approach is simpler to analyse, but relies on an index to be constructed. Many environmental indexes rely on very arbitrary ways of combining and summarising different elements, leading to situations where benefits can vary widely, depending on the approach taken (Walsh and Wheeler 2013). By comparison, the disaggregation approach avoids the underlying assumptions of combining elements, but involves the challenges of identifying relationships between costs and relevant elements, particularly where data are limited.
Our preferred approach for overall analysis was the disaggregation approach because it reveals more detail about how funds have been invested and allows benchmark costs to be estimated for each pollutant. For example, the regression analysis identified a negative coefficient for the pesticides component, implying that higher cost projects generated smaller reductions than lower cost projects. This insight would not have been available from an index approach.
Results of the analysis demonstrate large variations in cost-effectiveness of funded projects in the program. For example, the index approach revealed that the average costs of the fourth quartile of cost-effective projects were 197 times more expensive than the average costs of the first quartile of projects ($428.49 per unit compared to $2.18 per unit). The corresponding ratios for the pollutants estimated with the disaggregation approach were 152 times for sediment, 173 times for DIN and 106 times for pesticides. If the private costs of landholders were also included as part of the investment, these costs would  Cumulative PSII reduction Cumulative project cost Projects in ascending order by cost-effectiveness PSII projects in order of cost-effectiveness Cumulative cost Cumulative PSII reduction Figure 9 Cumulative sediment reductions and costs in order of cost-effectiveness be nearly three times higher. These variations are likely to be driven by multiple causes, including the factors identified by Pannell and Roberts (2010) explaining why public programs to deliver environmental improvements tend to be inefficient. There were some limitations in the cost-effectiveness analysis reported in this paper that should be noted. Some projects that have not generated quantifiable pollution reductions may still have generated other benefits, such as improved engagement with landholders, better encouragement to increase adoption and lower transaction costs for future projects. Other projects are focused on providing pathways to pollution reduction (such as training and extension programs). For these projects, more detail is required about the benefits being generated.
The results of this study suggest that much larger reductions in pollutants can be gained by better focusing on cost-effectiveness. It is particularly worth noting that almost one-third of funded projects were not modelled by the Paddock to Reef program to achieve any pollutant reductions. Beher et al. (2016) have previously suggested that efficiencies in these types of programs could be increased by up to four times. We recommend three pathways to progress towards this goal. First, take account of cost-effectiveness when allocating funds. Second, make more use of other mechanisms, such as reverse tenders that better provide farmers with incentives to propose least cost solutions. Third, identify the extent to which engagement and other preparatory projects trickle through to on-ground actions so that their costeffectiveness can be evaluated.