Conservation tenders: linking theory and experiments for policy assessment

Authors

  • Steven Schilizzi,

  • Uwe Latacz-Lohmann

    Search for more papers by this author
    • Steven Schilizzi is at the School of Agricultural and Resource Economics, The University of Western Australia, Perth, WA 6009, Australia. Uwe Latacz-Lohmann (email: ulatacz@agric-econ.uni-kiel.de) is at the Department of Agricultural Economics, Christian-Albrechts University, Kiel, Germany.


  • The authors extend their gratitude to the University of Western Australia and to the Christian-Albrechts University of Kiel for having helped with the logistics of this study. Much was also made possible by time available for reciprocal sabbaticals at the CAU in 2010 and at the UWA during 2011.

Abstract

Auction theory has mostly focussed on target-constrained auctions and is less well developed for budget-constrained tenders, which are the norm in environmental policy. This study assesses a theoretical model developed for budget-constrained tenders in its capacity to predict tendering performance under information deficiencies typical of field applications. If complemented by laboratory experiments, the model is able to make the correct policy recommendation when comparing the tender to an equivalent fixed-price scheme, even with poor predictive accuracy. This holds even if the policymaker has only limited information on the model’s key input variables.

1. Introduction and background

Buying environmental services from private landholders using tendering mechanisms usually involves budget-constrained, procurement-type auctions. In a budget-constrained conservation auction or tender, the program’s budget is predetermined; the risk lies with the number of participants or the area that might fail to come under contract, that is, with the policy’s outcome. The widespread use of the budget-constrained (BC) tender format in conservation policy poses a problem to the extent that auction theory has been well developed, since Vickrey’s 1962 paper (less well known than his much-cited 1961 paper), for target-constrained (TC) auctions, but much less so for BC auctions (Müller and Weikard 2002). As a result, in the field of environmental policy, there is a gap between theory and practice. A better theory would allow agencies to improve tender design and perhaps decide whether such a mechanism is worth going ahead with or not, given existing alternatives.

This study investigates the predictive capacity of a model developed for BC tenders applied to land conservation programs. By predictive capacity, we mean both the model’s capacity to predict bids and, more importantly, to predict the tender’s performance. This model was first proposed by Latacz-Lohmann and Van der Hamsvoort (1997) and further refined in 1998, where policy implementation was investigated. To the best of our knowledge, this is to date the only extension of auction theory which captures the particular features of conservation tenders. However, it does not conform to the standard assumptions of auction theory regarding optimal bid formulation, as in a BC tender, in contrast to the TC, bidders do not know in advance the number of winners. Müller and Weikard (2002) show that this results in multiple Nash equilibria with no dominant solution for choosing an optimal bid. Latacz-Lohmann and Van der Hamsvoort (1997) solve this problem by introducing an exogenous parameter, the bidders’ expectation of the highest acceptable bid (i.e., the bid cap), knowing the budget constraint and the number of bidders. Bidders then use this best guess of theirs to form their optimal bids. This yields a much simpler model than the more standard TC model, but at a cost, in that bidders’ expectations of the bid cap cannot be observed, so that the model cannot be used by policymakers to make ex ante assessments of the value of running a tender for conservation services. The model by Latacz-Lohmann and Van der Hamsvoort (1997) thus only predict bids but not how they are formed.

The purpose of this study is to investigate the validity and credibility of the BC model for assessing the economic performance of the BC tendering mechanism, using several performance criteria. Assessing the performance of the mechanism itself was investigated in Schilizzi and Latacz-Lohmann (2007) who compared, with repetition, the performance of the BC and TC tenders relative to an equivalent fixed-price scheme. The focus in the present study is to examine whether the BC model is capable of predicting the performance of the tendering mechanism using bids predicted from the model rather than observed bids. To the extent that government agencies have, to date, almost exclusively used the BC format in environmental policy, it seems important to test any model that might serve to recommend the use of this policy instrument. In Australia, for example, the Victoria BushTender and EcoTender conservation programs were directly inspired by the BC model (Stoneham et al. 2003).

We investigate the validity and credibility of the BC model with the help of controlled laboratory experiments. We first study how well the model can predict experimental bids. We then examine the model’s capacity to predict the economic performance of a BC tender with the information set available to the experimenter. We then repeat this analysis but with an information set typically available to the policymaker. The experiments allow us to acquire data on bidders’ expectations of the maximum acceptable bid (the bid cap) – an essential input to predicting optimal bids. The theoretical gap in the BC model’s not specifying how bidders form their bid cap expectations can thus be filled in. The first two steps represent the experimenter’s point of view: how does the BC model predict bids and policy performance with full knowledge of the model’s input variables (costs and bid caps)? The third step mimics the situation of a policymaker who has only limited knowledge of costs and none of bid caps. The role of the experiments is twofold: besides filling in the gap left open by the non-specification of how bid cap expectations are formed, they allow us to separately evaluate the model’s limitations because of poor information inputs and those that remain even under perfect experimental information.1

The remainder of the study is organised as follows. Section two presents the BC tendering model. Section three describes its experimental implementation. Section four links the theoretical model and the experimental results. Sections five and six provide and discuss the results. Section seven concludes.

2. The budget-constrained bidding model

The sealed-bid discriminatory price budget-constrained (BC) model examined in this study was first proposed by Latacz-Lohmann and Van der Hamsvoort (1997). This is the first bidding model that attempts to capture the particular features of conservation tenders. They considered landholders to hold private information about their costs of participating in the government’s conservation program. These costs arise when management prescriptions divert farmers’ land management practices away from their current plan, assumed to be the most profitable one. The government’s problem, to attract farmers into the scheme, is to compensate them for the lost profits without knowing their magnitude. Auctions have the property of revealing at least part of this information. In order for the landholder to participate in the scheme, the payment he or she receives must be at least equal to his or her cost of participation.

Latacz-Lohmann and Van der Hamsvoort (1997) first assume that landholders’ bidding strategies are predicated on the belief that the conservation agency (the procurer) will decide on a maximum acceptable bid, or payment level, β. This is a common practice when the agency is subjected to a constrained budget. In actual fact, this maximum bid β is determined ex post, after all bids have been received, as the last (highest) bid accepted within the available budget.2 In other words, no individual bids above β will be accepted. β represents an implicit reserve price per unit of environmental service, unknown to bidders (and also unknown to the procurer until all bids have been received). For simplicity, we shall work with a unique environmental good, although this is not a restriction. This external parameter β represents a deviation from standard target-constrained auction theory, where optimal bids are determined endogenously as a function of the number of bidders, the distribution of bidders’ costs (assumed common knowledge) and the target to be achieved. In the BC auction, this target – the number of winners or hectares contracted – is unknown. A landholder will tender a bid b if the expected utility in case of participation exceeds his or her reservation utility.

The second assumption in the model by Latacz-Lohmann and Van der Hamsvoort (1997) is that bidders, not knowing the value of the bid cap β, will form expectations about it, which can be characterised by the density function f(b) and by the distribution function F(b). The probability that a bid is accepted can then be expressed as

image(1)

where P is probability and inline image represents the upper limit of the bidder’s expectations about the bid cap, or the maximum estimate of the highest acceptable bid. Any uncertainty about the quantity of services offered, the size of the budget or the costs of rival bidders is reflected in bid cap expectations, which the model takes account of through the formation of beliefs about the value of β.

The essence of the bidding problem is to balance out net payoffs and probability of acceptance. This means determining the optimal bid that maximises the expected utility over and above the reservation utility.

Further assumptions are that there are no transaction costs in bid preparation and implementation, that payment is only a function of the bid (discriminatory price auction) and that bidders are risk neutral.3 A risk-neutral bidder simply maximises expected payoff. The optimal bid, b*, derived by Latacz-Lohmann and Van der Hamsvoort (1997) is given by Equation (2), where c represents the costs of participation:

image(2)

Latacz-Lohmann and Van der Hamsvoort (1997) further assume that bidders’ individual expectations about the bid cap β, unknown to them, are uniformly distributed in the range [β, inline image], where the lower and upper bounds represent the bidder’s minimum and maximum expectation of the bid cap. A bidder’s expectations are that any bid equal to or below β has a probability of one of being accepted, and any bid equal to or above inline image has a probability of zero of getting accepted. Then, the expression for the optimal bid becomes (Latacz-Lohmann and Van der Hamsvoort 1997):

image((3'))

This is true for each of the i bidders, so that expression (3′) also reads as:

image((3"))

Expressions (3′) and (3″) show that the optimal bidding strategy of a risk-neutral bidder increases linearly with both the bidder’s costs ci and his or her expectations about the bid cap, characterised by βi and inline image. Bids thus convey information about costs, which are private information unknown to the procurer; they thereby reduce the information asymmetry, but not completely: the auction’s cost revelation property is blurred by the fact that bids also reflect bidders’ beliefs about the bid cap. This creates room for bidders to bid above their true costs and thereby to secure for themselves an information rent (Latacz-Lohmann and Schilizzi 2006, pp. 21–23).

Budget-constrained (BC) tenders differ from the target-constrained (TC) format in that the predetermination of the budget and of the outcome is reversed. As discussed by Müller and Weikard (2002), TC tenders allow endogenous expectations to form and optimal bids to be formulated without the need for exogenous bid caps. Thus, while the TC model is a Nash-equilibrium model, the BC model is a best-response model. This is because by knowing the target, bidders know the number of winners or contracts to be allocated, thereby yielding fewer degrees of freedom than the BC auction. Not surprisingly, the TC auctions were modelled much earlier, by Vickrey (1961). Their application to multi-unit sealed-bid procurement tenders, relevant for government conservation schemes, was only modelled in 2005 by Hailu et al., who built on Harris and Raviv (1981) generalisation of Vickrey’s approach. In a discriminative (first) price setting, both BC and TC models predict that overbidding is an optimal strategy.4

3. Experimental implementation

The purpose of the experiments described below was to assess the capacity of the BC model to predict the tender’s economic performance. One wishes to know whether it is a credible tool for informing budget-constrained tendering design for allocating conservation contracts. We focus first on the difference between the observed experimental bids and those calculated based on Equation (3′′); secondly, we evaluate the economic performance of the tendering mechanism using bids computed with the BC model as opposed to using experimental bids. This should shed some light on whether experimental results can be used for guiding the use of BC tendering mechanisms.

Prior to holding the experiment, we surveyed our experimental subjects regarding their attitude towards risk. Bidders’ risk attitudes were measured using a certainty equivalent method, whereby they were asked to state the minimum price they would accept from selling a lottery ticket that had been given to them (see Cox and Harrison 2008).5 This measure was also hypothesised to explain possible differences in bid shading, whereby more risk-averse bidders would shade their bids less than the less risk-averse. As it turned out, risk attitudes, as measured, did have some impact in the expected direction.

Also prior to the experiment, we asked participants to state on a 5-point Likert scale their attitudes towards land conservation and agricultural productivity. Cason and Raymond (2011) have compared subjects’ experimental behaviour when the situation is ‘framed’ in some policy context and when it is not, showing that subjects do respond to such framing effects. However, in our study, there was no correlation between stated attitudes towards environmental conservation and the experimental bids, suggesting the contextual effect was minimal in this case.

Experiments were first carried out at the University of Kiel, Germany, then at the University of Western Australia in Perth. The Perth experiment replicated the Kiel experiment, to check for the robustness of results. The Kiel experiment was carried out with first-year students in agricultural economics. The tendering setup referred to reductions in nitrogen fertiliser (N) on a wheat crop, to meet EU regulations regarding limits to nitrate concentration in groundwater (50 mg/L). This is a serious concern in rural areas of northern Germany, and one which students in Kiel would be aware of and sensitive to. Participants were offered would-be contracts for committing themselves to reduce applications of nitrogen fertiliser from their currently most profitable level down to a predefined constrained level, equal to 80 kg per hectare. Each participant was given a different production function for nitrogen fertiliser in wheat production and thus faced a different cost resulting from the adoption of the nitrogen reduction program. Participation costs, labelled in Experimental Currency Units (ECU), were drawn randomly from a uniform distribution between 5 (the lowest-cost bidder) and 264 (the highest-cost bidder). Bidders knew their own costs but not those of rival bidders (see Appendix I). Participants were told that not all of them would be able to win contracts and that they were therefore competing against each other within a limited program budget. To keep things simple, each participant could put up just one land unit of wheat, the same area for all participants. They were told that if they won a contract, they would be paid the difference between their bid and their cost.

Each experiment involved three bidding rounds to investigate the performance of the auction with repetition (Schilizzi and Latacz-Lohmann 2007). In the present study, the complete data set of three repetitions is used only for the analysis of bid cap expectations in Figure 1 and Equations (4) and (5) below. The remainder of the analysis in this study is based on the data from only the first bidding round. The data from the second and third repetition are not used as they had been generated for a purpose different from the one focussed on in this study. As auctions are very sensitive to information structure, it was important to control for this aspect. Bidders were informed of the available budget, given this is a common practice in such contexts: see for example, the water buyback auctions in the state of Georgia (US), the Northeast US Groundfish Fishery Buyout Program and the Australian BushTender and EcoTender biodiversity auctions.6 The cost range (5–264 ECU) was not given, but bidders were told that their costs were randomly drawn from a uniform distribution. To approximate typical levels of information held by landowners, each bidder only knew his or her own cost and in which cost quartile they belonged: lower quarter, second quarter, third quarter and upper quarter (see Q values in Appendix II). No information regarding other bidders was given to participants. In particular, no information about the BC model or even its existence was mentioned. There were 44 bidders in the Kiel experiment and 27 in Perth.

Figure 1.

 Influence of bidder cost information on the distribution of bid cap expectations. The Perth data showed a similar pattern except that Q2 and Q3 curves (rather than Q1 and Q2) overlap.

The budget constraint announced (in ECU) was clearly distinguished from the actual payments made at the end of the session (in $ or €). Payments in hard currency would be proportional to gains in ECU terms and their gains were calculated as own bid minus participation cost. Bidders were asked two pieces of numerical information, their maximum estimate of the ‘highest acceptable bid’ (inline image) and their bid (bi). We made it clear to participants that we wanted them to give us their most optimistic estimate of what the cut-off bid might be. We did not ask for the lower bound βi as initial trial sessions revealed that asking both upper and lower bounds confused many participants. However, simulations later carried out with the experimental data showed that such lower bounds would not be binding; rather, the cost constraint, b*>c, turned out to be binding for some bidders. The implication for this study of not having data on both inline image and βi simply means that the validity of the BC model is probably underestimated. With knowledge of both inline image and βi, its capacity to predict bids and tender performance would most likely be enhanced.

The Perth experiment was identical to the Kiel experiment. Participants were mostly first-year students in Kiel and second-year students in Perth, with a few third and fourth years as well as a handful of postgraduates – all in the area of agriculture or natural resource management. To reflect the different number of participants, the budget constraint was modified proportionately, so as to result in the same competition intensity (ratio of budget to bidders) in both replicates: 3900 ECU in Kiel and 2300 ECU in Perth. To maintain high relevance to local conditions in Perth, the motivating context was nutrients running off into the local river, rather than nitrogen into the local aquifer, as in Kiel.

4. Linking theory and experiment for policy

4.1. Modelling the formation of bid cap expectations to fill in a theoretical gap

In policy applications, data on bid cap expectations (the inline image) are not available. The BC model cannot therefore be directly used for guiding policy, because computing (optimal) bids requires knowledge of the inline image. Two approaches are then available. One was chosen by Latacz-Lohmann and Van der Hamsvoort in their 1997 paper: assume the inline image are somehow distributed around a single average cost estimate. The other approach is to implement the model experimentally and use the experimental data on bidders’ costs (ci) and bidders’ stated expectations inline image to derive an empirical relationship between the two. One can then use this relationship to compute optimal bids and use the BC model to assess the tender’s expected performance. The question then is, do the inline image depend on bidders’ cost information? This information is twofold, the cost quartile7 to which they belong (cQ) and their own private cost (ci).

Figure 1 reveals that the individual distribution of the inline image does depend on knowledge of one’s cost quartile. On average, high-cost bidders expect the maximum bid cap to be higher than low-cost bidders: thus, the Kiel data show the βQ increase with cost quartiles (cQ) as 157, 162, 213 and 262. Secondly, across bidders, the inline image approximate a normal distribution within each cost quartile. Note that this is totally independent of the BC model’s assumption of a uniform distribution on [ βi, inline image], which holds for an individual bidder. Thirdly, the variance of the inline image falls with higher known costs. This is simply because of the smaller margin between one’s known cost (ci) and the maximum acceptable bid inline image, which appears most likely to the bidder: thus, in Kiel, the quartile βQ/cQ ratios evolve as 11.3, 1.9, 1.3 and 1.1. A similar trend obtains with the Perth data.

One can further ask how exactly the inline image might depend on costs. To answer this question, the relationship between individual inline image and the corresponding individual costs ci was investigated, using all participant costs. The Kiel data yielded the following best-fit linear relationship:8

image(4)

and the Perth data yielded

image(5)

where the stars indicate significance at the 1 per cent confidence level. A inline image computed using the average experimental cost of 122.5, valid for both replicates, would yield a value of 201 with the Kiel data and 219 with the Perth data, a difference of 9 per cent.

Given this difference, we may not yet have a reliable model describing the formation of bid cap expectations by bidders who have imperfect knowledge of the cost distribution. For the time being, we only have at our disposal some empirical relationships, the external validity of which is not guaranteed. More than two replicates would be needed to better understand the difference between relations (4) and (5). We therefore focus on exploring how far the BC model could be useful to policymakers if these experimental relationships could be reliably extrapolated to field data.9

4.2. Linking theory and experiment for policy assessment

With estimates of expected bid caps as obtained in the previous section, information on costs can be used to compute, using Equation (3″), landholders’ optimal bids. Costs being functionally linked to quantities abated, they can be considered in tandem. Estimates of quantities abated (N), costs (c) and optimal bids (b*) together determine the tender’s performance which can thus be assessed ex ante. The key issue, and the focus of the analysis, is the amount and quality of information on c and N available to the policymaker. Will the BC model be able to reliably compute optimal bids and assess tender performance ex ante if such information is of poor quality?

To elucidate this question, we need a benchmark that can help us disentangle the model’s intrinsic predictive potential from its sensitivity to the quality of information input. The limit case where costs and bid cap expectations are individually known can provide such a benchmark. This is the situation of the experimenter. The opposite, worst case scenario is defined by the situation where a policymaker has at his disposal only a single-point average estimate of abatement and costs, for example, a regional average, with no knowledge of local variations. An intermediate case is where the policymaker has available more than one-point estimate. We shall consider the case of four-point estimates, which typically represent landholder ‘cost-category pools’ in the target region.

This research strategy is represented in rows 1, 2 and 3 in Table 1. The lower indices a, q and i represent, respectively, the poor, the medium and the full information scenarios, which correspond to the one-point estimate, the four-point estimate and the full knowledge of the experimental (Ni, ci) set. The scenario in row 1 is of course irrelevant to the policymaker; its purpose is to evaluate the BC model, not the tender itself. Rows A and B in Table 1 define the theoretical and experimental benchmarks, respectively. Row A describes the strategy used by Latacz-Lohmann and Van der Hamsvoort (1997) in their theoretical analysis, and row B describes the results of its experimental implementation. Row A is the theorist’s approach; rows B and 1 describe the experimenter’s approach; and rows 2 and 3 describe the approach adopted in this study, linking theory and experiment for ex ante policy assessment and taking account of information deficiencies policymakers are usually confronted with.

Table 1. Use of the BC model for predicting the performance of a BC tender
 Information known byCosts and N abatementBid caps βBids bEvaluation of tendering performance
  1. Note: c, bidders’ per hectare abatement costs, a function of abatement (N). N, amount of nutrients (fertilizer) abated per hectare. Subscript a, average, single-point estimate. Subscript q, quartile, four-point estimate (3-point in Latacz-Lohmann and Van der Hamsvoort 1997). Subscript i, individual costs, bids or expected bid caps (as known only to the experimenter). b*, computed per hectare bids, using the BC model. β, bidder’s expected bid cap (highest expected cut-off bid per hectare). fe, empirical relationship using individual experimental data. In the one-point estimate scenario, the policymaker is assumed not to know upper and lower cost bounds. In the four-point estimate scenario, he only knows quartile averages.

ATheoretical model (Latacz-Lohmann and Van der Hamsvoort 1997) ci, cq and ca
 (q = 3-pt estim)
 with c = f(N)
β = f(c)
Assumed uniformly distributed ± 40% of average cost ca
b* = f{c, β(c)}BC model estimates of b* but with no info on β or bids.
Assumes validity of BC model
BExperimental benchmark Ni and ci
 (experimental)
βi(experimental) bi (experimental)Direct use of experimental bids
(No use of BC model)
1Experimenter (full information) Ni and ci
 (experimental) i = 44 or 27
Experimental βi
 βi = fe (ci)
bi* = f (ci, βi)BC model estimates of bi* (experim. bi serve as benchmark)
2Policymaker, with medium quality information Nq and cq (4-pt estimate) βq = fe(cq) bq* = f(cq, βq)BC model estimates of bq*
3Policymaker, with poor quality information Na and ca
 (1-pt estimate)
βa = fe(ca) ba* = f(ca, βa)BC model estimates of ba*

A key issue in Table 1 is the computation of the expected bid caps inline image from which, together with estimates of bidders’ costs, optimal bids (b*) are computed. The bid caps themselves are computed from the cost estimates, as per Equations (4) and (5), and are represented by the function fe in Table 1. The difference between rows A and 2 or 3 is that in the latter, one evaluates how well the BC model performs relative to the ‘true’ performance in row B, whereas the approach in row A just assumes the model is correct. As for row 1, it evaluates the BC model’s capacity to predict the ‘true’ results of row B given full experimental information on costs and abatement quantities. The effect of limited information can thus be isolated by comparing predicted policy performance in rows 2 or 3 with that in row 1. Again, the purpose of row 1 is purely to allow us to disentangle the role of limited information from the intrinsic potential of the model: it is not to be related to the policymaker’s information.

The results of this study are organised in section five according to the rationale of Table 1. Section six then builds on section five to examine under what information conditions the model might make the wrong policy recommendation. This is achieved by introducing an alternative but equivalent policy instrument, a fixed-price scheme with the same budget constraint as the BC tender.

5. How well does the BC model predict the tender’s performance?

We assess the performance of the tendering mechanism by using four different criteria, namely: outlay per unit of abatement (budget cost-effectiveness); cost of abatement per unit abated (economic efficiency); outlay per unit cost (rate of information rents); and the amount abated relative to the maximum possible amount if all bidders had been contracted (policy effectiveness).

5.1. The point of view of the experimenter: the model’s intrinsic predictive capacity

5.1.1. How well does the BC model predict individual experimental bids?

To assess how well the BC model predicts the tender’s performance, the experimenter must first assess how well it can predict the individual experimental bids. This establishes (or not) the model’s credibility. The two frames in Figure 2 plot predicted optimal bids against experimentally observed bids for the BC tender in replicates Kiel and Perth. The complete experimental data are provided in Appendix II. Optimal bids were computed for each bidder using Equation (3″). The 45° line represents perfect prediction. Two things can be observed. Firstly, prediction is less than perfect. Secondly, the model underestimates the experimental bids in Kiel slightly but systematically, the linear fit being everywhere above the 45° line, whereas (except for the lowest bids) the opposite is true in the Perth replicate.

Figure 2.

 Model performance: theoretically computed versus experimentally observed bids in Kiel and Perth for a BC tender. The 45° lines of perfect fit are shown. ***indicate significance at the 1 per cent confidence level.

One feature of the model may explain this slight over- or under-bidding: bidders are assumed in Equation (3″) to be risk neutral. The bidders in the Kiel experiment were measured to be somewhat risk-loving, with an average certainty equivalent ratio of 108 per cent, slightly greater than the risk neutral 100 per cent. Participants in Perth were clearly risk-averse, with an average certainty equivalent ratio of 78 per cent. The ratios of the average experimental bids to the computed bids were 1.08 in the Kiel replicate and 0.88 in the Perth replicate, indicating close agreement between two completely different mechanisms, the hypothetical lottery and the experimental tender with real money. This conforms to the expectation that risk-loving bidders ask more than if they were risk-neutral and risk-averse bidders ask less. In both Kiel and Perth experiments, the linear fit has a smaller slope than the 45° line, with the difference more marked in Perth. The BC model thus tends slightly to overestimate low bids.

Although not perfect, the BC model seems to yield reasonable predictions of the experimental bid data. If one relates the average of the absolute differences between computed and experimental bids to the overall average bid, the relative error ratio, or the average dispersion around the 45° line, it is 13 per cent for Kiel and 21 per cent for Perth. The correlation coefficients between the computed and the experimental bids are 91.6 per cent for Kiel and 83.4 per cent for Perth.

5.1.2. Predicting the tender’s performance

With full cost information, the experimenter evaluates the capacity of the BC model to predict the performance of the tender as follows. He first evaluates it with bids computed using the BC model, then compares this evaluation with the one obtained directly using the experimental data. The full cost information scenario provides an upper limit to the model’s predictive capacity. The results of this comparison can be read by comparing columns 1 and 2 of Table 2. The upper half of column 2 provides the four performance measures each in their appropriate units. The closer these measures are to the ones in column 1, the better the quality of the prediction. The lower half measures this quality in terms of per cent deviation from the results in column 1. Two things can be said from this comparison. First, even in the best of all worlds, the BC model’s predictive potential is not perfect. Perfection would require zero deviation on all performance criteria. However, secondly, the deviations remain small across all criteria and across both replicates Kiel and Perth, never exceeding 6 per cent. The BC model can thus be considered to be a credible tool to work with.

Table 2. Estimated BC tendering performance given information on abatement and bidder costsThumbnail image of

5.2 The point of view of the policymaker: the role of limited information on bidders’ costs

5.2.1. Information scenarios and cost distribution assumptions

In contrast to the experimenter, the policymaker will only have limited information on landholders’ costs. As per Table 1, we examine two information scenarios, a poor quality one where only a single-point (overall average) estimate is available on (N, c), and a medium quality one where four-point (quartile) estimates are available. In either case, the policymaker must make assumptions as to how the values are distributed around the single-point average or the four quartile averages, because the true distribution is unknown. He then simulates bids based on that information and his knowledge of the relationship between costs and bid caps as per Equations (4) or (5), for the Kiel and Perth data respectively. He finally simulates the selection of bids starting from the lowest assumed bid, until the budget constraint is met. Table 3 shows two possible distribution options which, in the absence of any other information, the policymaker might plausibly consider.

Table 3. Policymaker’s distribution assumptions for both information scenarios
Poor information scenarios: 1-point (average) estimate
Distribution assumption >(1)(2)(3)
Experimental data(Na, ua)(Na, Ca)
  1. Notes: u, c/N, average cost per unit of abatement. Index a, single-point overall average. Index q, four-point quartile averages. Underlined N refers to a non-distributed N.

Kiel data (44)
 Na = 59 (range)13–935959
 ua = 2.08 (range)0.38–2.810–3.03Irrelevant
 ca = 123 (range)5–2610–1765–241
Perth data (27)
 Na = 59 (range)13–935959
 ua = 2.07 (range)0.38–2.830–4.30Irrelevant
 ca = 122 (range)5–2640–2439–235
Medium information scenarios: 4-point (quartile) estimates
Distribution assumption >(1)(2)(3)
 Experimental data(N; c)(Nq, uq)
(N; c)
(Nq, Cq)
(N; c)
Kiel data (44)
 Q1 averages28; 2428; 3628; 29
 Q2 averages54; 8854; 9654; 90
 Q3 Averages71; 15971; 15371; 156
 Q4 averages86; 22886; 21386; 224
Perth data (27)
 Q1 averages28; 2428; 3528; 32
 Q2 averages54; 8854; 9654; 95
 Q3 Averages72; 16072; 15872; 166
 Q4 averages89; 24089; 21589; 236

In column 2, per hectare nutrient abatement N (indexed a and q for each information scenario, respectively) is kept constant and the cost per unit of abatement (u = c/N) is uniformly distributed from zero up to a maximum such that the initially known average remains unchanged; per hectare costs c are then distributed accordingly. In column 3, N is also kept constant but c is now distributed independently of N. The top half of the table provides the estimated ranges for Na and ca; the lower half provides quartile averages for each of the Nq and cq.

5.2.2 How well does the BC model predict with limited information?

The cost distributions of Table 3 serve as the basis for computing expected bid caps and optimal bids which then, together with the abatement distributions, determine the expected performance of the tender. Columns 3 and 4 in Table 2 present results for the two abatement and cost distributions in the poor information scenario, and columns 5 and 6 do so for the medium information scenario. The upper part of the table provides the predicted values for each of the four performance criteria. The lower part measures the quality of the prediction relative to the experimental data, measured in per cent deviations. We focus only on the absolute deviations.

Two things emerge. First, the BC model is able to predict tender performance very well in the medium information scenario (four-point estimate), but not in the poor information scenario (one-point estimate). However, even in the first case, the model cannot be considered to be reliable, because it predicts well in the Kiel replicate but poorly in the Perth replicate; it is only reliable for the criterion of economic efficiency (costs/kg N). Secondly, and rather surprisingly, cost distribution assumptions do not much affect these results. They make virtually no difference in the medium information scenario, and the results remain unreliable between the two assumptions in the poor information scenario. However, the difference between the Kiel and Perth replicates warrants further study. The number of bidders in Perth was smaller than those in Kiel (27 compared to 44), and the variance of Perth bids was higher, indicating poorer bidding consistency.

6. Would the BC model recommend the right policy?

A model that predicts wrongly can recommend the wrong policy. In particular, it can recommend that policy A be preferred to policy B when in fact the opposite would yield better results. This section investigates this possibility by considering as an alternative to the tender an equivalent fixed-price scheme. The equivalence is defined by the constraint that the total budget outlay must remain unchanged. More precisely, we are interested in the minimum uniform payment rate (MUP) that would have resulted in the same budgetary expenditure as the auction. The MUP is defined as the fixed-rate payment to the lowest-cost participants up to the budget constraint. That is, landholders are accepted into the scheme starting from the lowest opportunity costs until the budget is exhausted. The MUP thus represents the lowest possible fixed-rate payment subject to the budget constraint. This provides a least-cost uniform pay rate, a theoretical but ‘absolute’ benchmark for comparison. We construct a separate MUP for each information treatment under which the corresponding tender has been evaluated. We thus carry out as many comparisons between the tender and its equivalent MUP as there are information scenarios and cost distribution assumption, as per Table 2.

Of course, the number of contracts awarded will differ. Under the MUP, they number 26 instead of 29 in the Kiel replicate and 16 instead of 19 in the Perth replicate. This can be seen by comparing columns 1 in the top part of Tables 2 and 4. The top part of Table 4 is structured similarly to Table 2, except that the figures show for both replicates Kiel and Perth the performance of the MUP scheme instead of the BC tender, under the same information and cost distribution assumptions.

Table 4. Decision to run the BC tender rather than a fixed-rate minimum uniform price (MUP) schemeThumbnail image of

The second horizontal section of Table 4 shows for both replicates whether the BC model would recommend running the tender rather than the alternative policy, the fixed-price scheme with minimum uniform price (MUP). If so, a ‘yes’ is shown, otherwise a ‘no’ appears. Except for the fourth criterion, the lower the performance measure, the better. The ‘per cent max N abated’ on the other hand is better the higher it is. The ‘?’ indicates an indecisive outcome, insofar as the difference in the performance of the MUP and the tender are considered too small – here less than ±2.5 per cent.

These results are only an intermediary for examining the core question: will use of the BC model to predict the performance of the tender make the wrong recommendation? ‘Wrong’ is defined by a recommendation that differs from that made using the experimental data, taken as a benchmark (column 1). If the recommendation is the same (ie correct), a ‘1’ shows in the third (bottom) horizontal part of Table 4 and the corresponding cell is shaded; otherwise, a non-shaded ‘0’ shows.

With full information (column 2), the model always makes the correct recommendation. Except for an indecisive case (which disappears for an uncertainty of less than 2.5 per cent), the model also makes the correct recommendation under both cost distribution assumptions in the medium information scenario (columns 5 and 6). The same results obtain for the poor information scenario (columns 3 and 4). Comparing this result for the Perth replicate in both Tables 2 and 4 shows that the low accuracy of the model’s prediction does not prevent it from making the right recommendation. Columns 3–6 show that the Perth recommendations are as robust as those of Kiel. Interestingly, and contrary to the results shown in Table 2, the quality of recommendations in both Perth and Kiel does not appear to be sensitive to the quality of information available to the policymaker.

7. Conclusions

The purpose of this study was twofold. First, it aimed to show how theory and experiments can be linked to improve ex ante policy assessment. Second, it aimed to see whether a model used with limited information on input variables can still be useful for making policy recommendations.

The model for budget-constrained tenders developed by Latacz-Lohmann and Van der Hamsvoort (1997) formulated optimal bids by relying on an exogenous variable, the bidders’ expectations on the maximum bid that would be acceptable to the policymaker. However, it did not model expectation formation, when such expectations are not observable. This study therefore supplemented the theory by implementing the model in a controlled laboratory experiment where bidders were asked to state their bid cap expectations along with their bids. The experiments yielded empirical relationships between bidder costs and bid cap expectations which could then be used to compute optimal bids.

On the basis of these optimal bids and of the available budget, it is possible to measure the performance of a tender before actually running it in the field, and thus obtain information on whether a tender would be a desirable option or not. This study focussed on the fact that performance estimates will be affected by the quality of the information input typically available in the field. Can the theoretical model, complemented by its experimental implementation, still be useful under information limitations typical of policy environments?

The results obtained from the experiments have not yet allowed us to produce a reliable model for the formation of bid cap expectations. The small but significant difference across the two experimental replicates regarding the empirical relationships linking bidder costs to bid cap expectations calls for some caution until further replicates are run. Previous experiments by Brookshire et al. (1987) and List and Shogren (1998) suggest that, if properly designed, experimental auctions tend to be externally valid. Still, the validity of our experimental relationships for use with field data in a policy context is not guaranteed.

Overall, the study suggests that Latacz-Lohmann and Van der Hamsvoort’s (1997) model of a budget-constrained tender will make the correct recommendation when comparing the tender to an equivalent fixed-price scheme, even if the policymaker has only limited information on the model’s key input variables, namely averages of abatement quantities and costs. This holds even if the accuracy of the model’s predicted performance is far from perfect, in this study off by up to 20 per cent either way.

Footnotes

  • 1

    The most relevant material and references for the present study can be found in Kagel and Roth’s (1995)Handbook of Experimental Economics; in Plott and Smith’s (2008)Handbook of Experimental Economics Results; and especially in Lusk and Shogren’s (2007)Experimental Auctions: Methods and Applications in Economic and Marketing Research, in particular chapter 9.

  • 2

    This ex post reserve price should be distinguished from the ex ante reserve price, as they can have different impacts on the capacity of bidders to learn it in repeated auctions. See for example, Reichelderfer and Boggess (1988) and Shoemaker (1989) for an analysis of this in the case of the US Conservation Reserve Program.

  • 3

    This is not an essential assumption and could be relaxed to include risk aversion, as done by Latacz-Lohmann and Van der Hamsvoort (1997). However, it would not add much to the present argument and might confuse matters unnecessarily.

  • 4

    By contrast, uniform (second-price) sealed-bid auctions should in theory lead to bidding one’s true opportunity costs, both in TC and BC tenders; but they have rarely been used in conservation contracting programs, mainly because of the potential for the policymaker to ex post manipulate bids.

  • 5

    In particular, the two following chapters: Risk aversion in experiments: An introduction.C. Cox, and G.W. Harrison 2008 (pp. 1–7), and: Risk Aversion in the Laboratory.Harrison and Rutström 2008 (pp. 41–96) which provide the necessary background to this approach.

  • 6

    For Australia, see for example, Stoneham et al. (2003); for the Georgia water buyback auctions, see Cummings et al. (2004); for the Northeast Groundfish Fishery Buyout Program, see Walden et al. (2003).

  • 7

    This notation cQ differs from the one used later (cq), in that the former represents the bidder’s information, whereas the latter represents the policymaker’s information. cQ represents the knowledge a bidder has of his belonging to one of the four cost quartiles; cq will represent the quartile pool’s average cost as estimated by the policy maker.

  • 8

    For this purpose, the complete data set of three repetitions in both replicates was used, as there was no visible trend across them. The data from the second and third repetition were not otherwise used, as they had been generated for a purpose different from the one focussed on in this study.

  • 9

    This hinges on how well the experiment is calibrated to the policy context, namely w.r.t. to key parameters defining auction design (budget-to-bidders ratio, cost spread, etc.). We do not elaborate any further here on external validity and experimental calibration in policy test-bedding, an area that, in spite of Brookshire et al.’s (1987) and List and Shogren’s (1998) early work, has only seriously begun to be investigated in recent years. See for example, Schramm (2005), Garcia and Wantchekon (2009), Boly (2009), Bardsley (2010) and, for an overview, Lusk and Shogren (2007: chap. 9) and Cox and Harrison (2008, chap. 2 and 5).

Appendices

Appendix I: Pages 2 and 4 of the budget-constrained tender in Perth

(Page 1 provided the ‘story’ and the motivation.)

Individual farm data (page 2) to work out the costs of your participation in our Swan River protection program

Suppose you are a horticulturalist and producing vegetables for Perth. Output as a function of N fertiliser use is given by the following graph:inline image

The optimal fertiliser amount maximises value of output minus cost of inputs (N fertilisers).

This results in the following:

 With N = 80With N*Difference
Net revenue (ECU/ha) Experimental Currency Units   

My costs of participation are ………………… ECU/ha (= the income difference)

Important:

  • Your costs of participation are known only to you and your private adviser; they are not known by the environmental authority or anyone else.
  • Your competitors all have different participation costs. So that you may have a better idea of how you compare relative to your competitors, we give you the following information: you are in one of the following four quartiles:

inline image (Page 3 provided “some advice from your private consultant”).

Bidding sheet (page 4)

Now it is time you put in your bid. Please first write in your full name. We shall need it to pay you your gains if you are among the winners.

inline image1. First please write down the highest possible bid you believe will be accepted. This must be your best guess:

inline image2. Now please write in the amount we must pay you so that you accept to participate in our Swan River protection program:

inline imageThe selection of participants will be made on the ground of their bid in ECU/ha. The lowest bid will be selected first, then the second lowest, then the third lowest, and so on until the available budget of 2300 ECU is exhausted.

For paying the winners in real money ($), the following rules hold

  • The successful bidders will be paid, not their bid, but the gains from their participation in the program, that is, bid minus participation costs.
  • Unfortunately, because of limited research funds, we cannot pay out the full value of the gains, but only a fixed percentage of the gains. This percentage will be calculated after the end of the bidding session. Of course, the higher your gains, the higher your proportional payment. For this session, the funds we have available for payment to this group total an amount of approximately $300.

Appendix II: Raw experimental data from both replicates (ci, inline image and bi)

Data are ordered by bids (bi), with indication of selection cut-off line.

#KielPerth
Q ci inline image bi Q ci inline image bi
  1. Note: The postmarginal bid in Perth of 191.01 was indeed put in as such by the participant. Q, Cost Quartile. The inline image refer to the highest acceptable bids estimated by the participants.

1118504811327525
2115300601910050
3131856111840055
4254806313314860
51575751510065
6111100851398069
727710510014913070
8135250100256200100
9259125100287160119
102811201002108190128
11127120109127400150
121491351303137155154
13298140130265160160
141391501332103180160
152108150140315785162
161441451443171250180
1731371481483164250186
182653001502116300190
1921191751503186195191
2031441701603179210191.01
2131501881663125300200
22162001704237245245
2331311801704203500253
2421141781774229400260
2531861951944249280264
2631712001984258150268
2721032502004264175275
283125200200    
293177210200    
304216219219    
3119275225    
324210140230    
334221235233    
344224250235    
354205240239    
363191250240    
374234246245    
383157256255    
393182350260    
404255270264    
414249279274    
424237295283    
434261290285    
444200295290    

Ancillary