Evaluating the impact of biodiversity offsetting on native vegetation

Abstract Biodiversity offsetting is a globally influential policy mechanism for reconciling trade‐offs between development and biodiversity loss. However, there is little robust evidence of its effectiveness. We evaluated the outcomes of a jurisdictional offsetting policy (Victoria, Australia). Offsets under Victoria's Native Vegetation Framework (2002–2013) aimed to prevent loss and degradation of remnant vegetation, and generate gains in vegetation extent and quality. We categorised offsets into those with near‐complete baseline woody vegetation cover (“avoided loss”, 2702 ha) and with incomplete cover (“regeneration”, 501 ha), and evaluated impacts on woody vegetation extent from 2008 to 2018. We used two approaches to estimate the counterfactual. First, we used statistical matching on biophysical covariates: a common approach in conservation impact evaluation, but which risks ignoring potentially important psychosocial confounders. Second, we compared changes in offsets with changes in sites that were not offsets for the study duration but were later enrolled as offsets, to partially account for self‐selection bias (where landholders enrolling land may have shared characteristics affecting how they manage land). Matching on biophysical covariates, we estimated that regeneration offsets increased woody vegetation extent by 1.9%–3.6%/year more than non‐offset sites (138–180 ha from 2008 to 2018) but this effect weakened with the second approach (0.3%–1.9%/year more than non‐offset sites; 19–97 ha from 2008 to 2018) and disappeared when a single outlier land parcel was removed. Neither approach detected any impact of avoided loss offsets. We cannot conclusively demonstrate whether the policy goal of ‘net gain’ (NG) was achieved because of data limitations. However, given our evidence that the majority of increases in woody vegetation extent were not additional (would have happened without the scheme), a NG outcome seems unlikely. The results highlight the importance of considering self‐selection bias in the design and evaluation of regulatory biodiversity offsetting policy, and the challenges of conducting robust impact evaluations of jurisdictional biodiversity offsetting policies.


Victoria's Native Vegetation Framework
The Native Vegetation Framework ran in Victoria from 2002-2013, when it was superseded by new native vegetation regulation associated with a slightly-altered offset policy.Under the Framework, applications to remove native vegetation were sent to local councils and processed through the planning system, with larger impacts and those to ecologically significant biodiversity conventionally referred to the state authorities for approval (this pathway comprised approximately one third of applications in 2010/2011; DSE 2012).Offsets required to compensate for clearance events that were referred to the State government were then registered.
Entering into an offset agreement committed landholders to both protect the registered native vegetation in perpetuity, and implement management actions (most commonly grazing exclusion and invasive plant or weed removal) under a 10-year management plan to deliver enhancements in biodiversity across that time period.Biodiversity gains were calculated using the 'habitat hectares' (HH) currency (Parkes et al. 2003), which allowed an estimate of the predicted gains in biodiversity over the 10-year management lifetime.These biodiversity gains translated into biodiversity credits which could be used directly to compensate for native vegetation clearance conducted by the same entity as that creating the offset ('first party offset'), or sold to other land clearers to offset their liabilities ('third party').The State government implemented the Bushbroker programme, an initiative to create a regulatory market in offsets whereby land clearers could purchase offset credits to offset their native vegetation liabilities, which has since developed into a fully-fledged state offsetting sector brokered predominantly by private firms.

Habitat hectares
The HH approach is one of the original and most influential area*condition biodiversity metrics implemented in biodiversity offsetting systems around the world, which has served to underpin numerous derivative metrics such as England's Biodiversity Metric (Crosher et al. 2019).To calculate a site's HH score, a qualified consultant conducts a site-based assessment, and scores the ecological quality of each ecological vegetation class (EVC) on the impacted site according to a number of ecological criteria (Table S1; Parkes et al. 2003).Each criteria is scaled so an EVC scores the maximum number of points if the ecological criteria are equivalent to those found at an intact reference patch of that EVC.Different ecological criteria contribute variably to the overall habitat score for the site, with the most ecologically important criteria contributing more to the overall habitat score.The total habitat score for the site adds up to a maximum of 100.This habitat score is then multiplied by the total area of the site in ha to yield the HH score.For example, 10ha of an intact reference EVC would score 10HH, and 10ha of a moderate condition EVC which achieved an overall habitat score of 50 would yield 5HH.

Ecological criteria
Maximum value (sums to 100)

Coverage of large trees 10
Canopy cover 5 Richness and degree of modifications of understory strata 25 Invasiveness and coverage of weeds 15

Plant recruitment 10
Coverage of organic litter 5 Total length of logs on site 5

EVC patch size 10
Neighbourhood / connectivity with surrounding vegetation patches 10 Distance to core area (vegetation patch >50ha) 5

Criteria for including offsets in evaluation
To decide which EVCs to include in our analysis, we used the information from the EVC benchmarks (https://www.environment.vic.gov.au/biodiversity/bioregions-and-evc-benchmarks).We include all EVCs which, when in good condition, would be expected exceed the threshold of >2m vegetation height and >20% canopy cover based on the information provided in the EVC benchmarks (Table S2).

Statistical matching
Smaller standardised mean differences in covariate values between offsets and matched controls indicate better matches, with standardised mean differences <0.1 considered high-quality matches (Greifer 2022).For all specifications, we match 1:1 and without replacement, as our pool of potential controls is larger than treated observations.As a robustness check we conduct two commonly-used matching methods (Sonter et al.

Background trend analysis
To test for parallel trends before implementing the difference-in-difference analysis, we followed the methods of Devenish et al. (2022).We regressed the pre-intervention woody vegetation cover data against the interaction between whether the site is from the control or intervention sample, and year.If the interaction is significant, it implies that there is a significantly different time trend between the offsets and controls.Regression outputs are given in Table S4.

Regression outputs
Here we present full outputs of our regression comparing changes in woody vegetation cover between early regeneration offsets and matched controls (Table S5), and early regeneration offsets and late regeneration offsets (Table S6).

Sensitivity analyses
Effects of varying the threshold for assigning offsets to 'regeneration' or 'avoided loss' In our main analysis, we chose the threshold of a proportion native vegetation cover of 0.95 to assign offsets to the regeneration or avoided loss category, because this retains an effective sample size for the regeneration offsets.As we lower the threshold, the sample size declines (0.9, N early offsets=37, total area of regeneration offsets 307 ha; 0.8, N early offsets=29, total area of regeneration offsets 227 ha), and the mean woody vegetation cover in our offsets declines.This means that there is greater potential for woody vegetation cover to increase over the 10-year evaluation period.Therefore, as we reduce the sample size and lower the threshold, the effect size of the impacts of offsets on native vegetation increases slightly (these offsets that start with lower baseline woody vegetation cover experience larger increases in woody vegetation cover than offsets starting with a higher baseline woody vegetation cover).
The regression outputs for the diff-in-diff regression comparing the change in woody vegetation cover in early regeneration offsets and matched non-adopter parcels (using our core model) at varying baseline woody vegetation thresholds is presented in Table S7.When the threshold for regeneration offsets is set at a baseline proportion woody vegetation cover <0.9, early offsets are associated with an increase in woody vegetation cover of 3.09% per year relative to controls, implying that regeneration offsets led to a mid-point additional increase in woody vegetation cover of 95 ha.When the threshold for regeneration offsets is set at a baseline proportion woody vegetation cover <0.8, early offsets are associated with an increase in woody vegetation cover of 4.04% per year relative to controls, implying that regeneration offsets led to a mid-point additional increase in woody vegetation cover of 92 ha.The regression outputs for the diff-in-diff regression comparing the change in woody vegetation cover in early regeneration offsets and future regeneration offsets at varying baseline woody vegetation thresholds is presented in Table S8.When the threshold for regeneration offsets is set at a baseline proportion woody vegetation cover <0.9, early offsets are associated with an increase in woody vegetation cover of 1.85% per year relative to controls, implying that regeneration offsets led to a mid-point additional increase in woody vegetation cover of 57 ha.When the threshold for regeneration offsets is set at a baseline proportion woody vegetation cover <0.8, early offsets are associated with an increase in woody vegetation cover of 2.09% per year relative to controls, implying that regeneration offsets led to a mid-point additional increase in woody vegetation cover of 47 ha.

Effects of removing sites burned by wildfires during the analysis period
Removing the landholding containing offsets which experienced catastrophic loss of woody vegetation cover in the 2009 Black Saturday fires altered the outputs of the regression models, as the rapid vegetation regrowth in these offsets caused by the fire but coincident with the onset of offset management in 2008 contributed in increasing the effect size of offset management.Full results of the core regression analyses excluding these offsets is in Table S9.
2019; Devenish et al. 2022), and progressively reduce the caliper until there are no further gains in balance or until large numbers of observations are dropped.We use: a) nearest neighbour matching on propensity scores derived using logistic regression; b) Mahalanobis distance matching with exact matching on land use and a caliper of 1 standard deviation; c) and d) the same as b) but with 0.5 and 0.25 standard deviation calipers respectively.The performance of our alternative matching specifications is detailed in Figures S3 and FiguresS4.

Figure
Figure S3.Loveplots showing the standardised mean difference between full and matched datasets and treated observations (regeneration offsets) under various matching specifications.All specifications achieve full matching of treated and control observations: A) 1:1 propensity score matching without replacement; B) 1:1 Mahalanobis distance matching with 1 standard deviation calipers and exact matching for the land use for each land parcel; C) As B, with 0.5 standard deviation calipers; D) As B, with 0.25 standard deviation calipers Figure S3.Loveplots showing the standardised mean difference between full and matched datasets and treated observations (regeneration offsets) under various matching specifications.All specifications achieve full matching of treated and control observations: A) 1:1 propensity score matching without replacement; B) 1:1 Mahalanobis distance matching with 1 standard deviation calipers and exact matching for the land use for each land parcel; C) As B, with 0.5 standard deviation calipers; D) As B, with 0.25 standard deviation calipers

Table S2
. Summary of all of the EVCs included in offsets in the Victorian offset database, noting which would be expected to be classified as complete woody vegetation cover in our outcome dataset and therefore which are included in our evaluation

Table S3 .
Summary of the data layers used as covariates in the regressions and statistical matching, and justifications https://www.worldpop.org/geodata/summary?id=17302Remoteness 1km resolution remoteness raster."ARIA+ measures remoteness in terms of access along the road network from populated localities to each of five categories of Service Centre based on population size.If one thinks of ARIA as based on the distances people have to travel to obtain services, then populated localities are where they are coming from, and Service Centres are where they are going to." Remoteness ranges from 1-15.https://arts.adelaide.edu.au/hugocentre/services/ariaCollated data representing fire locations from Ward et al. (2019).https://conbio.onlinelibrary.wiley.com/doi/full/10.1111/csp2.117

Table S4 .
Regression outputs for the regressions testing the parallel trends assumptions.Values represent regression coefficients, standard errors in brackets.Significance (p<0.05) is indicated by *

Table S5 .
Regression outputs for our linear mixed effects model estimating the impact of offset management on woody vegetation cover, comparing regeneration offsets with matched control land parcels.Coefficient estimates and associated standard errors are presented.For the categorical Land use variable, the baseline land use against which alternatives are compared is agriculture.P-values are denoted by stars: *= p<0.05, ***=p<0.001

Table S6 .
Regression outputs for our linear mixed effects model estimating the impact of offset management on woody vegetation cover, comparing early regeneration offsets with late regeneration offsets.Coefficient estimates and associated standard errors are presented.For the categorical Land use variable, the baseline land use against which alternatives are compared is agriculture.P-values are denoted by stars: *= p<0.05, ***=p<0.001

Table S7 .
Regression outputs for our linear mixed effects model estimating the impact of offset management on woody vegetation cover, comparing early regeneration offsets with matched non-adopters, and assuming different thresholds for categorising regeneration offsets.Coefficient estimates and associated standard errors are presented.P-values are denoted by stars: *= p<0.05, ***=p<0.001

Table S8 .
Regression outputs for our linear mixed effects model estimating the impact of offset management on woody vegetation cover, comparing early regeneration offsets with future offsets, and assuming different thresholds for categorising regeneration offsets.Coefficient estimates and associated standard errors are presented.For the categorical Land use variable, the baseline land use against which alternatives are compared is agriculture.P-values are denoted by stars: *= p<0.05, ***=p<0.001

Table S9 .
Regression outputs for our linear mixed effects models estimating the impact of offset management on woody vegetation cover, excluding sites burned by wildfires.Coefficient estimates and associated standard errors are presented.P-values are denoted by stars: *= p<0.05, ***=p<0.001