Measuring how far protected areas (PAs) reduce threats to nature is essential for effective conservation. This is especially important where a high degree of threat is coupled with opportunities for increasing conservation investments, such as in the Brazilian Cerrado. We examined the effectiveness of strictly protected and multiple-use PAs as well as indigenous lands (ILs) in reducing conversion in Cerrado from 2002 to 2009 by using matching methods to sample protected and unprotected sites similarly exposed to pressures. We found that both types of PAs and ILs experienced lower habitat conversion during this period than did matched unprotected sites, whether results were analysed for individual PAs or for PA networks as a whole. Judging from their matched unprotected sites, strictly PAs had similar levels of baseline conversion to multiple-use PAs, but were more effective at reducing it. This may be expected as multiple-use PAs are under less restrictive land-use rules. ILs had a strong effect in reducing conversion, though baseline rates in matched areas were also high. Our results highlight the usefulness of PAs in the Cerrado and the value of research that differentiates among PA categories.
Protected areas (PAs) are an important focus of global conservation efforts, so measuring their effectiveness is essential for conservation. To achieve this, quantifying the extent of PAs is important, but it must be complemented by evaluation of their effectiveness in reducing threats to wild nature (Chape et al. 2005). Such evaluation is especially urgent for ecosystems that are experiencing a great level of threat coupled with opportunities for increasing conservation investments. This is the case for the Brazilian Cerrado.
A global hotspot of biodiversity (Myers et al. 2000) that originally covered nearly 2 million km2 of Brazil, the Cerrado biome is a mosaic of savannas, forests, and grasslands (Sano et al. 2010) with high species richness, in particular for birds (Silva 1995) and vascular plants (Ratter et al. 2003). Conversion rates accelerated over the past four decades as advances in farming technology enabled Cerrado to become Brazil's main agricultural frontier (Klink & Moreira 2002). By 2002, around 40% of the biome had been converted, mostly to cultivated pastures and agriculture (Sano et al. 2010). On the other hand, conservation investments appear to be growing, with a total grant of US$ 13 million approved by 2010 for the Sustainable Cerrado Initiative (Marinho & Aguillar 2010), and the release of the Action Plan for Preventing and Controlling Deforestation and Fire in Cerrado—the operational instrument for Brazil's target of reducing Cerrado's conversion rate by 40% by 2020 (MMA 2011).
One of the main strategies of the Action Plan is to improve and expand the PA network. However, little is known about its effectiveness and whether it varies across PA types. The PA system in the Cerrado is governed through the Sistema Nacional de Unidades de Conservação (SNUC) (MMA 2003), which classifies them as strictly protected areas (SPs), for categories that allow only the indirect use of natural resources, and multiple-use protected areas (MUs), for categories aiming to reconcile conservation with sustainable use. An additional designation that has been shown to contribute to conservation outcomes elsewhere in Brazil (Nepstad et al. 2006) is indigenous lands (ILs). In 2010 SPs covered roughly 3% of the biome and MUs 5%, whilst ILs covered 4% (MMA 2011). Despite this variation in PAs objectives in the Cerrado and elsewhere, studies that compare effectiveness across PA types are scarce (Nelson & Chomitz 2011; Nolte et al. 2013; Pfaff et al. 2013).
Recent improvements in measuring PA effectiveness through the use of matching methods provide an opportunity to address this gap. These involve comparing PAs with unprotected areas that have similar characteristics that could affect their exposure to pressures, such as soil quality and proximity to infrastructure (Joppa & Pfaff 2009). Failing to control for these characteristics can provide misleading results, as PAs are often located in areas where pressures are relatively low (Joppa & Pfaff 2009). Matching has increasingly been applied in conservation-related research (Costello et al. 2008; Morgan-Brown et al. 2010), with recent matching studies quantifying PA effectiveness in Costa Rica (Andam et al. 2008), Sumatra (Gaveau et al. 2009) and the Amazon (Nolte et al. 2013), as well as regionally and globally (Joppa & Pfaff 2011; Nelson & Chomitz 2011).
However, to date no such studies have examined PA effectiveness in the Cerrado, or in other savanna biomes. Here we apply matching methods to explore whether habitat conversion is lower inside Cerrado PAs than in comparable unprotected areas, and whether effectiveness levels vary among PA types.
The fact that conversion in Cerrado is not distributed homogeneously throughout the landscape (Figure 1) indicates that pressures are spatially varied. In order to measure PA effectiveness whilst controlling for this spatial variation, we used the study design detailed later.
Our outcome was a binary score indicating whether a given area was converted between 2002 and 2009 according to conversion data from the Integrated Warning Deforestation System (SIAD) (LAPIG 2012). SIAD data are generated using MODIS satellite imagery, with a vegetation change threshold of 15% followed by visual inspection using higher resolution imagery (Landsat TM and CBERS CCD) to help separate conversion events from vegetation changes caused by seasonal droughts (Rocha et al. 2011). A validation study based on field inspections and using a threshold of 30% found that SIAD data had a 79% success rate in detecting conversion, with 21% misclassification due to the effects of seasonal droughts and fire (Ferreira et al. 2009).
We included all PA types in the Brazilian SNUC legislation (henceforth SNUC PAs) and ILs. We considered SNUC PAs separately from ILs, as they are governed by different legislation. SNUC PAs were subset into SPs and MUs, which comprise 5 and 7 SNUC categories, respectively, with varying levels of restrictions on land use and a wide range of management objectives. In the international classification (Dudley 2008), SPs and MUs correspond broadly to IUCN categories I–IV and V–VI, with ILs classed as a governance type that can span any IUCN category. We used PA boundaries data provided by the Brazilian database (MMA 2012), and downloaded from the SIAD database (LAPIG 2012). We were unable to include private reserves as they are not yet mapped.
In a Geographical Information System we created a grid of 250 × 250 m cells covering the officially recognized extent of the Cerrado (IBGE 2012) and a points layer corresponding to the center of each cell. In our sample we used only those points that intersected native vegetation remnants present in the 2002 land-cover data (Sano et al. 2010). These points were then identified as being in PAs, ILs, buffers, or control samples, as described later.
PA or IL sample
We included all points that intersected SNUC PAs (1,137,972) and ILs (1,202,068) designated before 2002. Points that fell in more than one SNUC PA type were assigned to the PA with the strictest protection, for example, a point shared by an SP and an MU was assessed as part of the former only. Where a point fell both within a SNUC PA and in an IL it was assigned to the SNUC PA. Finally, we excluded 49 points that fell in overlaps between PAs of the same category, as these could not be assigned to a single PA. Our sample included 107 SNUC PAs (62 SPs, 45 MUs) out of 116 designated by 2002, and 62 out of 67 ILs, leaving out those which had no remnants of native vegetation in 2002 or where such remnants did not intersect our sample points. Other studies using matching methods pooled all sample points within a PA network (Andam et al. 2008; Joppa & Pfaff 2011; Nelson & Chomitz 2011), with the result that larger PAs had more weight than smaller ones in the overall estimates. Here, we sampled (and generated a matched, unprotected control sample for) each PA individually, which allowed us to analyse not only the network effect, but also the average effect of individual PAs (by weighting each PA equally) as well as the variation in PA effectiveness across our sample.
In order to test for displacement of conversion from inside PAs to their immediate surroundings (i.e., local scale leakage; Ewers & Rodrigues 2008) we sampled all points (1,471,059) within 10 km of our PA or IL sample, excluding any that fell inside other PAs or ILs.
We sampled a random set (8,129,688) of those points at least 10 km away from any PA or IL. This gave us approximately 7 control points per PA or IL point, as well as 5 per buffer point, which was considered adequate for matching.
As our study included small PAs, we followed the recommended approach for small sample sizes (Brookhart et al. 2006) of focusing on matching variables potentially associated with the outcome in question, which is habitat conversion. Based on knowledge of Cerrado (Klink & Machado 2005) and previous studies (Andam et al. 2008; Joppa & Pfaff 2009; Nelson & Chomitz 2011), these variables were: measures of accessibility (distance to paved roads and nearest state or federal capital city (IBGE 2012), complemented by travel time to the nearest city with population ≥50,000 (Nelson 2008)); rainfall and agricultural suitability (based on features, such as soil fertility, slope, salinity, and risk of flooding) (IBGE 2012); vegetation type according to Sano et al. (2010) split into major formations—seasonal deciduous forest, seasonal semi-deciduous forest, dense ombrophilous forest, open ombrophilous forest, pioneer formations and steppe savanna—and cerrado physiognomies—florestado, arborizado, parque, and gramínio-lenhoso; and elevation (Reuter et al. 2007).
We used the Matchit package (Ho et al. 2007), in R environment ×64 2.14.0 (R Core Team 2012), which fits a logistic generalized linear model where the treatment assignment is the response variable, and the matching variables are the predictors. The predicted values from the model, termed propensity scores, range from 0 to 1 and represent the probability that a given site belongs to the treatment group based on the matching variables. These values are then used for matching treatment and control samples with similar propensity scores in order to improve balance between samples (Rosenbaum & Rubin 1983).
As our minimum matching criterion, we used a caliper of 0.25 standard deviations of the propensity scores, and exact matching for agricultural suitability and vegetation type. Due to the large volume of data, we considered each combination of vegetation type and agricultural suitability in isolation for the largest sets, since matching for these two variables had to be exact. Some combinations still included too large a dataset; in these cases, we used a random sample of 600,000 control points when the control pool alone was too extensive. When the number of treatment points was higher than 100,000 we used a subset of the data (600,000 points, 90,000 of which were treatment) to estimate the coefficients of the generalized linear model that defines the propensity scores, and then estimated the scores for the full dataset. We used a no-replacement policy (where each control site is matched to only one treatment) to increase independence in the control sample (Stuart 2010), and one control per treatment to improve the number of matches.
We were able to match 85.7% of the original SNUC PA sample to controls that suited the criteria, resulting in 975,661 matched pairs. For our IL sample, we matched 57.4% of the points, resulting in 690,425 pairs. We then grouped these pairs according to the PA or IL intersected by the treatment point, resulting in a final sample of 106 SNUC PAs (62 SPs, 44 MUs) and 62 ILs, as for one PA there were no successful matches. As the number of matched pairs per PA varied, we created a cohort of 88 SNUC PAs and 50 ILs that included only those with at least 50% successful matches and 20 matched pairs, and used this cohort to check the impact of sampling intensity on our results. For our buffer sample, we matched 95.0% of the points, resulting in 1,397,620 matched pairs.
Measures of the effect of PAs
Previous studies have quantified the effect of PAs in reducing threats in different ways (Andam et al. 2008; Gaveau et al. 2009; Joppa & Pfaff 2011; Nelson & Chomitz 2011); we therefore used a variety of metrics.
This is the difference between conversion in a control sample and in its matched PA (so a positive value means that the PA experienced less conversion than its control sites). We calculated these metrics for each PA in order to estimate their individual average effect (where all PAs had the same weight in the overall results regardless of their size), in addition to their network effect (where the result for each PA was weighted by its total area of native vegetation). We compared the median absolute effect of each of our groups of PAs (all SNUC PAs combined, SPs and MUs subsets, and ILs) with zero, and with each other (SNUC PAs vs. ILs and SPs vs. MUs), using unpaired Wilcoxon tests. We performed the same comparisons for network-wide medians (area-weighted) by bootstrapping (10,000 samples).
This metric complements the information provided by the absolute effect metric by measuring how far baseline conversion rates have been changed by protection, thereby allowing for comparing results with those of regions or periods of time with different baselines. We calculated the relative effect of each PA as the difference between conversion in a control sample and in its matched PA, divided by conversion in the control sample. We compared the median relative effect of each of our groups of PAs (all SNUC PAs combined, SPs and MUs subsets, and ILs) with zero, and with each other (SNUC PAs vs. ILs and SPs vs. MUs), using unpaired Wilcoxon tests. We performed the same comparisons for network-wide medians (area-weighted) by bootstrapping (10,000 samples).
Pooled relative effect
Because previous papers have measured PA effectiveness by pooling data across an entire PA network (Andam et al. 2008; Gaveau et al. 2009; Nelson & Chomitz 2011), we also calculated the pooled relative effect for each of our groups of PAs. For each network (all SNUC PAs combined, subsets SPs and MUs, and ILs), we computed the mean conversion found in the pool of matched control locations and subtracted it by the mean conversion found in all matched locations within the network, and divided the result by the mean conversion found in the pool of matched controls. This metric differs from our previous network estimates as it pools all locations within a given group (protected and controls) rather than keeping the pairing information of which controls belong to which PA (which is preserved in the area-weighted median). To make the comparison with other studies (with differing baseline conversion rates) meaningful, we expressed their results as relative effects, using information in the papers and additional data sent by their authors.
Conversion in buffers versus control areas
We explored the possibility that any effects of PAs may be offset by displacement of conversion into surrounding areas (local leakage) by comparing conversion in a 10 km buffer around PAs or ILs with that in the buffer's matched control sample. Because many PAs in Cerrado are adjacent, there was great overlap between their 10 km buffers. Therefore, we used a pooled sample of points from all buffers instead of a discrete sample for each buffer.
Absolute effect of PAs
From 2002 to 2009 SNUC PAs in Cerrado had a positive absolute effect overall (Figure 2a, Wilcoxon test, V = 3844, N = 106, P < 0.001), and so did each of their two major subsets—SPs (V = 1431, N = 62, P < 0.001) and MUs (V = 607, N = 44, P < 0.001). This effect was generally larger for SPs than for MUs (unpaired Wilcoxon test, W = 1,822, N = 62 and 44, P = 0.003), although conversion in control sites for the two categories was similar (W = 1,352, N = 62 and 44, P = 0.941). ILs also had a positive effect (Wilcoxon test, V = 1,730, N = 62, P < 0.001), which was larger than that for SNUC PAs (unpaired Wilcoxon test, W = 2,023, N = 106 and 62, P < 0.001). Conversion in IL control sites was also higher than in those for other PAs (W = 1,962, N = 106 and 62, P < 0.001).
When analysed at the network level (using area-weighted medians, Figure 2b), Cerrado SNUC PAs also had a positive absolute effect (P < 0.001, 95% CI [0.7, 2.99]), and so did SPs (P < 0.001, 95% CI [1.22, 3.85]). The network-wide effect of MUs showed a positive trend (P = 0.082, 95% CI [−0.06, 3.08]), due to the great contribution of large MUs with relatively high conversion. Overall, there was no difference between SPs and MUs in their network-wide absolute effect (P = 0.203, 95% CI [−0.15, 3.83]), or between conversion in their control sites (P = 0.256, 95% CI [−0.78, 1.31]). ILs also had a positive absolute network effect (P< 0.001, 95% CI [2.2, 4.87]), which was larger than that for SNUC PAs (P = 0.001, 95% CI [0.45, 3.71]). Conversion in their control sites was again higher than in those for other PAs (P = 0.04, 95% CI [0.02, 3.03]).
Relative effect of PAs
Examined individually, SNUC PAs in both categories and ILs had a positive relative effect (Figure 2c, Wilcoxon test, all SNUC PAs, V = 3,865, N = 91, P < 0.001; SPs, V = 1,431, N = 53, P < 0.001; MUs, V = 591, N = 44, P = 0.001; and ILs, V = 1,745, N = 59, P < 0.001). This effect was again lower for MUs than for SPs (unpaired Wilcoxon test, W = 1,563, N = 53 and 38, P < 0.001), but it was similar for SNUC PAs and ILs (unpaired Wilcoxon test, W = 2,568.5, N = 91 and 59, P = 0.625). Note that some sample sizes are reduced in these analyses as it is not possible to calculate the relative effect of PAs that have zero conversion in control sites.
At the network level (using area-weighted medians), SNUC PAs overall also had a positive relative effect (P < 0.001, 95% CI [84.62, 100.00], Figure 2d), and so did SPs (P < 0.001, 95% CI [99.95, 100.00]), and ILs (P < 0.001, 95% CI [93.78, 100.00]). The network-wide effect of MUs showed a positive trend (P = 0.100, 95% CI [−16.50, 100.00]), due to the contribution of large MUs with low effect, and it was lower than the network-wide effect of SPs (P < 0.001, 95% CI [19.2, 121.45]). The network-wide relative effects of ILs and SNUC PAs were similar (P = 0.329, 99% CI [−2.32, 66.65]).
When we checked the effect of sampling intensity on our results, we found that repeating all the analyses above (both for absolute and relative effects) using the cohort of 138 PAs and ILs with at least 50% of successful matches and 20 matched pairs did not produce significantly different results.
Pooled relative effect of PAs
The pooled relative network-wide effect estimates are illustrated in Figure 2d. These estimates differ somewhat from the network-wide area-weighted medians, as the latter preserves the information of which controls belong to which PA (whilst the pooled metric ignores such pairings). The pooled estimates of relative effectiveness of Cerrado SNUC PAs and ILs (Figure 3) were broadly similar to those seen in comparable studies for tropical forests. However, when SPs and MUs were split, SPs tended to have larger effects than those reported in other regions, while the effects of MUs tended to be smaller.
Conversion in buffers vs. control areas
The 10 km buffers surrounding PAs did not appear to experience higher conversion than did their matched unprotected sites during the study period (3.0% vs. 3.5%), showing no evidence of local-scale leakage.
Our findings are consistent with those of recent matching studies in tropical forests (Andam et al. 2008; Gaveau et al. 2009; Nelson & Chomitz 2011; Nolte et al. 2013; Pfaff et al. 2013) and a global analysis (Joppa & Pfaff 2011), all of which found lower deforestation inside PAs than outside. This is the first detailed evidence based on matching methods that PAs also reduce habitat conversion in a tropical savanna biome, and it supports the view that they play an important role in Cerrado conservation despite varying effectiveness across PAs.
Such variation is often a result of differences in their location, as this will influence the baseline threat to which they are compared (Pfaff et al. 2009). In general agreement with previous findings in the Amazon (Nelson & Chomitz 2011; Nepstad et al. 2006; Nolte et al. 2013), Cerrado ILs were located in areas with baseline levels of threat higher than those for SPs and MUs, which provided the scope for greater absolute effect in terms of avoided conversion. When comparing SPs and MUs, previous studies in Latin American tropical forests (Nelson & Chomitz 2011; Pfaff et al. 2013) found that PAs under greater land-use restrictions were located in less accessible lands with lower threat, and therefore had lower effects, although the opposite was found for PAs created since 2005 in the Amazon (Nolte et al. 2013).
In contrast, SPs in Cerrado were located in areas with baseline levels of threat similar to those for MUs, but were generally more effective. Since their baselines were similar, this may be expected as MUs are under less restrictive land-use rules (MMA 2003). This is particularly true in Cerrado, as most MUs are classed as environmental protected areas (locally termed APAs), which broadly allow more land-use change than other MU categories, and are commonly used for buffering stricter reserves such as National Parks (Rylands & Brandon 2005). Given their different objectives, it may be argued that such MUs should not be compared directly with other PAs in terms of their effectiveness in slowing conversion, but should be assessed using different scales more appropriate to their expected outcomes.
Looking at the relative effect reduces the influence of baseline differences and may highlight other factors that contribute to variation in PA effectiveness (such as aspects of management and enforcement, Bruner et al. 2001; Vanclay 2001). Interestingly, Cerrado PAs were broadly similar in relative effectiveness to the overall average for forest PAs in Latin America, even though the estimates are based on different measures of habitat loss (deforestation, fire incidence, or conversion). Although MUs in the Cerrado have lower relative effectiveness than those in other regions in Latin America, the MU categories involved are different and may not be comparable (see earlier, and Nolte et al. 2013).
Based on our findings, conversion avoided by Cerrado PAs was not offset by increased conversion in their immediate surroundings. This apparent absence of local leakage is generally consistent with the findings of other studies using matching methods and a similar 10 km buffer area (Andam et al. 2008; Gaveau et al. 2009). However, various studies suggest that leakage can involve complex mechanisms with both positive and negative effects, varying with location and scale (as extensively reviewed in Pfaff & Robalino 2012); we were unable to address these issues in this study. Two other caveats should also be considered. First, we do not evaluate how far the extent, location, and connectivity of Cerrado PAs adequately represent the biome's important habitats (Silva et al. 2006) and threatened species (Nóbrega & De Marco 2011; Rodrigues et al. 2004). Also, we did not explore other threats such as habitat degradation (e.g., by frequent fires), fragmentation, or spread of invasive species—all of which are pervasive pressures in Cerrado (Klink & Machado 2005).
Despite these limitations, our results fill an important knowledge gap, particularly in the context of conservation efforts planned for Cerrado (MMA 2011). Further research incorporating a broader suite of threats would be an important additional step toward the evaluation of PA systems in Cerrado and beyond.
We thank K. Andam, K. Chomitz, P. Ferraro, and A. Nelson for contributing with data for deriving the relative effect of PAs in their studies, and three anonymous referees for their useful comments. We are also grateful for the databases compiled and made freely available by SIAD-Cerrado, the Brazilian Ministry of Environment, and the Brazilian Institute of Geography and Statistics. This project was supported by Arcadia.