Engaging Stakeholders: Assessing Accuracy of Participatory Mapping of Land Cover in Panama

Full and effective participation of indigenous peoples and local communities, and high accuracy estimates are two current requirements for the purposes of monitoring forests at international level. We produced two land cover maps, both of which were based on digital image processing (decision trees) using Rapideye imagery, and a land cover participatory map, for indigenous territories of eastern Panama. Accuracy of the three maps was evaluated using field data. Classification that was based on participatory mapping gave best overall accuracy of 83.7% (κ = 0.783), followed by the decision tree that included textural variables (DT2: overall accuracy of 79.9%, κ = 0.757). We have demonstrated for the first time that local knowledge can improve land cover classification and facilitate the identification of forest degradation. The plea of the UNFCC for the full and effective participation of local and indigenous people could, therefore, improve the accuracy of monitoring.


Introduction
Reducing Emissions from Deforestation and Forest Degradation in developing countries (REDD+) is a mitigation mechanism that has now been agreed upon under the United Nations Framework Convention on Climate Change (UNFCCC). For the first time, developing countries might be compensated for their efforts in either reducing carbon dioxide emissions from the forestry sector or increasing forest carbon stocks. The integrity of such forest-based carbon-trading schemes will strongly depend upon the accuracy/precision of forest measuring/monitoring systems (Herold & Skutsch 2009). In the context of REDD+, the accuracy of actual carbon stock change estimates will be especially important for countries that are interested in claiming credits for their efforts in reducing deforestation.
The identification and delineation of land cover are the first critical tasks requested for evaluating and monitoring changes in forest carbon stocks. While there are multiple approaches to classifying land cover, the mapping of land cover categories is never considered to be a perfect representation of the landscape (Lowry et al. 2007). Despite the evolution of remote sensing technologies over the last few decades, interpretation is still plagued by difficulties when the time comes to identifying specific land cover categories, in particular with medium to low resolution satellite imagery. For instance, Pelletier et al. (2011) identified the lack of understanding of fallow land dynamics as a significant source of uncertainty for Panama, given that fallows occupy a substantial fraction of the national territory. In many parts of the world, fallows and other successional stages of forest lands can cover large areas. Thus, methods for improving the classification success of areas that are in various forest successional stages, together with logging activities having reduced impacts, would contribute to reducing the uncertainties surrounding changes in forest carbon stocks. Classification of primary forest, selectively logged forest, and degraded forest is likewise prone to error (Herold et al. 2011;Bucki et al. 2012;GOFC-GOLD 2013).
The UNFCCC has repeatedly called for the full and effective participation of indigenous peoples and local communities in carbon measuring and monitoring, as described in paragraph 3 of Decision 4/CP.15, paragraph 72 of Decision 1/CP.16, paragraph 2 of Appendix I, and paragraph 71(d). The specific guidelines on how to engage indigenous peoples and local communities has been left to the discretion of the individual countries that are implementing REDD+ Skutsch & Trines 2011). Alternatively, it has been also highlighted that the evaluation of accuracy of locally based forest monitoring is a key task for monitoring REDD+ systems, for instance, Danielsen et al. (2011) have emphasized that "further quantitative assessments of the ability of locally based forest monitoring methods to detect changes in forest condition are needed." This article examines the extent to which local knowledge, through participatory mapping, could improve the accuracy of land cover classification. Participatory mapping is a powerful tool that allows the inclusion of key local knowledge about location, land cover, and land use history of the landscape and serves to help communities make land use decisions (NOAA 2009;Coomes et al. 2011;Danielsen et al. 2013). During the past decade, participatory mapping has become widely popular in both developing and developed countries (Corbett 2009). While there are several variants, ranging from lowresource and low-cost to high-resource and high-cost approaches, the selection of the participatory method will depend on how the final product will be utilized, the expected impact of the tools to be utilized, the expected accuracy of the final product, and the resources available (Chambers 2006;Corbett 2009). Different forms of technological support have been utilized in its implementation, including satellite images, aerial photographs, global positioning systems (GPS), and geographic information systems (GIS), among others (Corbett 2009). We used here a combination of local knowledge, and training of landowners in image interpretation and use of technological tools (satellite images and GPS devices) as a way to increase accuracy of land cover classification. We chose to use scale maps drawn on existing, scaled satellite maps as the selected method in this research.
In Tanzania and Nepal, Skutsch et al. (2011) have demonstrated that participatory carbon measurements can be reliable, given that they observed no more than a 5% difference in the estimates of mean carbon stocks between professionals and the community. We are not aware of similar evaluations for participatory mapping that employs digital image processing techniques. Here, we compare the accuracy of two land cover maps in this article: one that uses participatory methods and another that uses a digital image classification, which is based upon a decision tree. This research is part of a broader participatory initiative started in 2010, in which indigenous peoples of Panama were both a project partner and proponent (Appendix S1). Our objective was to determine if locally produced maps could provide reliable information in the context of REDD+. The study took place in the complex landscape of the Emberá people in the Bayano area of eastern Panama, where multiple successional forest stages and forest structures are present.

Study area
The study took place in indigenous territories that are located in the Province of Panama, close to the Pan-American Highway and Bayano Lake (78°30'-78°49' W, 8°54'-9°05' N). These territories are under the authority of the General Congress of the Collective Lands of Alto Bayano (CLAB), and include the collective lands of Ipeti (3285 ha), Piriati (3869 ha), and Majé Emberá-Drua (18920 ha) ( Figure 1). Elevations in the CLAB territories range from 60 to 1080 m above sea level, with the highest areas in Majé. The territories are covered by "tropical moist" and "premontane wet" forest, according to the Holdridge Life Zone system (Smithsonian Tropical Research Institute 2013). Average annual precipitation ranges between 2000 to 3000 mm at high altitude. Annual temperature averages 26°C in the lowlands and 22°C in highlands, with a pronounced dry season from December to April (Autoridad Nacional del Ambiente 2010). CLAB is inhabited by ß1,500 Emberá people, who constitute one of three indigenous groups in eastern Panama that migrated from Colombia to the Bayano region in the 1950s. Indigenous territories in Panama are constituted as legally recognized areas (comarcas) and as areas being claimed by indigenous groups who wish to obtain legal recognition (claimed lands). Claimed lands in Panama are based on customary ownership. CLAB correspond to a claimed land and is currently in the process of legalization under the country's Law 72 (Gaceta Oficial 2008) and Decree 223 (Gaceta Oficial 2010). Due to the collective nature of tenure in the CLAB, families do not have individual legal rights to their parcels; however, land use decisions are taken at the household level. Primary economic activities include subsistence cultivation, cattle ranching, day laboring, and handicraft production ) (Additional information in Appendix S2).

Figure 1
Bayano area showing the three indigenous territories that were included in the study. Data sets that were used for validation, including randomly selected points (circles) and forest inventory plots (squares), are also shown in the map.

Land cover classification
The mapping was based on two preprocessed 5-meter resolution multi-spectral Rapideye R images (Appendix S3) that were taken on February 5, 2012, where terrain images containing clouds and cloud shadows were excluded. This yielded total areas of 2,685 ha, 14,723 ha, and 3,083 ha, respectively, for Ipeti, Maje, and Piriati. These net areas were used as a reference for all subsequent analyses. Our methodology evaluates the accuracy of participatory mapping in terms of land cover classification in relation to satellite image classification that was based on a decision tree.

Land cover participatory mapping
The first step in this project was to obtain authorization to determine the land cover in the CLAB in a participatory manner. Therefore, we held meetings with local traditional indigenous authorities of each local congress of the CLAB to explain the purpose and objectives of the mapping and to request the necessary authorization. During these meetings, participants agreed to use a combination of local knowledge, image interpretation, and satellite imagery to increase the accuracy of the land cover maps in the CLAB. After receiving a written authorization for every local congress of the authorities of the CLAB, we also informed the Coordinadora Nacional de los Pueblos Indígenas de Panamá (COONAPIP, National Assembly of Indigenous Chiefs of Panama). We then carried out participatory land cover workshops in Ipeti, Piriati, and Maje in February 2012. The workshops were jointly coordinated with the local traditional authorities. A total of 95 participants attended the workshops (27 in Ipeti, 45 Piriati, and 22 in Maje). During the workshops, a printed Rapideye R satellite image of the territory, including borders and other geographic landmarks such as villages, roads, and rivers, served as a base map.
We also brought a blank map where the satellite image had been extracted, but the aforementioned land-marks were included. At the onset of the workshop, the attendees (including local traditional authorities and landowners) discussed how to reach a consensus for the land cover classes in their territories. For all territories, the land cover classes that were adopted included primary forests, intervened forests (logged forests), tall fallows, short fallows, plantations, pastures, cultivation, bare soil, communities (villages), and water bodies. During the second part of the workshop, landowners were invited to identify their parcels and they assign the corresponding land cover categories that had been previously defined for that portion of territory. The satellite image was used to guide the classification; meanwhile, the blank map was used for drawing the interpreted areas of the satellite image. To complete the mapping exercise, we visited the landowners who were unable to attend the workshops. The exercise was explained and they then classified their plots using the same classes that had been adopted during the workshop. In addition to the workshop participants, over 80 landowners were visited and consulted at this stage: 48 in Ipeti, 21 in Arimae, and 14 in Maje. The final map was presented and validated by the attendants at a later meeting (Appendix S4).

Land cover mapping using remote sensing
Decision tree classification (DT) was used to create a second set of land cover maps that were based on the spectral and textural attributes of the Rapideye images. DT is a hierarchical method that involves recursive partitioning of a training data set, which is separated into increasingly homogeneous subsets on the basis of tests that are applied to one or more of the feature values or attributes (Pal and Mather 2003) (Appendix S5). In this method, binary splits are performed according to maximum likelihood tests that are based on one (univariate) or several predictor variables (multivariate) or, in the case of other methods, are based on formal t-, F-, or chi-square tests. DT belongs to the larger family of machine-learning approaches that include vector support machines, artificial neural networks, classification, and regression tree analysis.
Two variants of the DT method were employed in our analyses, with one correcting for reflectance values of the five bands in the Rapideye images (DT1) and the other (DT2) adding eight textural features to the input data (Appendix S6). Training areas (subset of the data) were selected in the Rapideye images using ENVI-5.0 R software (https://www.exelisvis.com/envi-5/) for the same land cover categories that had been defined in the participatory maps to make the classifications comparable. All training areas were selected from "pure" spectral and homogenous areas so as to choose the most appropriate categories and, thereby improve classification (Lillesand & Kiefer 2009). The training areas are based on a priori knowledge of the region, including field knowledge and scientific sources. Training areas represented 4% of the total study area.
Training areas were also used to define threshold values for the nodes and branches of the decision tree. Decision tree classification was performed using the Waikato Environment for Knowledge Analysis (WEKA; http://www.cs.waikato.ac.nz/ßml/weka/), which is an open-source data mining software suite that includes machine-learning algorithms for data mining tasks. The J48 decision tree algorithm of Quinlan (1993), which is available within WEKA, was used for training the Rapideye image data set (Appendix S7). The resulting rules that were generated were implemented on the Rapideye satellite image data for classification. This work was also carried out in ENVI 5.0 software.

Ground truthing and accuracy assessments
As defined by Foody (2002), classification accuracy is the degree of "correctness" of a map or classification. Field assessment of accuracy was carried out using two sets of validation data. The first data set consists of 56 randomly selected GPS points in the areas of Ipeti (20) and Piriati (36). These points were selected in ArcGIS 10.1 R and visited on the ground in July 2013 by trained indigenous technicians, who identified the associated land cover. The second set of independent data consisted of 38 forest carbon plots that had been measured in the areas of Maje (16) and Ipeti (22) from July to August 2012 (Figure 1). These 25-m-radius plots were measured by local indigenous technicians, who had been trained in forest mensuration, and which were established in areas that covered a full range of elevational and human intervention gradients, where land cover of these points had also been registered (Appendix S8). The accuracy classification of the three final maps was then evaluated using a confusion matrix (Congalton & Green 2002), which estimates a classification error indicating a discrepancy between the situation that is depicted on the map and the reality that was observed in the field (Foody 2002).
Kappa (κ) inter-rater reliability assessments that compared results of the three classifications were carried out (Cohen 1960), where a theoretical maximum value of 1 represents complete agreement between a given classification method and the field data. In addition, two tests were used to identify significant differences among methods, i.e., Cochran's Q (among all classification methods) and McNemar tests (McNemar 1947) (Appendix S9).

Results and discussion
Our analysis revealed substantial variation in the classification of land cover among the methods that were considered ( Figure 2). Participatory mapping maximizes the areas of forests (4,878 ha) and of grasslands (4,667 ha)   in which intervention had taken place, while DT1 maximizes primary forests (11,771 ha) (Figure 3). DT1 further yields the lowest coverage of tall fallows (981 ha) of all methods. Significant differences were found in the number of correct cover categories that were produced by the three classification methods (Cochran's Q test: χ 2 (2) = 20.26, P < 0.05), while pairwise comparisons using Mc-Nemar's test revealed significant differences between DT1 and DT2 (P < 0.001), and between DT1 and the participatory mapping (P < 0.005). Participatory mapping had the greatest overall accuracy (83.7%, κ = 0.783), followed by DT2 (79.9%, κ = 0.757).
Participatory mapping accuracy varies from 20% for bare soils to 100% for primary forests, grasslands, and waters ( Table 1). The bare soil category has the lowest accuracy, given that this category was apparently confounded with grasslands and short fallows. In forested areas,participatory classification was respectively 100% and 97% for primary and intervened forests, which was significantly higher than classification that was based on digital image processing. The two classification methods that were based on decision trees apparently overestimated primary forests while underestimating forests in which there had been intervention. Classification that was based only on remote sensing, however, had high accuracy for tall and short fallows (Table 1).
Indigenous peoples who participated in this study demonstrated a high degree of knowledge regarding the 436 Table 1 Accuracies (%) of predicting land cover categories and classification methods, relative to field observations. Classification methods include: participatory = participatory method, DT1 = decision tree using regular Rapideye bands, DT2 = DT1 plus textural features land cover and historical land use of their territories, which we validated on the ground. We assume that the higher accuracy of the participatory approach-in identifying primary and intervened forest-is a result of this local knowledge that allows increased detection of land cover and forest degradation. A similar observation was made by Danielsen et al. (2013) for identifying forest strata in Indonesia, China, Laos, and Vietnam. In tropical countries where slash-and-burn agriculture is practiced, the development of vegetation from recently cleared forests to short fallows, then to tall fallows and more advanced second-growth forests makes the implementation of land cover classifications a challenging task (Pelletier et al. 2011). This is particularly relevant in areas where indigenous forms of agriculture produce a complex landscape mosaic of grasslands and annual crops that are interspersed with areas in different regrowth stages, as well as older forests in more inaccessible areas . Such a complex and highly dynamic land cover makes it difficult to achieve high accuracy solely through digital image classification that is based on decision trees. Our results show that in digital image processing, intervened forests are easily confounded with primary forest and that local knowledge could more efficiently contribute to differentiating these otherwise relatively similar forest types. According to GOFC-GOLD (2013), digital image processing is of limited use in identifying logged areas and human interventions that result in forest degradation. With gradual losses of biomass and the creation of small clearings in the canopy, forest degradation cannot be effectively measured using standard optical remote sensing methods, since their resolution is too coarse or the effects of logging too well-hidden to be detected either visually or by computer analysis (DeFries et al. 2007). The complexity that is involved in identifying more subtle changes in vegetation has triggered the identification of proxies (i.e., road proximity) for determining these potential impacts on the forest and simplifying the identification of forest degradation (Bucki et al. 2012). We have demonstrated for the first time that local knowledge can improve land cover classification and facilitate the identification of forest degradation. The plea of the UNFCC for the full and effective participation of local and indigenous people could indeed improve the accuracy of monitoring.
One caveat needs to be kept in mind. Our results show that the accuracy of participatory maps varied according to the three territories in the CLAB, the Ipetí map had the highest level of accuracy (0.925, κ = 0.87) and the lowest one (0.67, κ = 0.58) in Piriati. We have identified two main reasons behind these differences. First, the areas with lower accuracies in the three territories have a greater extent of grassland and short fallows. Most landowners in these territories labeled both bare soils and short fallows as grasslands, suggesting that landowners in these areas tend to classify the parcels according to their land use instead of their land cover. Second, high accuracy in the Ipeti area is not surprising because many leaders and local dwellers have had an extensive experience in working with other land cover classification and carbon projects for more than 10 years (Kirby & Potvin 2007;Potvin et al. 2007). While Danielsen et al. (2013) argued that even local stakeholders with limited education can measure forests with acceptable standards, the differences that they observed among villages suggest that prior training can help improve the detailed spatial knowledge of territories. If participatory mapping is to be successfully incorporated into the REDD+ tool-box, we propose that the preparation and training of local dwellers in interpreting basic aspects of aerial or satellite images becomes a fundamental step before any participatory mapping exercise takes place (Rambaldi 2010). In doing this, the trainers should avoid complex aspects and terminologies of conventional scientific methods, and keep the training stage as simple as possible (Fry 2011).
Finally, we must concur with Danielsen et al. (2013) in that involvement of local communities could improve the capacity of many developing countries for monitoring forest emissions at a reasonable cost and within a short time-frame. It has shown that local knowledge is a valid option that complements satellite imagery, but participatory mapping could also be helpful in resolving issues that are related to cloudiness, a pervasive problem for many countries in the humid tropics. Complementarity between locally based data and remote sensing data can also be valuable in identifying land cover areas with similar spectral properties (training areas) for other indigenous territories in the country, or for areas that are not under the control of local communities but where national governments can have satellite coverage to generate land cover maps. Meanwhile, different communities can propose different land cover classifications making it difficult to manage in a REDD+ national context like Measuring, Reporting and Verifying (MRV) systems, such local classification systems could be translated to a more general or standardized system to be utilized in a national context. Given the importance of indigenous people as forest custodians in Panama and many other Latin American countries (Vergara-Asenjo & Potvin 2014), engaging them in forest monitoring under REDD+ appears to be a win-win opportunity for improving mapping accuracy, while also unlocking the sometimes complex relationship between indigenous people and national REDD+ strategies. In moving away from the fear of REDD+ (Potvin & Mateo-Vega 2013), indigenous peoples and local community participation in forest carbon assessment or in national forest inventories could establish a new starting point that is based on real collaboration and mutual benefits.