Assessing the impacts of uncertainty in climate‐change vulnerability assessments

The trait‐based vulnerability assessment (TVA) uses Boolean rules to assess species sensitivity, adaptive capacity and exposure to climate change to identify those that are climate‐change vulnerable. The protocol is being increasingly used to assess climate‐change impacts to a diversity of taxa, as it requires fewer data compared to niche and mechanistic models. However, uncertainty in TVA results remains unevaluated. We present the first quantitative investigation of the impacts of uncertainty on TVA, using global freshwater crayfish (574 species) as a representative data‐poor taxon.


| INTRODUC TI ON
Climate change is acknowledged as a major threat to biodiversity, and studies show that species have already begun to respond to changing conditions (IPCC, 2014). Several approaches have been developed to assess species' vulnerability to climate change such as correlative niche models (Guisan & Thuiller, 2005), mechanistic models (Kearney & Porter, 2009), trait-based vulnerability assessment (TVA) protocols (Foden et al., 2013) and combinations of these models (Rougier et al., 2015;Willis et al., 2015). The outputs of correlative niche models (such as projection of range or population decline) have been further integrated with IUCN Red List criteria (Akcakaya, Butchart, Mace, Stuart, & Hilton-Taylor, 2006) to infer species' extinction risk due to climate change. However, each of these approaches presents a number of shortfalls (Trull, Böhm, & Carr, 2018;Winter et al., 2016).
Correlative niche models (i.e., species distribution models) use information on species' current distributions and observed climatic conditions to estimate species' climatic niche. Projections from global circulation models are then used to estimate the location of future climatically suitable areas (Guisan & Thuiller, 2005). Although correlative niche models have seen wide applications for predicting species' range changes under future climates, the approach cannot be used where insufficient occurrence data are available (i.e., for most lesser-known taxa on earth; Westgate, Barton, Lane, & Lindenmayer, 2014). Correlative models have also been criticized for being too simplistic and for failing to account for population dynamics, biotic interactions and species' dispersal abilities (Kearney et al., 2008). Mechanistic models account for population dynamics (e.g., vital rates and demography), but often not for non-climatic factors such as dispersal barriers and interspecific interactions (Rougier et al., 2015). The data requirements of mechanistic models are also too high for many species, precluding their extensive use (Kearney et al., 2008).
The IUCN Red List of Threatened Species™ can account for threats occurring over long timeframes, such as climate change (Pearson et al., 2014) and habitat destruction. However, the IUCN Red List deals primarily with symptoms of risk rather than their causes (IUCN, 2018) and often uses unstructured expert elicitation to identify threats (http://www.iucnr edlist.org/techn ical-docum ents/class ifica tion-schem es/threa ts-class ifica tion-scheme). A number of authors have pointed out this shortcoming, including a lack of consistency in listing threats among different groups of threatened species (Bland, 2017;Böhm, Williams, et al., 2016;Trull et al., 2018).
For these reasons, a specific protocol was needed to assess risks from climate change, leading to the development of the trait-based protocol (Foden et al., 2013).
The trait-based vulnerability assessment (TVA) (Foden et al., 2013) protocol has been developed to fill the need for a more generalized approach that can be used for large numbers of species across taxa. TVA is useful for timely conservation prioritization for species that lack data to directly model future range changes and where detailed information on every species in a taxon is difficult or prohibitively expensive to collect (Böhm, Cook, et al., 2016).
TVA is based on the following conceptual model (Figure 1; Foden et al., 2013): if a species cannot live in its habitat (sensitivity) due to climate change, is unable to adapt through dispersal or evolution (adaptive capacity) and is exposed to the changing climate (exposure), the species is considered vulnerable to climate change. In the TVA approach, traits are selected to represent species' sensitivity (e.g., ecological specialization and interspecific interactions), adaptive capacity (dispersal and reproductive ability) and exposure (e.g., to sea-level rise, temperature and precipitation changes) to climate change. These traits are scored as high or low using Boolean coding (i.e., yes or no). For categorical traits, this is done by assigning trait categories to Boolean outcomes, while for continuous traits, arbitrary thresholds (e.g., 25% lowest/highest data values) are used. The number of traits can vary in each dimension of TVA. For example, in the global TVA assessment for reptiles, 10 traits were used for K E Y W O R D S adaptive capacity, climate models, exposure, freshwater crayfish, sensitivity, trait-based vulnerability assessment F I G U R E 1 Conceptual model of trait-based protocol for assessing species' vulnerability to climate-change (after Foden et al., 2013). For each dimension (sensitivity, adaptive capacity and exposure), traits are selected and scored based on known characteristic of species. If a species scored high for any trait within a dimension (e.g., sensitivity, adaptive capacity or exposure), it is marked high for that dimension (i.e., highly sensitive, low adaptive capacity or highly exposed to climate change, respectively). Species that are highly sensitive, have low adaptive capacity and are highly exposed are considered as climate-change vulnerable sensitivity, three traits for adaptive capacity and five traits for exposure (Böhm, Cook, et al., 2016;). Species with a high score on any trait within a dimension (e.g., sensitivity, adaptive capacity or exposure, respectively) are marked high for that dimension (i.e., highly sensitive, low adaptive capacity or highly exposed to climate change, respectively). Species that score high in all three dimensions are considered climate-change vulnerable.
TVA has been increasingly used in a diversity of taxa (Böhm, Cook, et al., 2016;Carr, Hughes, & Foden, 2014;Foden et al., 2013) but the approach remains unevaluated against empirical data based on species' observed responses to climate change (as for many other methods; Foden & Young, 2016). Therefore, it is important to develop methods to assess the impacts of different types of uncertainty on TVA and to assess the reliability and robustness of the approach.
Most trait-based assessments suffer from knowledge gaps within at least some of the traits assessed (e.g., 10%-79% missing trait data in reptiles; Böhm, Cook, et al., 2016). Species with missing trait values can be systematically scored high (i.e., a precautionary, risk-averse approach) or low (i.e., an evidentiary, risk-seeking approach), which may influence the TVA outcome. For example, Hossain et al. (2018) showed that the proportion of climate-change vulnerable species in freshwater crayfish ranged from 15% to 43% depending on whether species with missing trait values were scored as low or high, respectively.
For traits with continuous values, there often is no ecological threshold that could be used to separate high and low scores. The common practice in TVA is to assign high scores to all species within a certain high/low percentage with highest/lowest values for that trait (Foden & Young, 2016). For example, the 25% threshold is often used in TVA (Foden et al., 2013), despite no theoretical underpinning justifying this threshold.
Global climate models robustly reproduce many of the observed large-scale changes in climate in the recent past; however, structural and stochastic uncertainty across different modelling methods and potential emission pathways means that future climatic conditions cannot be predicted with precision (Wang, Zhang, Lee, Wu, & Mechoso, 2014). Buisson, Thuiller, Casajus, Lek, and Grenouillet (2010) showed that global climate models produce highly variable results when projecting species' range changes with correlative niche models (variation ranged from 12.6% in 2020 to 19.2% in 2080). The impacts of model variability and alternative futures on TVA results have thus far not been explored.
In this study, we assessed the impacts of uncertainty in TVA using global freshwater crayfish as a case study. We used this taxon because crayfish are widely distributed, economically important, and many species are currently vulnerable to extinction, making them the seventh most threatened group in the IUCN Red List (IUCN, 2018;Richman et al., 2015). Many species in this group also do not have sufficient distribution data to model future range changes (Troja & McManamay, 2016), therefore making them a suitable example taxon for TVA.
We investigated how the selection of species' traits, trait thresholds and global climate models affects the outcomes of TVA. More specifically, we identified the impact of threshold selection, as well as the contribution of traits and data gaps in the assessment. We asked three questions: (a) which traits contribute most to species' vulnerability to climate change across TVA dimensions and how the relative contribution of traits correspond to data gaps in those traits? (b) How sensitive are TVA outcomes to arbitrary thresholds used for continuous traits? and (c) How does estimated vulnerabilities to climate change differ among climate models? In summary, we conducted the first uncertainty assessment of TVA using freshwater crayfish as a case study to provide recommendations to improve the robustness and reliability of TVA applications across taxa and geographical scales.
These traits and categories were selected based on the literature and ecological assumptions on species' responses to climate change (see Hossain et al., 2018, for details on trait selection). Categorical traits were scored as high or low depending on whether these met specified ecological conditions. Specifically, species with the following characteristics were scored high for the corresponding traits: dependence on one habitat type or any dependence on microhabitat (e.g., burrows or caves); dependence on high dissolved oxygen concentrations; rarity; dependence on habitats susceptible to sea-level change (e.g., mangroves or coastal lakes); highly fragmented ranges; extrinsic barriers to dispersal; and declining population trends.
Our quantitative traits included narrow tolerance to temperature and precipitation changes (based on historic estimates of climatic variation within distributions); interspecific dependency (based on body size, as smaller species are more susceptible to invasions by larger crayfish); exposure to changes in temperature and precipitation (based on global climate-change projections); range restriction; and clutch size (used as a proxy for reproductive output) (Table 1). For species' narrow tolerance to temperature and precipitation changes, we measured the average absolute deviation (AAD) from contemporary (years 1961-1990, referred to as 1975) mean temperature and precipitation within their current range. Exposure to future climate change was measured as the expected change in mean temperature and rainfall, as well as change in temperature and rainfall variability, within species ranges by two future time points: 2050 (2041-2060) and 2070 (2061-2080).
For changes in mean temperature and temperature variability, we used the absolute difference in means and the absolute difference in AAD across species' ranges and the two timeframes, respectively. For precipitation, these were measured as the absolute relative change in mean precipitation and AAD across the species' ranges, respectively. For all quantitative traits, we first used the standard trait-based approach (treatment one) giving species with the 25% lowest/highest values (depending on the trait) a high score (Table 1).
In treatment 1, temperature and precipitation data were col- Next, we explored the sensitivity of TVA outcomes to arbitrary threshold selection for quantitative traits (treatment 2). We created a null hypothesis for species to be identified as vulnerable to climate change solely as a result of using the 25% (lowest/highest) threshold for quantitative traits. To do this, instead of scoring high 25% of the species with the lowest/highest value in each quantitative trait, we selected these 25% of species randomly and independently. This procedure was repeated 20,000 times to create a TA B L E 1 List of traits included in this study null distribution of the number of species that would be expected to be identified as vulnerable by chance. We then compared this distribution with the result obtained from the standard trait-based approach. Because the proportion of species randomly selected for each quantitative trait on each iteration is always the same (25%), deviation from the null hypothesis means that the results from treatment one are not explained by chance alone and that the default approach (using 25% threshold) may be detecting an ecological signal.
In treatment 3, to explore variation introduced by different climate models, we repeated the standard TVA but instead of using the mean of four global climate models we used (

| RE SULTS
We found large variation in the number of species scored high for traits in a dimension with some traits showing dominant contribution than others (Figure 2). Of the 17 traits included in this study, species were most often scored high due to habitat specialization and least often due to dependence on sea-level inundation habitats.
For nearly half (49%) of the species, restricted range was the only  (Table S1).
Using the standard TVA (treatment one) approach with a 25% threshold for quantitative traits and an ensemble mean projection of four climate models, we identified 87-107 crayfish species to be vul- with only one to two species differences, across emission scenarios and time periods ( Figure S1 and Both ensemble models also identified comparatively fewer species (maximum 114) than most of the individual climate models (maximum 155) (Figure 4a,b).
Geographically, the highest variations in the numbers of vulnerable species across the different climate models were found in the Americas and in Australia, and the lowest in Europe (Figure 5a

| D ISCUSS I ON
Assessing species' vulnerability to climate change is the first operational step towards safeguarding biodiversity from the potentially adverse impacts of climate change. We present the first quantitative uncertainty assessment of a trait-based climate-change vulnerability analysis (TVA), using global crayfish (574 species) as a case study. We found that uncertainty in TVA can arise from multiple sources, including the selection of traits, and the choice of global climate models to infer species' exposure to climate change. Our results highlight the need for both establishing a systematic protocol for trait and threshold selection, and a more rigorous reporting of result uncertainty when conducting TVAs.
Data were missing for many traits in freshwater crayfish ( Figure 2), a typical problem in TVAs for data-poor taxa. The common approach of assigning systematically either high or low scores to all species with missing trait values gives a sense of the range of uncertainty in the assessment results (Hossain et al., 2018); however, a closer examination of the data gaps can reveal aspects that are more informative for the interpretation of results and in guiding future research efforts, as some traits may contribute more than others to the results. For freshwater crayfish, of the four traits included in the adaptive capacity dimension, range restriction alone contributed to high score to 49% of species included in the assessment, thereby listing them as having low adaptive capacity to climate change ( Figure 2b). Of particular interest are traits for which many species receive high scores, but which also have notable data gaps.
For example, in our crayfish study clutch size (a proxy for reproductive output) was available for only 35.5% of the species (Figure 2b), reflecting general deficiencies in life-history information for this taxon (Hossain et al., 2018;Moore, DiStefano, & Larson, 2013). At the same time, clutch size was the only trait underpinning classification as having low adaptive capacity for many species. As TVAs often include traits for which further efforts to collect data are desirable, understanding their relative contributions to listing species as climate vulnerable may guide the allocation of finite resource budgets (Joseph, Maloney, & Possingham, 2009).
Decisions on which traits to include and how to assign them to the different dimension are crucial steps in TVA (Foden & Young, 2016). Because of the structure of the TVA protocol, correlations among traits placed in different dimensions should be carefully assessed (Table S1), as correlated traits among dimensions may result in more species being assessed as climate-change vulnerable. Whether this effect is desirable or can be considered an artefact of the chosen traits depends on the objectives of the rule-based protocol (Keith et al., 2015). Given that TVA follows a precautionary approach, correlated traits may be acceptable to be included in different dimensions in some applications of TVA, but specific guidelines should be produced to address this question. The TVA approach also commonly assumes that one high score on any trait in a dimension results in a high overall score for that dimension. Within each dimension, some traits showed dominant contribution followed by diminishing returns as more traits are added in that dimension. It is therefore likely that the more traits are included in each dimension, the more species will be identified as vulnerable to climate change (Burgman, 2015). Although not systematically tested here, our results support this notion as the number of species scored high in each dimension echoed the number of traits used in them.
Previous TVAs have included variable numbers of traits in each dimension (Böhm, Cook, et al., 2016;;Carr et al., 2014;Foden et al., 2013;Luckett, 2009;Ofori, Stow, Baumgartner, & Beaumont, 2017) (see Table S3 for a list of traits and thresholds used among taxonomic groups). Further differences arise from whether a given trait pertains to sensitivity, exposure or adaptive capacity. For example, species with small ranges will be highly exposed to stochastic events resulting from climate change (Murray et al., 2017), but may also suffer from low evolutionary capacity, as well as narrow environmental tolerances. Therefore, species' range size may arguably be included in any of the three dimensions (Dickinson, Orme, Suttle, & Mace, 2014;Garnett & Franklin, 2014). The choice of traits related to climate-change impacts may vary between taxonomic groups, but in general should be consistent within a taxon (Ellison, 2010).
Yet, when assessing sensitivity of reptiles to climate change, Ofori et al. (2017) used sexual maturity (≥5 years as a threshold) to infer generation length, while Böhm, Cook, et al. (2016)) used longevity (>10 years) as a proxy. Though these studies vary in spatial scales and data availability, a lack of consistency between scientific studies for a particular taxon hinders comparisons and may result in stakeholders and policymakers questioning the overall usefulness of outcomes to aid decision-making (Kujala, Burgman, & Moilanen, 2013).
A general consensus for the selection of traits to be included and to which dimension in TVA they contribute to should be reached for each taxon based on expert advice. A list of alternative traits could be provided under each dimension to account for differences in data availability among geographical scales and areas. However, in poorly known groups, a key hindrance may be the limited information available to justify trait selection and associated assumptions about the mechanisms that predispose species to risks from climate change.
We found this to be a key issue in the application of TVA for crayfish (Hossain et al., 2018), and similar problems will undoubtedly arise in applications to many poorly known groups.
The absence of ecological thresholds for quantitative traits is an important issue in TVA and other rule-based protocols . Corals are the only taxon studied so far where a threshold for significant exposure to climate change was based on a value with an explicit ecological interpretation (aragonite saturation; Foden et al., 2013 based on Guinotte, Buddemeier, & Kleypas, 2003). Our estimated number of vulnerable species was significantly higher (55%) than the distribution of values obtained for our null hypothesis ( Figure 3). We also found no strong correlation between traits among dimensions. Together, these findings suggest that our TVA results for crayfish are not merely a product of chance but that, despite the arbitrariness of the current 25% threshold selection, the assessment is capturing genuine ecological effects of climate change on species. Interestingly, our result (87 vulnerable spp. for RCP6.0 in treatment 1) is also similar to the number of crayfish species (91) for which climate change is listed as a threat based on the qualitative assessment of the IUCN Red List (IUCN, 2018), although both the identity and the distribution of identified vulnerable species varied between TVA and the IUCN Red List (Hossain et al., 2018). In the absence of ecological thresholds, TVAs only provide ranks of species' vulnerability to climate change rather than absolute assessments of vulnerability, limiting comparisons among taxa.
The set of species identified as vulnerable to climate change by the 11 climate models stayed relatively consistent for all RCPs in 2050 and 2070 (Table S2). However, there were notable differences in assessment results when different climate models were used, not only in the number of species but also in the identity of the species deemed vulnerable (Figure 4a,b). The projection of RCPs shows that RCP6.0 has lower radiative forcing than RCP4.5 which is consistent till 2060 and becomes higher than RCP4  (Figure 4a,b). Our findings echo those of Döll and Schmied (2012) and Karmalkar and Bradley (2017), who found that the choice of global climate models introduced higher uncertainty to climate-change predictions than the selection of climate scenarios and time periods. In our study, uncertainty associated with global climate models was spatially concentrated in the Americas and Australia; these regions had the largest variation in the number of vulnerable species, and overall, the highest crayfish species richness (Figure 5a,b).
Which of the global climate models best represents future climate-change impacts is unknown. Current recommendations suggest the use of ensemble mean of three or four models to reduce the uncertainty arising from the predictions of individual climate models (Foden & Young, 2016). We found that ensemble means of 11 models still produced different outputs and that both ensemble approaches predominantly resulted in lower numbers of vulnerable species in comparison with individual models (Figure 4a,b). The decrease in the number of vulnerable species is likely explained by the fact that model averaging tends to dampen the influence of extreme values and, overall, reduce expected changes in climate-related mean values and variability (Knutti, Furrer, Tebaldi, Cermak, & Meehl, 2010).
The possibility of opposing changes being cancelled out in the process of model averaging is supported by our finding that individual climate models predominantly identified different pools of species as vulnerable to climate change (Table S2). Although ensemble models are frequently used in climate-change assessments based on correlative models, concerns about their use are being voiced (Knutti et al., 2010). Indeed, whether ensembles act as a method to remove noise, and hence reduce uncertainty in predictions, or instead mediate the inclusion of poorly performing models and mask important details, can be very case specific (Dormann et al., 2018). The use of both ensemble models and carefully chosen single models may be justified (depending on the objective of the TVA), but as climate models clearly introduce large variation in the results, this variation should be made explicit. Assessors using TVA should therefore conduct sensitivity analyses to explore and report levels of variation among climate models.
Many of the uncertainties highlighted in this study are not unique to TVA. Both correlative and mechanistic models are equally susceptible to uncertainties arising from selection between alternative climate models and possible futures (Buisson et al., 2010;Kearney et al., 2008). Indeed, the relative uncertainty in TVA results caused by the different components of climate predictions (climate model, scenario or timeframe) seems to agree with results found with correlative and mechanistic models. Similarly, data gaps and variations arising from the selection of traits could be seen as analogues to those of variable selection and data biases in modelling (Willis et al., 2015). However, whereas several established tools exist for quantifying the impact of such uncertainties in correlative and mechanistic approaches, these are predominantly lacking from the TVA protocol. Nevertheless, the strength of TVA is that it does not depend on detailed species' occurrence information, which is lacking for many poorly known taxa such as invertebrates, preventing comprehensive assessments (e.g. based on statistical modelling) of climate-change impacts on biodiversity. Unlike correlative and mechanistic models, TVA also includes many of the biotic and abiotic traits related to species' vulnerability to climate change. Furthermore, as long as trait selection and scoring are carefully conducted and documented, the TVA provides a structure, transparent and reproducible approach for assessing species' vulnerability to climate change. This complements the current IUCN Red List threat assessments, which are predominantly based on unstructured expert elicitation (http://www. iucnr edlist.org/techn ical-docum ents/class ifica tion-schem es/threa ts-class ifica tion-scheme).
We used tools that allow researchers to explore the different sources of uncertainty in TVA, such as trait and threshold selection and the use of climate-change models. Given the lack of detailed information available for most taxa on earth (Westgate et al., 2014), researchers conducting TVAs should be aware of these uncertainties and conduct sensitivity analyses, including the following: (a) critically assessing assumptions and correlations underpinning the selection of traits in different dimensions; (b) using ecological thresholds for quantitative traits, where possible; and (c) capturing variability among climate-change models. Given that climate change is impacting species, ecosystems and ecological processes (Wiens, 2016), assessing species' vulnerability to climate change with appropriate tools and consistent frameworks is necessary to prioritize species for conservation and management.