Language inclusion in ecological systematic reviews and maps: Barriers and perspectives

Systematic reviews and maps are considered a reliable form of research evidence, but often neglect non‐English‐language literature, which can be a source of important evidence. To understand the barriers that might limit authors' ability or intent to find and include non‐English‐language literature, we assessed factors that may predict the inclusion of non‐English‐language literature in ecological systematic reviews and maps, as well as the review authors' perspectives. We assessed systematic reviews and maps published in Environmental Evidence (n = 72). We also surveyed authors from each paper (n = 32 responses), gathering information on the barriers to the inclusion of non‐English language literature. 44% of the reviewed papers (32/72) excluded non‐English literature from their searches and inclusions. Commonly cited reasons included constraints related to resources and time. Regression analysis revealed that reviews with larger author teams, authors from diverse countries, especially those with non‐English primary languages, and teams with multilingual capabilities searched in a significantly greater number of non‐English languages. Our survey exposed limited language diversity within the review teams and inadequate funding as the principal barriers to incorporating non‐English language literature. To improve language inclusion and reduce bias in systematic reviews and maps, our study suggests increasing language diversity within review teams. Combining machine translation with language skills can alleviate the financial and resource burdens of translation. Funding applications could also include translation costs. Additionally, establishing language exchange systems would enable access to information in more languages. Further studies investigating language inclusion in other journals would strengthen these conclusions.


Highlights
What is already known • Non-English-language literature is often overlooked in systematic reviews and maps.
What is new • 44% (32 of 72) of assessed reviews did not include non-English language literature.• Reviews with more authors, authors from diverse countries, and multilingual author teams included more non-English languages.• Barriers to using non-English language methods in reviews include language skills and limited funding.• The rate of language inclusion did not change over time.
• Language diversity in the review team can improve inclusiveness.
• Machine translation combined with the review team's language skills may help reduce the financial and resource burdens of translation.
Potential impact for Research Synthesis Methods readers • Authors undertaking a systematic review or systematic map should carefully consider language as a potential barrier, and should understand and address this limitation by taking steps to mitigate its effects.

| INTRODUCTION
Evidence-based decision-making relies upon evidence synthesis, which involves the collation of evidence about a specific topic.Combining a large pool of evidence in a way that minimises bias allows for greater validity of and confidence in the findings.Due to this, systematic reviews and systematic maps are widely regarded as some of the most robust forms of evidence in science and have been used to inform decision-making and policy creation for addressing many global challenges including biodiversity conservation. 1,2The field of health science has long relied on evidence synthesis to inform healthcare decision-making. 3any authors make the choice to exclude non-English-language literature from their systematic reviews and maps. 4,5In the discipline of ecology, Zenni et al. 5 discovered high rates (82%) of review studies searching in English alone, and no trends for increased multi-lingual searching through time.However, this could bias results, reducing their relevance and usefulness, especially for decisionmaking.For example, the exclusion of non-English language literature can introduce language bias, wherein statistically significant results are more likely to be published in English. 6Similarly, there is a language bias in study characteristics because certain types of studies (e.g., specific species, topics and taxa, single species studies, studies conducted at the local scale) are more likely to be published in non-English languages. 7[10] Multiple factors could cause authors to exclude non-English-language literature from their systematic reviews and systematic maps.First, systematic reviews and systematic maps are often time and resource intensive.For example, systematic reviews and systematic maps published by the Collaboration for Environmental Evidence have been found to demand an average of 164 and 211 full-time equivalent days of working respectively. 11his high demand of time and other resources can lead to the exclusion of non-English language literature since authors may simply not have time to complete task of engaging with non-English language literature.Second, authors may believe that the quantity of relevant non-English-language literature is not high enough to be worthwhile, even though non-English language literature represents a large body of knowledge, 12 and the rate of publication is increasing at similar rates to English in many non-English languages, at least in biodiversity conservation. 13Third, some may believe that it is not necessary to search in languages other than English due to the perceived lower quality of non-English language literature, 14 despite an analysis showing that methodological quality in non-English language literature on biodiversity conservation is only slightly lower than English language literature. 8,9Lastly, a lack of relevant language skills within the review team and inaccessibility of/lack of knowledge for how to find non-English language literature have also been shown to be major impediments to language comprehensiveness in science.
In systematic reviews in social sciences, it has been reported that international review teams are more likely to include non-English language literature in systematic reviews and maps, 4 and that lack of time was also frequently cited as a barrier to including non-English language literature.However, the prevalence of, and barriers to, the use of non-English-language literature in environmental systematic reviews and maps are still poorly known.This is concerning, given that non-English-language literature seems to play an especially important role in biodiversity conservation. 8,9Environmental Evidence is the journal published by the Collaboration for Environmental Evidence (CEE).This is the only journal that focuses primarily on the publication of systematic reviews and systematic maps relevant to conservation decision-making.Due to rigorous review processes and editorial triage, 15 the reviews published in the journal can be considered representative of the highest-quality systematic reviews being produced globally in the field.Topics range across a wide spectrum of ecology, environmental science, and conservation and include an array of authors from around the globe.The CEE guidelines explicitly discuss the issue of inclusion of non-English languages and recognise language bias as a serious potential issue for many systematic reviews. 15The CEE guidelines also mention the need to search in multiple languages to achieve a representative sample of literature. 15Despite recommendations such as these, there is little information on whether systematic reviews and systematic maps typically include non-English-language literature, and what kinds of barriers are faced by authors in their pursuit of inclusion of non-English language literature.
Several methods have emerged as potential solutions to help authors include more non-English language literature.One such method is a language exchange system, as is described by Khelifa et al. 16 Within this system skills in a non-English language (reading and interpreting papers) can be exchanged for skills in another non-English language or English language proofreading.To our knowledge, no such system exists with the discipline of ecology and conservation science.Another known potential solution is machine translation.This technology is promising, although in its current form, it can be prone to altered meanings and other inaccuracies. 17,18More study is needed to understand the current strengths and limitations of using machine translation in this context.
Here we aimed to address this knowledge gap by quantifying the use of non-English-language literature in systematic reviews and systematic maps published in Environmental Evidence and identifying any factors that might predict the inclusion of non-English-language literature.We also aimed to understand the major barriers that limit the inclusion of non-English language literature, and the authors' perceptions of some suggested methods for overcoming these barriers.Understanding these factors is a crucial step in mitigating barriers in the future and allowing for greater inclusion of non-English language literature in systematic reviews and systematic maps, critically important tools to inform decisionmaking and policy in conservation.
Overall, we expected several findings to emerge from our data.The number of non-English languages searched was expected to increase over time, since the understanding that there is relevant non-English language literature has also increased.It is expected that studies which found a higher number of articles in the initial search would have lower rates of language inclusion, due to the high cost involved with translating a higher volume of articles.We also expected systematic reviews and systematic maps with a broader spatial remit to include a wider range of languages, to encompass the knowledge from the countries being studied.It is expected that a larger or more diverse author team would allow for a greater number of languages to be searched through greater linguistic diversity of the review team.

| Database
This paper analysed systematic reviews and systematic maps published in the journal Environmental Evidence since its launch in 2012 until January 2022.All records were extracted using Scopus (https://www.scopus.com/).As this study aims to assess the use of non-English-language articles in systematic reviews/maps, only systematic reviews and systematic maps were included (i.e., not commentaries or methodologies), resulting in 72 articles for inclusion in this study (Supplementary Data S1).

| Data extraction
Metadata containing bibliographical information (title, year of publication) and information about authors and their institutional affiliations were extracted from each of the 72 systematic reviews and systematic maps using the information downloaded from Scopus.Authors were also classified as being affiliated with a country where English is a primary language or not.Countries were defined as being English-speaking where English was listed as one of the official languages of the country according to Ethnologue (https://www.ethnologue.com/).Further data were manually extracted from each review/map (see Figure 1 and Table S1 for the summary of data collection).First, the abstract and title (and the main text, when needed) of each review/map were assessed to find information related to the study region of the review/map, which was recorded as both the main region studied and the spatial scale.The spatial scale was recorded in the following categories: global, multi-national, national, and regional.Articles that assessed a specific biome found globally were considered global despite having some biogeographic restrictions.The main region studied was recorded according to the spatial scope of the article.Multi-national articles were categorised into a potential 17 regions of the world based on the Intergovernmental Science-Policy Platform on Biodiversity and Ecosystem Services' (IPBES) defined subregions. 19For national reviews/maps, the country was recorded, and for regional reviews/maps the specific region studied was recorded.
The topic covered by each systematic review and systematic map was also extracted from the abstract or the main text and was categorised into one or multiple categories, with the categories being agriculture, biodiversity conservation, climate change, environmental economics, human health, invasive species, pollution, and resource management.The number of each type of database/resource searched (bibliographic, web-based, and organisational websites) was recorded.Next, the methods section was assessed, providing information about the search strategy.The languages used for literature searching (e.g., the languages of keywords used in searching) were recorded.However, we know that the presence of non-English language search terms is not the only way to capture non-English language literature in searching.For this reason, we also recorded these papers that searched in English alone but included studies in multiple languages and treated them separately from those that searched in English alone.Similarly, the presence of any geographic references in search strings, referring to the inclusion of things such as place names, were recorded.Data was also extracted from the methods section, which documented any imposed language restrictions on identified papers during the screening phase, as well as any justifications for the exclusion of certain languages.This section also occasionally referenced the methods used by the review team to analyse non-English language literature (e.g., review team language skills, the use of machine translation).If such information was available, it was recorded as well.The limitations section of each review/map was analysed to see if the authors acknowledged any imposed language restrictions as a limitation.Each review/map published in Environmental Evidence is required to include a RepOrting standards for Systematic F I G U R E 1 Summary of extracted data items from different stages of the review process.See Supplementary Table S1 for more details.
Evidence Synthesis (ROSES) flow diagram.This provides information on the number of papers included/excluded at each stage of the review process and occasionally reveals the number of papers excluded specifically due to language.Each articles' additional file with metadata on the included systematic reviews or systematic maps was assessed to determine the total number of papers included and the language of each of these papers if this information is recorded by the authors.For 17 systematic reviews and systematic maps, metadata relating to the languages of included sources was not available.For 12 of these, this was manually extracted by excluding the reviews' included papers published in English-only journals, and then manually assessing the remaining papers to determine their language.For the remaining five systematic reviews and systematic maps, this manual extraction was not possible due to the excessively large number of included papers (n = 2) or the unavailability of the metadata (n = 3).Finally, a global search was performed within the text of the review manuscript of each paper for 'language', 'English' and any relevant non-English languages, depending on the review's language inclusions (e.g., 'French' or 'Japanese') to ensure that no relevant information had been missed.Further information on the extraction process can be found in Supplementary Table S1.

| Survey
An online survey was sent to the corresponding author of 66 of the 72 articles.Six articles had one of the authors of this paper as their corresponding author and thus were excluded from the analysis.If we received no response from the corresponding author, the next listed author was contacted.Respondents were asked to provide information on the number of languages spoken by or understood by their review team (i.e., fluent enough to be able to interpret a scientific paper written in the language, whether or not this language skill was used in the review process, including those not listed as co-authors but who were involved in literature searching/screening/data extraction), to help understand the factors contributing to higher or lower inclusion of non-English language literature.Respondents were also asked about the barriers that they have experienced in trying to include non-English-language literature in their systematic reviews and systematic maps and any processes they have used to overcome them.Specifically, we asked authors to identify which barriers (if any) they had faced when conducting their systematic review/map (e.g., lack of relevant language skills within the review team, lack of time, inaccessibility of non-English language literature).Authors who had faced any barriers were then asked about how likely they would be to expand their review/map to include non-English language literature had this barrier been removed.This allowed us to understand the power that these barriers have to reduce authors' willingness/capacity to include non-English language literature.Further questions regarding these methods of minimising or overcoming barriers when including non-English-language literature were asked to gain insight into the best methods and any necessary improvements to processes to overcome barriers to including non-English-language literature.We identified several common approaches that authors may use to overcome barriers, which were provided in the survey along with free text response: machine translation (e.g., Google Translation or other machine translation tools); paid professional human translation; engagement with others with relevant language skills who were not involved as coauthors; and engagement with others with relevant language skills who were involved as co-authors.Authors were asked about their use of these tools, and/or the main barriers to using these tools (e.g., lack of resources, time, unsure how to use).For each question, authors could select responses from the list, and/or provide their own response.The survey is provided as Supplementary Text S1.
The survey was implemented on Qualtrics. 20We created a link to the survey, which was used for its distribution.The corresponding author of each paper was first contacted via email and invited to respond to the survey and was later reminded if we had received no response after 2 weeks.The authors of multiple papers were invited to fill out the survey for each review or map that they were involved in.Authors were also asked to let us know if another author of their paper may be more suited to answering the survey and were invited to forward the invitation or provide us with their details.The authors were given 1 month to answer the survey, with a reminder at the 2-week mark.If a month passed without a response from the first-contacted author, we approached a second author from the paper, usually the senior author (assuming this is the last listed author) or the first author.In this round, authors were given a 2-week time frame to complete the survey if they wished.The survey was completed between May and July 2022 in accordance with the University of Queensland's Institutional Human Research Ethics Approval (approval number 2022/HE000517).All participants were at least 18 years old and provided written consent indicating their agreement to participate in the survey.The Participant Information Sheet clarified the voluntary nature of participation, the aims of the research, how the data would be used, and that all data would be confidential.After the timeframe, the survey was closed to prevent any future responses.
Two multivariate models were developed in R version 3.6.0. 21The first model was a Poisson generalised linear model (GLM) identifying factors associated with a higher number of languages searched by the authors.In this model, the response variable was the number of languages searched in each systematic review/map.We selected explanatory variables that we expected could potentially correlate with a higher number of languages searched: the year of publication, spatial level (two categories: national and provincial vs. multi-national and global as the reference category), number of studies found in the initial literature search, number of authors, number of author countries (defined as the number of distinct countries of the authors' affiliations), and the percentage of authors affiliated with countries where English is a primary language.The second model was a binomial GLM, which assessed whether a paper was language inclusive (searched for and/or screened non-English-language literature) or not as the response variable.For this, we used the same explanatory variables as the Poisson GLM above.Both models used data from the extracted information only and not the survey.In running these models, a clear outlier was detected in the Poisson GLM with the full dataset (Figure 3).This model was rerun with this outlier removed to assess whether it affected the conclusion of the analysis (also see Supplementary Figure S1 and Table S2).The Variance Inflation Factor (VIF) was sufficiently small (<4.18, calculated with the package car in R 22 ) for all explanatory variables.The author's working country may not be a perfect measure of the linguistic capacity of a review team.For this reason, another Poisson GLM and another binomial GLM were run with a smaller dataset from the systematic reviews and systematic maps where the authors  responded to the survey.Both the Poisson and Binomial models remained the same as described above with the addition of the variable number of languages spoken by the review team in each model.In addition, for both models, the variable number of author countries was removed from the analysis due to high VIFs (>5).Due to our small sample size, and the relatively large number of authors with multiple papers in our sample (e.g., the three most prolific authors lead-or co-authored 18 of the 72 papers in this study), we repeated all analyses by incorporating the first author as a random factor, while keeping all other explanatory variables same to test if the reoccurrence of authors was creating bias in our analysis.For this we fitted either Poisson or binomial generalised linear mixed models, implemented with the R package lme4. 23| RESULTS

| Searches for non-English-language articles
Of the 72 included reviews/maps, 44% (n = 32) did not search for papers or screen papers in any language other than English.A further 18% (n = 13) did not search for papers in any language other than English but did screen papers in multiple languages which were captured by their English-language search.The remaining 38% (n = 27) searched for and screened papers in at least one language other than English.47% (n = 15) of the 32 reviews/maps that did not search for or screen for non-English-language literature provided some justification for this restriction.Of the 13 reviews/maps that did not search in any language other than English but did screen papers in multiple languages, five (38%) also provided some justification.Across both categories, the most common justifications were resource and time constraints, the linguistic knowledge of the review team and that it was outside of the political or geographic context of the review (n = 11, 6 and 2, respectively).
Of the 32 reviews/maps that performed searching and screening only in English, 59.4% (n = 19) mentioned language as a limitation and acknowledged that relevant literature was likely to exist in other languages outside the review's imposed language scope.A further four reviews mentioned language in the limitations section, but justified the exclusion of non-English language literature, stating that they do not believe it would have influenced their findings.
F I G U R E 3 Relationships between the number of languages searched in each systematic review/map and (a) the number of authors associated with countries where English is a primary language and (b) the number of author countries.The regression lines are based on the fitted Poisson generalised linear model (Table 1) with 95% confidence intervals shown as shaded areas and with the exclusion of an outlying datapoint.Jitter is used to show all data points.The same figure with the outlier included can be found in Supplementary Figure S1.n = 71.
Of the 26 reviews/maps that searched for literature in languages other than English, the number of non-English languages searched ranged from one to seven (median = 3).The range of languages searched by those reviews/maps was extremely limited, with most (92%) languages being of European origin, in spite of the fact that many of those reviews/maps had a global focus (Figure 2).The most common non-English languages used in the searching stage was Swedish, followed by French and Finnish (n = 18, 15 and 12).42.3% of the 52 global-scale reviews/maps did not search for or screen papers in any language other than English.
Of the 40 reviews that screened non-English language literature, 22 provided information on how they assessed non-English language literature.Screening was enabled mostly through the language skills of the review team (n = 14).A few other reviews utilised human and machine translation (n = 1 and 3, respectively).Two reviews used both the language skills within their review team and translation (not specified if machine or human).Another two reviews only screened non-English language literature which had provided an English translation of their title and/or abstract.

| Use of non-English-language literature
In 42 reviews/maps that identified at least one potentially eligible non-English-language article and reported their reasons for article exclusion, a median of 24.5 non-English-language articles (range: 1-323) were excluded before being screened, simply due to being outside of the imposed language scope.Thirty-five reviews provided a list of these excluded non-English articles to allow for further analysis.These lists had a median of 13 additional non-English language papers (range: 1-201).Of the 26 reviews/maps which searched for or screened at least one non-English language, a median of 3.5 non-English language articles (range: 0-164) were included in each systematic review/map, constituting a median 4% (range: 0%-41.4%) of the total number of articles included.F I G U R E 4 Relationship between the number of languages searched in each systematic review/map and the number of languages spoken by the review team.The regression line is based on the fitted Poisson generalised linear model using survey data (Table 2).The shaded area represents 95% confidence interval.
Jitter is used to show all data points.n = 32.

| Factors associated with language inclusiveness
The Poisson GLM found that the total number of languages searched in each review was negatively associated with the percentage of authors affiliated with countries where English is a primary language, and positively with the total number of author countries (Table 1 and Figure 3).The model was run again with the exclusion of an outlier visible in Supplementary Figure 1B, yielding qualitatively similar results (Table 1 and Figure 3).Figure 3 is presented with the exclusion of an obvious outlier.The significance did not change even when including all datapoints in the analysis (Supplementary Figure S1).In addition, the analysis was run with the first author as a random factor to understand if any bias was introduced by the small number of authors in our sample, which showed that the result did not change either (Supplementary Table S3).
With the smaller dataset including information from the survey (n = 32), only the number of languages spoken by the review team showed a significant positive association with the number of languages searched (Table 2 and Figure 4).
The binomial GLM revealed that the level of language inclusiveness (searched for and/or screened non-English-language literature or not) also showed a significant negative association with the percentage of authors affiliated with countries where English is a primary language in both the full analysis (Table 3 and Figure 5) and survey analysis (Table 4 and Figure 6).Neither of these results changed when the outlier was excluded from the analysis.The full data was also analysed with the first author as a random factor to understand if any bias was introduced by the small number of authors in our sample, which again showed that the result did not change (Supplementary Table S4).

| Language barriers experienced by review teams
Our survey received 32 responses from authors of unique papers from the 66 different systematic reviews/maps T A B L E 3 Results of binomial generalised linear model testing the association between the level of language inclusiveness (searched for and/or screened non-English-language literature or not) in each systematic review/map and explanatory variable.F I G U R E 5 Relationship between whether a review was inclusive of other languages at either the searching or screening stage and the percentage of authors affiliated with a country where English is a primary language.The regression line is based on the fitted binomial generalised linear model (Table 3).The shaded area represents 95% confidence interval.Jitter is used to show all data points.n = 72.

Variable
that we reviewed (response rate = 48%).Fifty-three authors listed as the corresponding authors of the papers were contacted in the first round.Twenty-eight authors were contacted in the second round because no response was given in the first round.The second round of authors consisted of the highest listed author other than the corresponding author.Responses revealed that the review teams spoke a median of three languages (range: 1-9, including English for all reviews).The most common barriers that impeded the searching for and screening of non-English-language articles were a lack of relevant language skills within the review team (n = 21), followed by limited time (n = 18, Figure 7).Only four authors stated that they had not experienced any barriers in preparing their review.For authors who had faced some sort of barrier, 68% stated that if the barriers had been removed, they would have been somewhat (32%) or extremely (36%) likely to expand their search to include non-English language literature.During the searching stage, 28% (n = 9) of authors utilised one or more tools to enable the assessment of non-English-language literature.The most frequently reported processes were engagement with others with relevant language skills who were included as co-authors (n = 7), followed by machine translation (n = 3).A further 31% (n = 10) considered using those processes but ultimately decided against it, due to time constraints (n = 8), lack of funding (n = 3), limited resources (n = 3), and the thought that non-English languages would not hold much relevant literature (n = 3).During the screening stage, 22% (n = 7) of authors utilised these tools to enable the assessment of non-English language literature, and a further 38% (n = 12) considered it but ultimately decided against it, due to time financial and/or resource limitations (n = 8), or because the author team did not have experience using processes such machine translation (n = 2).
Authors were concerned that using machine translation in both the searching and screening stage might cause loss of some of the original meaning (Figure 8a).The quality of translations and the time it may take to T A B L E 4 Results of binomial generalised linear model (with the survey dataset, testing the association between the level of language inclusiveness) (searched for and/or screened non-English-language literature or not) in each systematic review/map and explanatory variable.F I G U R E 6 Relationship between whether a review was inclusive of other languages at either the searching or screening stage and the percentage of authors affiliated with a country where English is a primary language.The regression line is based on the fitted binomial generalised linear model using survey data (Table 4).The shaded area represents 95% confidence interval.

Variable
Jitter is used to show all data points.n = 32.
perform machine translation were selected as the next major barriers in the searching and screening stages, respectively.The authors stated the major barrier to using professional human translation in the searching and screening stage was financial limitations, followed by difficulty finding translators with relevant subject-specific Count of barriers that impeded the searching for and screening of non-English-language articles in the specific systematic review/map published in Environmental Evidence.The 32 respondents were allowed to select multiple barriers (see Question 3 in Supplementary Text S1), so the total count of barriers exceeds the number of respondents.language skills (Figure 8b).Respondents identified the main barriers to engaging others with relevant language skills as the difficulty in finding contributors with relevant subject-specific language skills in both the searching and screening stages (Figure 8c).Finally, the authors were asked questions regarding a study by Khelifa et al., 16 which proposes a system where skills in a non-English language (reading and interpreting papers) can be exchanged for skills in another non-English language or English language proofreading.When asked how likely authors would be to access a system like this, 49% of authors responded that they would be somewhat or extremely unlikely to participate (Figure 9a).Most authors stated that the time requirements and unbalanced workloads were the main reasons not to access such a system (Figure 9b).

| DISCUSSION
We found that engaging with non-English-language literature, at any stage, was not widespread in ecological systematic reviews and maps published in Environmental Evidence, although the journal's guidelines strongly recommend searching in multiple languages for reviews to identify relevant articles in as unbiased a way as possible. 15Our study revealed that only 38% of the 72 reviews/maps, and 36.5% of the 52 global reviews/ maps published in Environmental Evidence included non-English languages at the searching stage.Even in those reviews/maps that identified potentially eligible non-English-language articles, a median of 24.5 non-English-language articles per study was not assessed and ultimately excluded from the study, due to being outside the language scope of the review.This represents a large number of articles that were picked up in the searching stage and potentially could have been relevant to the review.
Articles that searched in multiple languages identified a median of 4 non-English language studies (range 0-164) that were deemed relevant and included in the review/ map.These included non-English language articles represent a median of 4% (range 0-29) non-English language articles per study.This supports the need for article searching in multiple languages, as information could easily be missed if literature searching is restricted to English.The inclusion of relevant non-English language articles could reduce the effect of language bias in published research, potentially increasing the validity of conclusions drawn, 6,7 or potentially increase the taxonomic or geographic coverage of the data. 8,9imiting search languages to those spoken by the author team can cause studies to exclude relevant literature in other languages.To ensure all relevant information is captured, the languages used in searching should reflect the geographic scope of the review or the topic.While this should be a consideration that authors incorporate into all work, this can also be enforced by journals The count of responses of participants' likelihood to participate in a language exchange system proposed by Khelifa et al. 16 (b) Proportion of selected barriers to the use of this system.Respondents were allowed to select only one measure of likeliness, but multiple barriers.n = 32.
through journal guidelines and checked by editors and reviewers during the review process to ensure that the breadth of languages searched is appropriate for the geographic scope of the review.This scope should be dictated by what is applicable to the focus of the study.Studies may declare a global scope but often will be geographically restricted to the assessed species' ranges.In this case, a smaller array of languages may appropriately cover the scope of the review.It is recommended that authors aim to perform searching in and consider including relevant materials in any relevant languages.
Multilingual searching should always be the ideal option for authors due to the inaccessibility of non-English language literature within monolingual searches.5][26] Due to this, relevant databases should be accessed for each search language to ensure that an appropriate breadth of literature is being discovered.5][26][27][28] However, we understand that resource limitations sometimes prevent multilingual searching.Even when searching is performed in English alone, non-English language literature can often be captured.This is reinforced by our study, where 18% of included systematic reviews and maps did not perform searching in any non-English languages but included papers in multiple non-English languages.This demonstrates that searching in multiple languages, though important, is not the only way to capture information across an array of languages.If English-only searching is to be performed, controlled vocabulary should be used for better sensitivity in retrieving potentially relevant non-English language literature.To ensure appropriate breadth of synthesis, it is recommended that regardless of the number of languages used in searching, all potentially relevant search results should be screened, rather than being excluded based on language.We also recommend that the scope of a review should be articulated with greater transparency, specifically listing languages used at both the searching and screening stages and providing a list of any articles excluded due to language.Specifically, geographically or linguistically biased evidence should be declared, and the review's scope should be adjusted accordingly.
Our survey revealed that most (87.5%)authors had faced at least one barrier which hindered their use of non-English language literature in their systematic review/map, but most (67.9%)authors would also be at least somewhat likely to expand their search had this barrier been removed.This reveals that most authors have some desire to include a broader range of languages but have found the barriers too great to overcome.When asked about processes to reduce the barriers and enable greater assessment of non-English language literature, the majority (78%) of authors did not utilise any tools or processes.This may be because of some of the seemingly immovable barriers, such as lack of time or funding which are not primarily controlled by the authors themselves. 25he most common method to screen non-English articles was utilising language skills within the review team.We found that author teams with a higher proportion of authors affiliated with countries where English is not a primary language tended to search in more non-English languages and were more likely to include any papers in non-English languages in both the searching or screening stage.Similarly, our analysis also demonstrated that more diverse review teams, in terms of author countries and languages spoken, used more languages in searching.These results suggest that a purposeful expansion of author teams to include a wider representation of linguistic abilities would allow for more comprehensive synthesis containing evidence sourced from multiple languages.More diverse research teams are also able to provide varied cultural perspectives on a topic, which may result in a deeper understanding of the topic and the context surrounding it. 29The identification of necessary language skills can be done in the initial stages of planning the review, where authors should consider languages based on geographic scope and where relevant literature may have been produced.
One alternative to increasing the size or composition of the review team would be a language skill exchange system, such as proposed by Khelifa et al. 16 To our knowledge, systems such as this exist within other disciplines (e.g., References 15, 30), but not within conservation science.However, this system would still need to address the issue of time-intensity and unbalanced workloads of participants, which were the main concerns raised by the authors in our study.The low level of interest of authors in our study in a system such as this may be explained by the existence of these barriers and there may also be an imbalanced demand for such a system based on linguistic background, with native-English speakers potentially not seeing the personal benefit in such a system due to the under-appreciation of non-English language literature.
Another method used by research teams to identify and assess non-English language literature was translation, in the form of both human translation (outside of the author team) and machine translation.Professional human translation is a translation option that is often unused due to expense.Although professional translation will usually produce better results than machine translation, it can be expensive and difficult to find someone with subject-specific language skills.For this reason, translation costs could be built into funding applications, and/or could be distributed at an institutional level.
Machine translation is a lower-cost alternative, but authors in our study were concerned about potential alterations of meaning, and an observed low quality of machine translations.This is important to overcome, as any alterations of meaning may entirely alter the interpretation of the work, leading to inaccurate conclusions. 17,18nderstanding the validity of using machine translation in academic work is crucial but is largely understudied.The effectiveness of machine translation varies depending on the specific language pairs involved. 31,32Generally, machine translation systems tend to perform better when translating between languages with larger amounts of available training data and linguistic similarities.Language pairs such as English-Spanish, English-French and English-German are considered 'high resource languages' and have received significant attention and development, resulting in relatively higher quality translations. 31,33,34ifferent machine translation platforms such as Google Translate and DeepL may also vary in their performance for different tasks. 35The performance of machine translation can still vary depending on the complexity and nuances of the languages involved. 34,36The utilisation of artificial intelligence in machine translation is a promising tool that may play a crucial role in increasing the multilingualisation of conservation science. 34,36However, more testing and development is needed before it will be able to be widely utilised.With the currently available machine translation technology, several methods can be employed to decrease the chance of errors, mostly requiring additional human input to assess the translation systems output. 37,38Machine translation combined with the utilisation of the review team's knowledge may reduce the financial and resource burden of translation.The combination of both methods might reduce the inaccuracy of machine translation through manual checking while reducing the individual time burden on authors performing full translations. 39nother major barrier faced by authors was a lack of time and lack of funding.These limitations may cause authors to impose restrictions that would otherwise not exist on reviews, such as restricting the languages used in searching for and screening.These barriers are often not directly influenced by the authors themselves and are instead imposed by restrictions and pressures from institutions and funding bodies.These organisations could take responsibility for overcoming this barrier by encouraging the use of non-English language literature through funding and support for authors. 25,40Consideration of language inclusion at the grant application and planning stage will also help to minimise these barriers by building these costs into the overall estimates for the time and financial requirements of a project.However, we do recognise that these kinds of studies are often already considered expensive, so additional costs may be perceived as unreasonable by some.Due to this, there is a need for more studies to quantify the time requirements versus benefits in terms of rigour at each stage.When resources are limited, a cost-benefit analysis may be advantageous in identifying which parts of a review are the most beneficial to emphasise.Walpole 40 provides greater guidance on these recommendations, and how to realistically facilitate the inclusion of non-English language literature at all stages of a systematic review.
We acknowledge the limitations of our study.First, our study's relatively small sample size may limit the broad applicability of findings, though this study investigated all systematic reviews and maps published in Environmental Evidence, the only journal that specifically publishes ecological systematic reviews and systematic maps in the field of conservation.Reviews published by CEE are of a high quality due to the rigorous standards authors must adhere to.Appraisal of this body of work highlights meaningful areas of improvement and methodological ideals to work towards.Expanding this scope to include a wider pool of systematic reviews and maps could result in the appraisal of studies with poorer methodological standards, meaning that the results of the analysis may be less applicable to this higher standard of systematic reviews and maps, but more representative of the wider pool of work.
Zenni et al. 5 have undertaken a similar study, assessing the use of non-English language literature in ecological evidence synthesis.This study also identified a large proportion of articles that did not include non-English languages when searching for literature, with 49 of 60 syntheses being specifically limited to English.The study also identified no change in the tendency for monolingual literature searching over time.Studies similar to ours, but with a wider scope, would be recommended to accurately describe the state of non-English-language use in systematic reviews and systematic maps.Nevertheless, given that systematic reviews/maps published in this journal follow the strictest guidelines, we expect that the level of use of non-English-language articles among broader ecological studies is much lower.Our study was also limited by the information provided in the papers.For some of the reviews/maps, relevant information to our study (e.g., the number of non-English-language articles included) could not be found.Our survey received a good response rate (48%), although the absolute sample size was still rather small.We also acknowledge that the working country of authors is not a perfect measure of the diversity in languages that may be understood by a review team.For this reason, we also conducted our analysis with the language data provided in the survey, which was also found to significantly affect the number of languages used in searching.This gave validity to using author working country as a measure of review team language diversity in our wider analysis.However, asking authors in the survey for the language skills within the review team is also imperfect, as it relies on one author having knowledge of the language skills of the entire author team.However, the language skills that were discussed and/or used within the review or map should be represented.
This research extends a body of work that exists in other disciplines but has not been fully explored within conservation science.Understanding the impact that language barriers have on conservation research shows what pools of knowledge are being utilised most, and what is being ignored.Since systematic reviews are often designed to be used by practitioners and decisionmakers, any bias or missed information from restricted language reviews could be detrimental to the usefulness of the review.However, there is no simple solution, so it is imperative to understand why authors put these restrictions in place to create more effective solutions.Our survey allowed us to understand the authors' perspectives on these issues, the difficulties they have faced including multiple languages, and the approaches they have utilised to overcome this issue.Careful consideration of language as a barrier should be exercised by any authors looking to undertake a systematic review or systematic map, any editors and reviewers who assess the validity of a systematic review/map, and any funding body that supports relevant projects.Understanding the most effective use of resources for the specific review will allow teams to build provisions for the assessment of non-English language literature into their planning and could lead to greater inclusion of non-English language literature.assessing the importance of non-English conservation science.
Neal R. Haddaway is a freelance researcher working in evidence synthesis methodology.His background is in environmental science, but since 2012 he have been developing and testing methods in interdisciplinary evidence synthesis (systematic reviews, systematic maps, evidence maps, etc.).His specialities include systematic mapping methods, evidence synthesis technology (tools to support evidence synthesis), developing communities of practice, Open Science and evidence synthesis (Open Synthesis), evidence synthesis capacity building and education, and stakeholder engagement in evidence synthesis.
Richard A. Fuller is a conservation biologist who has been working to integrate several disparate disciplines to answer fundamental questions around a sustainable environmental future.To do this, he studies the impacts of people on the environment, and what can be done to mitigate those impacts.He has received continuous ARC funding for his research in these areas since 2010.Richard was appointed by the Australian Government to serve on the Ecosystem Research and Monitoring Panel and frequently advises the Australian Government on migratory species conservation.
Tatsuya Amano -As a conservation scientist, Tatsuya is working on understanding changes in global biodiversity and providing scientific evidence for its conservation.Through his work and his background as a conservation scientist originally from Japan, he has become increasingly interested in, and is committed to, unveiling the importance and consequences of language barriers in conservation and more broadly in science.Tatsuya is currently a UQ Amplify Senior Lecturer based at the School of the Environment and the Deputy Director in Research at the Centre for Biodiversity and Conservation Science, the University of Queensland, Australia.

F
I G U R E 2 Non-English languages searched by systematic reviews/maps with a specific geographic scope.Scope has been split into global, single continent (Europe), and smaller regions (Central and Western Europe) to accurately reflect the spatial scope of the reviews.Some reviews/maps searched in multiple languages.The reviews covering four other regions (n = 2 Africa, n = 2 North America, n = 1 Southern Africa, and n = 1 North Asia) did not search in any non-English languages, thus are not shown.Every review/map performed searching in English.n = 72 in total, 55 for Global, 2 for Europe, 9 for Central and Western Europe.

F I G U R E 8
Barriers relating to different known methods to facilitate the inclusion of non-English-language literature in the screening and searching stages of systematic reviews/maps: (a) machine translation, (b) professional human translation, and (c) engaging others with relevant language skills.Authors could select multiple barriers.n = 32 for each category as every author answered every survey question.

1
Results of a Poisson generalised linear model testing associations between the number of languages searched in each systematic review/map and five explanatory variables.
Note: The spatial level was grouped into national or provincial versus multi-national or global (the reference category).Significant results are highlighted in bold.See Supplementary Table2for the results with the inclusion of the outlier.n = 71.
T A B L E 2 Results of Poisson generalised linear model (with the survey dataset) testing associations between the number of languages searched in each systematic review/map and four explanatory variables.
Note: The spatial level was grouped into national or provincial versus multi-national or global (the reference category).Significant results are highlighted in bold.n = 32.