Including non‐English language articles in systematic reviews: A reflection on processes for identifying low‐cost sources of translation support

Non‐English language (NEL) articles are commonly excluded from published systematic reviews. The high cost associated with professional translation services and associated time commitment are often cited as barriers. Whilst there is debate as to the impact of excluding such articles from systematic reviews, doing so can introduce various biases. In order to encourage researchers to consider including these articles in future reviews, this paper aims to reflect on the experience and process of conducting a systematic review which included NEL articles. It provides an overview of the different approaches used to identify sources of low‐cost translation support and considers the relative merits of, among others, seeking support through universities, social media, word‐of‐mouth, and use of personal contacts.


| INTRODUCTION
Non-English language (NEL) articles are commonly excluded from published systematic reviews (e.g., References 1-6). The main reasons for this often relate to practical barriers such as the high cost and time commitment associated with translating articles, as well as lacking language resources such as translators, or translation software. 7 Whilst some have argued that the exclusion of NEL articles has a limited impact on the findings and overall conclusions of reviews, 6,8,9 excluding such articles may lead to an increased risk of bias, 5,10 or missing key evidence, 11 and may limit the generalisability of findings. 12 Furthermore, excluding NEL articles from qualitative reviews may mean that participants' experiences of culturally specific issues are not captured, and may serve to limit the transferability of the results. 13 From a practical perspective, excluding NEL articles during the search stage of a review risks the exclusion of relevant English language articles, where language values have been incorrectly defined or are missing. 12 Excluding NEL articles during the eligibility assessment stage instead allows reasons for ineligibility to be recorded, providing greater transparency about the number of articles excluded on this basis. 10,12 However, both the Cochrane Handbook and the Campbell Collaboration recommend that authors should assess all relevant articles for inclusion, irrespective of language. 14,15 It is therefore important to ensure barriers to including NEL articles are minimised as far as possible.
Be that as it may, researchers with access to smaller research grants may struggle to meet the costs associated with traditional translation methods, as many researchers are unlikely to have costed for this type of activity. * Furthermore, those new to the review process may lack knowledge about where to access translation services or perceive the task of locating affordable translators to be time-consuming.
The purpose of this article is to reflect on my experience of including NEL articles in a recent qualitative systematic review, to provide an overview of the different approaches used to identify sources of low-cost translation support.
Google translate was initially used to assess eligibility of article abstracts. Articles deemed eligible for full-text assessment were then translated by volunteer translators (i.e., students, researchers, other professionals). † Thirteen translators completed the tasks, some of whom translated multiple articles. Where possible, Google Translate was used to convert the documents before sending to the translators, who were then asked to proof-read and edit the text.
A nominal payment was offered for completion of the translation. The amount of work per article varied considerably, owing to the length of the document and the language (Google Translate is more effective at translating some languages, than others 17 ). The payment offered for each task was therefore adjusted accordingly (ranging from £15 to £40 per article).
To locate low-cost translators, a number of approaches were used, a summary of which is reported herein.
The first approach was to contact the University languages department via email, who forwarded details to their postgraduate students. Seven students responded with offers of help, and of these, three students were allocated the task.

| Clubs and societies
Through the University, I also contacted student societies specific to the article languages. Both the Nordic Association and the Francophone [French speaking] Society were contacted, requesting translators for the Danish and French articles, respectively. Using this approach, one additional translator was found.

| SOCIAL MEDIA
The most successful approach utilised was using social media (Facebook and Twitter) to post requests for translation support. Whilst no offers of help were made in response to the Facebook post, a fairly large response was received on Twitter; Over a three-month period, five requests were posted, which were retweeted 27 times. In response, numerous offers of help were made. Using this approach, six translators were found.

| PERSONAL CONTACTS AND WORD-OF-MOUTH
Two translators were also identified using personal contacts and word-of-mouth. One translator was suggested by a colleague, who's mother was an ex-professional French translator, and another by a colleague who had contacts with a Psychology department in Finland.
The last translator was found through re-contacting the 12 translators who had helped previously, to ask if they knew of any Danish-speakers who might be interested in the task. This approach is also known as 'snowball' sampling 18 and was employed having exhausted all previous methods.

| ALTERNATIVE APPROACHES
Whilst the purpose of this article is to reflect on my own experiences of sourcing translation support, there are a couple of additional approaches I have since become aware of that I wish to share.

| Cochrane Task Exchange
Cochrane Task Exchange 19 is an online platform allowing researchers to request, or offer, peer-support for systematic reviews and research synthesis in biomedical science. 20 In way of compensation, those providing research support are offered authorship, acknowledgement, and/or payment. High response rates to tasks posted on the site have been reported, and user feedback has been positive. 20 However, it is important to highlight that this resource may be better suited to identifying translation support for quantitative reviews, although there are no restrictions by methodology.

| Direct invitation
An additional approach that could be utilised is to identify and contact potential translators directly. Society member directories and bibliographic databases, for example, could be used to identify researchers residing in, or publishing in, the country of interest, who may be well placed to assist.
See Table 1 for a summary of all approaches discussed.

| REFLECTIONS ON THE PROCESS
Using the above reported methods to identify potential translators was hugely effective and all 17 articles were translated to a high standard, for a low cost. However, there are some considerations that must be taken into account.

| Quality
Translation quality is an important consideration when using non-professional translators, especially when conducting qualitative reviews, as it is crucial that the meaning of the data is not lost in the translation process. This was emphasised to all translators who helped with this review, the majority of whom were extremely thorough.

| Payment
There is a sparsity of guidance available on payment for informal activities such as this and as such, I was concerned that the nominal payments offered may be inadequate. However, it was evident that a number of translators were keen to contribute to the review regardless of payment, or were students who were eager to earn some extra income. Whilst I provided monetary compensation for the work completed, it is important to highlight that authorship or acknowledgement could also be offered as alternative sources of compensation.

| Lack of contract
The approaches discussed, whilst convenient, depend upon casual arrangements, often with individuals who are not known to the researcher. It was my experience that some translators decided not to complete the work, after committing to the task, for various reasons. Whilst this happened infrequently, a benefit of using a professional service would be that the work is guaranteed to be completed.

| CONCLUSION
Whilst it remains an empirical question as to whether different sources of translation support vary in any meaningful way, there is no doubt that reducing barriers to including NEL articles will result in more thorough reviews that are far more representative of the current literature. For researchers who are hesitant about making the decision to include NEL articles in their own work, I am hopeful that the insight shared here might help to facilitate it. However, it is important to consider that the success I experienced in securing translation support will have been influenced by the fact that a monetary incentive was offered and that as a PhD student, I may have had more time available to spend identifying translators. I acknowledge that some researchers will continue to experience financial and/or time constraints that prevent them from identifying or accessing even low-cost sources of translation support.
Going forward, it is therefore important for researchers to factor in the costs associated with translating NEL articles when writing grant proposals, and for funding bodies to allocate funding appropriately, to reduce barriers to inclusion.