Appreciation of literature by the anaesthetist: A comparison of citations, downloads and Altmetric Attention Score

Background Different metrics exist to evaluate the impact of a paper. Traditionally, scientific citations are leading, but nowadays new, internet‐based, metrics like downloads or Altmetric Attention Score receive increasing attention. We hypothesised a gap between these metrics, reflected by a divergence between scientific and clinical appreciation of anaesthesia literature. Methods We collected the top 100 most cited and the top 100 most downloaded articles in Acta Anaesthesiologica Scandinavica (AAS) and Anesthesia & Analgesia (A&A) published between 2014 and 2018. We analysed the relationship between the average number of citations per year, downloads per year and Altmetric Attention Score. Results For both AAS and A&A, a significant correlation between the 100 most cited articles and their downloads (r = .573 and .603, respectively, P < .001) was found. However, only a poor correlation with Altmetric Attention Score was determined. For the 100 most downloaded articles, download frequency did not correlate with their number of citations (r = .035 and .139 respectively), but did correlate significantly with the Altmetric Attention Score (r = .458 and .354, P < .001). Conclusion Highly cited articles are downloaded more frequently. The most downloaded articles, however, did not receive more citations. In contrast to the most cited articles, more frequently downloaded papers had a higher Altmetric Attention Score. Thus, a ‘trending’ anaesthesia paper is not a prerequisite for scientific appreciation, reflecting a gap between clinical and scientific appreciation of literature.


| INTRODUC TI ON
The traditional way of ranking individual scientists is, among others, based on the number of citations their papers receive and the impact factor (IF) of the journals their articles are published in. In the internet era, new metrics to measure the impact of a paper, such as the number of downloads or Altmetric Attention Score, are available.
Those can be used to measure the impact of publications on both the clinical and non-clinical community. It quantifies the influence and attention an article receives by looking at social media, Wikipedia citations, reference managers, blogs, media coverage and public policy documents. 1 In recent years, many journals have reduced or abandoned their printed issues. As a result, researchers and other interested readers have to download an article before being able to read, or cite, it. This makes the number of downloads an increasingly accurate metric for the interest an article arouses with its readership.
In this study, we examined how literature is appreciated by anaesthetists by studying the relationship between the number of citations, the number of downloads and the Altmetric Attention Score Background: Different metrics exist to evaluate the impact of a paper. Traditionally, scientific citations are leading, but nowadays new, internet-based, metrics like downloads or Altmetric Attention Score receive increasing attention. We hypothesised a gap between these metrics, reflected by a divergence between scientific and clinical appreciation of anaesthesia literature. Results: For both AAS and A&A, a significant correlation between the 100 most cited articles and their downloads (r = .573 and .603, respectively, P < .001) was found.

Methods
However, only a poor correlation with Altmetric Attention Score was determined. For the 100 most downloaded articles, download frequency did not correlate with their number of citations (r = .035 and .139 respectively), but did correlate significantly with the Altmetric Attention Score (r = .458 and .354, P < .001).

Conclusion:
Highly cited articles are downloaded more frequently. The most downloaded articles, however, did not receive more citations. In contrast to the most cited articles, more frequently downloaded papers had a higher Altmetric Attention Score.
Thus, a 'trending' anaesthesia paper is not a prerequisite for scientific appreciation, reflecting a gap between clinical and scientific appreciation of literature. of popular anaesthesia papers. We hypothesise a gap between these metrics, reflecting a divergence between scientific and clinical appreciation of anaesthesia literature.

| Data collection
We reached out to two prominent anaesthesia journals, one from the United States and one from Europe, asking for their 100 most

| Statistical approach
We suspected that each list might contain one or two significant outliers, papers that received such a high number of citations or downloads that this could influence the analyses. To detect significant outliers, we calculated centred leverage values and standardised residuals for each paper in our lists. The Supporting information contain a detailed description of this process. To test the relationship between the different metrics, we used Spearman's rank correlation coefficient. Our data were analysed using the Statistical Package for the Social Sciences (SPSS, version 25).  For the most downloaded papers, Figure 1C represents the relationship between downloads and citation count, with r = .035 (P = .734). Figure 1D illustrates the correlation between downloads and Altmetric Attention Score, with r = .458 (P < .001).

| Acta anaesthesiologica scandinavica
When looking at the relationship between date of publication and Altmetric Attention Score for all 146 papers, a moderate correlation was found, indicating that more recent papers have higher Altmetric Attention Scores ( Figure 3A, r = .170, P = .04). Figure 3C shows the correlation between publication date and downloads, which gives r = .185, P = .025.  Figure 2C), but a relatively strong correlation with Altmetric Attention Score, r = .354 (P < .001, Figure 2D).

Editorial Comment
This article tells us that there seem to be a gap between scientific and the clinical appreciation of the literature, at least as judged by the relationship between citations, downloading and Altmetric Attention Scores. The rate of downloading and the Altmetric score probably reflect more clinical than scientific interest.  Secondly, we found that the 100 most downloaded articles revealed a strong correlation with the Altmetric Attention Score, while the 100 most cited articles did not. The Altmetric Attention Score is generated quickly after publication, making it a potential predictor of more slowly acquired metrics, such as citations or downloads.

| Main results and conclusions
Our data suggest that Altmetrics might be a good indicator of clinical interest and frequency of reading, but does not predict scientific interest and later citations. Furthermore, we found a significant correlation between the publication date and the Altmetric Attention Score. This illustrates 'recent bias', whereby metrics are influenced by the duration of a paper present in literature. 4 Therefore, it seems to be an unfair measure for comparing articles published in different years. However, once the use of Altmetric input variables, like social media, stabilises, the usefulness of Altmetric Attention Score as a way to evaluate papers might increase. 5 We found an even stronger correlation between the publication date and the frequency an article is read, reflecting the same recent bias we found for Altmetrics. For Acta Anaesthesiologica Scandinavica,  Gisvold argued that it is not research quality that determines the number of citations, but rather the medical specialty one publishes in (field effect) and whether one publishes on a popular topic that receives much scientific attention, also known as the band wagon effect. 12

| Comparison with previous studies
Despite being fairly new, many journal websites display an Altmetric Attention Score with their articles and the Altmetric Attention Score has also received attention in literature. Contrary to citations, the input variables for the Altmetric Attention Score are generated quickly after publication, which facilitates a fast and easy evaluation.
Altmetric Attention Score is an excellent indicator of public interest, and although the importance of public engagement in research should not be disregarded, it might fall short in predicting scientific quality. A study assessing the topics of the 100 highest scoring Altmetric articles revealed medical and health sciences being the most popular theme, with 36 papers in the top 100. However, the interest was limited to topics directly relevant to the public, such as diet and exercise. 5

| Limitations and strengths
The different metrics we studied are dynamic and might continue to grow for decades, which is in particular true for citations.
Consequently, our data might differ if we repeat this study a year from now. In an attempt to compensate for timing bias, we used the average number of citations and downloads per year since publication.
A strength of our study is that we analysed data from two different anaesthesia journals, a European journal and an American journal. The fact that both analyses yielded nearly identical results strengthens our observations.

| Conclusion
Highly cited articles are downloaded more frequently and, hopefully, read. Contrarily, the highly downloaded articles revealed no correlation between download frequency and citations count, showing that most of these downloads do not necessarily indicate scientific interest. There was, however, a strong correlation between downloads and Altmetric Attention Score. Thus, the number of downloads and Altmetrics both reflect a more short-term interest in a paper. However, such a 'trending' anaesthesia paper is not a prerequisite for scientific appreciation, supporting our hypothesis of a gap in the way scientific literature is appreciated by researchers and clinicians in the field of anaesthesia.