Cochrane review abstracts are getting longer, but this has no large impact on the reporting quality

To evaluate the reporting of abstracts of Cochrane Reviews according to PRISMA‐A and to investigate a possible association with the abstract´s length.

7][8][9] Bahadoran et al. (2020)   stated that "a manuscript's abstract is one of many features that compete for readers' attention" and points out that its main functions are to provide a summary of the work, to "sell" the work to editors, reviewers, and potential readers, and to help index the work so that it can be retrieved by various researchers. 8In this regard, an abstract of an article should be able to stand on its own, while at the same time reflecting all the essential content of the main manuscript in an "informative, accurate, attractive, and concise" 8 form that is also "pleasant to read." 7 To address the importance of abstract quality in SRs, PRISMA for Abstracts (PRISMA-A) was developed in 2013 10 and updated in 2020. 11In our previous study on the reporting quality of abstracts of SRs according to PRISMA-A, 12 we found a positive correlation between the number of words and reporting quality.Cochrane Reviews (CRs) were better reported than non-Cochrane Reviews and differed substantially in terms of the structure and length of an abstract.At a median length of 507 words (vs.249 words), CR abstracts are much longer than non-Cochrane review abstracts and appear to be steadily increasing in length over recent years. 12However, the considered sample of CR abstracts was too small to draw robust conclusions.Overall, there is a general lack of studies addressing the quality of reporting or trends in the length of CR abstracts.
Therefore, this study aims to evaluate the reporting of CR abstracts according to PRISMA-A and to investigate the possible influence of abstract length.

Study design and search strategy
This is a retrospective, observational study based on all CRs indexed in Medline (via PubMed).This database, where CRs have been indexed by default since 2000, was searched from inception to November 18, 2022, without restrictions and using the search term ""Cochrane Database Syst Rev"[jour]".All references and their abstracts were imported into SAS for Windows version 9.4 (SAS Institute Inc, Cary, North Carolina).
The study is based on two parts of analysis.The first part includes the full sample of identified CRs and aims to present trends in the number of publications per year as well as the length of abstracts.The second part includes a randomly drawn sample and focuses on the abstracts' adherence to PRISMA-A.

Inclusion and exclusion criteria
The Cochrane Database of systematic reviews was originally focused on publishing SRs.Over the years, the database has been opened to include editorials, methodological reviews, and overviews.In this study, however, we are interested only in SRs published in the Cochrane Database.References deposited in PubMed as methodological reviews (MR number) or as editorials (ED number) were excluded.In addition, titles and abstracts that identified themselves as an overview, protocol, or a withdrawn article (e.g., by the wording "WITHDRAWN" in the title or "review has been withdrawn" in the abstract) were excluded.
SRs without an available abstract were also excluded.
The second part included a randomly drawn sample of 440 CRs.
Based on a mean PRISMA-A score of 6.3 and a standard deviation of 1.3 from our previous study, 12 a sample size of 105 CRs per quartile was calculated for a moderate effect size (d = 0.5) at a significance level of α = 0.05 and a power of β = 0.95.As PRISMA-A primarily addresses SRs on effectiveness and is only partially applicable to other types, diagnostic reviews were excluded from the analyses.Assuming a 5% proportion of diagnostic reviews based on our study on trends in epidemiology and reporting in SRs, 3 the sample size was increased from 420 to 440 CRs.

Data extraction
For the first part of the analyses, which included all identified CRs, the following information was exported from PubMed: year of publication, title, authors, Cochrane Database number, and the abstract.The number of words in the title and abstract as well the number of authors were determined using SAS.No additional data was extracted.
For the second part of the analysis, further information was assessed.Two independent reviewers (KW, JH) extracted data on PRISMA-A adherence using a standardized and piloted extraction form.This included all twelve items defined in PRISMA-A from 2013 10 (1. title, 2. objectives, 3. eligibility criteria, 4. information sources, 5. risk of bias, 6. included studies, 7. synthesis of results, 8. description of the effect, 9. strengths and limitations, 10. interpretation, 11. funding, and 12. registration), operationalized and scored according to our previous study in which all items were presented in detail. 12As items 3 (eligibility criteria), 4 (source of information), and 6 (included studies) contain more than one component, subcategories for partial compliance were extracted for these items.Before data extraction, the two reviewers calibrated themselves using 30 CRs.The total PRISMA-A adherence was assessed by a score ranging from 0 to 12 points, with one point awarded for each fully reported item (fully reported: yes = 1; partly reported = 0; no = 0).Higher scores indicate better reporting.
Any disagreements between raters were noted and resolved through discussion.
Based on the expectation that the complexity of the underlying research question has an impact on the length and reporting of abstracts, the number of included studies was also extracted for all CRs examined in the second part of the study.If the number of included studies was not indicated in the abstract, this information was obtained from the full text.

Data analyses
For the first part of the study, we analyzed trends in the number of publications per year and the number of words included in the abstracts using descriptive statistics.Frequencies and proportions were calculated for categorical data and means with standard deviations (SDs) as well as medians with interquartile ranges (IQRs) for continuous data.
The second part of the study involved conducting an analysis of PRISMA-A adherence on the basis of a random sample.First, disagreements between the two raters (KW, JH) were calculated as percentages by items.Then, frequencies were calculated for all reported items, and means with standard deviations (SDs) and medians with interquartile ranges (IQRs) were calculated for overall adherence.The Kruskal-Wallis test was used to determine whether there were differences in adherence to PRISMA-A between quartiles of abstract length.The results were stratified by the number of included words in the abstracts (quartile) and by year.Assuming that the publication of PRISMA-A in 2013 had an impact on abstract reporting and that implementation took some time, we first used a categorization according to our previous study 12  Analyses for both parts of the study were performed using SAS.

Search results
Our search identified a total of 16,029 records.After excluding 841 references due to article type (methodological review, editorial, protocol, or overview), missing abstract, or withdrawal, 15,188 CRs were included in the first part of the analysis (Figure 1).The titles of the CRs had a median length of 9.0 words, and the abstracts contained a median number of 469 words (IQR 398-686).2).The percentage of abstracts ≤300 words decreased from 30.7% in 2000 to 6.6% in 2010 and less than 1% in 2022 (Table S1).

Characteristics of the random sample
In the second part of this study, we considered a random sample of 440 CRs (Figure 1).After screening the references, three diagnostic reviews were identified and excluded.

Disagreement between raters
The overall agreement between the two raters, who each independently reviewed the 437 abstracts using PRISMA-A and extracted 5928 items for analysis, was 93.7%, with interrater disagreements ranging from 0% to 19.3% between items (Table 1).While 100% agreement was achieved for 4 of the 12 items (item 1 (title), item 2 (objective), item 11 (funding), item 12 (registration)), 3 items had more than 10% disagreement (item 8 (description of the effect)) with 2 items having more than 15% disagreement (item 6 (included studies), and item 9 (strengths and limitation)).For all other items, disagreement ranged from 0.7% to 8.5%.

Overall adherence to PRISMA-A
Overall the abstracts of the 437 CRs had a total mean score of 6.1 out of 12 fully reported items.Adherence to individual items varied widely (Table 1), and only 4 of the 12 items (item 2 (objective), item 4 (sources of information), item 7 (synthesis of results) and item 10 (interpretation)) were fully reported by more than 85% of the abstracts.Item 1 (title) and item 5 (risk of bias) were reported only in 0.7% and 7.8% of CRs, respectively, and items 11 (funding) and 12 (registration) were never reported.Adherence to all other items ranged from 39.6% to 77.4%.

Year of publication
Looking at PRISMA-A compliance by year, mean adherence increases from 5.7 fully reported items in 2000-2010 to 6.5 items in 2011-2015 and further to 7.2 items in 2016-2022 (Table S3).
In 2000-2010, there was a significant difference in PRIMSA-A adherence between quartiles (p = 0.0050); mean adherence increased with a rising abstract word count from 5.3 items in the first quartile (≤ 373 words) to 6.2 items in the fourth quartile (≥ 472 words; Table S4).Also, during the period 2011-2015, a significant difference (p < 0.0001) and an increase in adherence can be seen from 6.0 items in the first quartile (≤ 458 words) to 7.1 items in the fourth quartile (≥ 677 words; Table S5).In 2016-2022, there was no significant difference in PRISMA-A adherence between quartiles (p = 0.9835), with PRISMA-A compliance remaining constant at 7.3 items in the first quartile (≤ 616 words), 7.2 items in the second quartile (617-750 words), 7.2 items in the third quartile (751-907 words), and 7.3 items in the fourth quartile (≥ 908 words; Table 3).

DISCUSSION
In Despite extensive operationalization and calibration, there was still a high discrepancy between raters (discrepancy range: 0% to 18.5%), indicating the high complexity of some PRISMA-A items and the difficulty for the authors to fully implement them.This is in line with an earlier study by Maticic et al. 21In 2020, PRISMA-A 11 was updated, and although it contains almost the same items, the checklist has now been restructured to make it easier to implement.However, PRIMSA-A was only updated and published as part of the PRISMA checklist 11 and lacks a more detailed description of the requirements for each item, particularly those that have been changed or added.Thus, the PRISMA-A checklist remains difficult to apply for some items, leading to different interpretations, including for authors of SRs.
At a mean total score of 6.1 items, CR abstracts seem to be better reported than abstracts of non-Cochrane Reviews, which completely report an average of 5.2 items. 12Nevertheless, they show some potential for improvement, especially regarding item 1 (title), item 5 (risk of bias), item 11 (funding), and item 12 (reporting).Cochrane uses standardized editorial processes 13 and provides very comprehensive information on what content to report, where to report it, and how to report it in both their handbook 14 and the Methodological Expectations of Cochrane Intervention Reviews (MECIR). 15Both explicitly refers to the PRISMA guideline.No specific information is provided on the title (item 1), funding (item 11), and registration (item 12).Since PRISMA is explicitly referenced, one might assume that PRISMA-A is applied for the items that are not further specified by Cochrane.
However, the opposite is true, as the items not listed in the handbook or MECIR have the lowest PRISMA-A compliance.Regarding risk of bias (item 5), inconsistencies between the specifications of PRISMA-A, the Cochrane Handbook and MECIR are apparent.The Cochrane Handbook did not include the assessment of risk of bias until 2019 16 and then, instead of referring to the risk of bias tool, 14 refers to the GRADE tool, which methodologically does not describe the risk of bias but assesses the overall quality of the evidence. 17In contrast, MECIR already addressed risk of bias back in 2012 18 but indicated that for many reviews, it is sufficient to state that standard Cochrane procedures were followed. 15,18PRISMA-A requires reporting both the methods used to assess the risk of bias and the corresponding tool.
Funding and registration are also not reported in other reviews 12,[19][20][21][22] and seem to be less important in abstracts than the information on results and methodology.The importance of the title, which is very rarely PRISMA-A compliant in CRs, should be re-emphasized in particular.CRs are very rarely referred to as SRs or meta-analyses in the title or abstract.This makes finding CRs in the bulk of literature much more difficult 23 when searching for the term "systematic review."In comparison, several studies show that more than 80% of non-Cochrane Reviews are labeled as SRs already in the title. 12,19,21 this study, CR abstracts had a median length of 469 words.This is significantly longer than the abstracts of non-Cochrane Reviews.
Furthermore, the length of non-Cochrane Reviews' abstracts has remained constant over the past 20 years. 12In contrast, abstracts of CRs have become longer, with the length remaining constant at a median of just under 400 words until 2010 and then steadily increasing up to 800 words.This conflicts with the Cochrane Handbook, which recommended a maximum length of 400 words until 2019. 16ereafter the handbook 14 was adapted to MECIR, 18 which has recommended a length of <700 words since 2012 but allows up to 1000 words.The introduction of MECIR 2012 may explain why the length of abstracts increased already from 2010, but more significantly from 2012.However, the longest CR abstract found, which contains 1807 words, 24 is far above all recommendations ever made by Cochrane.In its handbook, Cochrane emphasizes that the abstract must be able to stand alone because many readers only have access to the abstract, not the full text.This aspect is also discussed in other methodological papers on writing good abstracts, but the latter simultaneously emphasize that an abstract should be attractive and concise. 8Previous studies suggest that authors need a certain number of words to report in compliance with PRISMA-A, 12,20 especially in case of reviews that address multiple or complex interventions, but they also indicate that a word count of 300 words appears to be sufficient. 25CRs contain a median of seven studies, with more than 25% containing one to three studies, compared with a median of 13 studies included in non-Cochrane Reviews. 12CRs thus appear to be less complex than non-Cochrane that is also "pleasant to read." 7 Evidence is lacking, however, on how long an abstract should be to fulfill all these functions at the same time.To the best of our knowledge, there are also no studies assessing whether readers of the much longer CR abstracts more often refrain from reading the full text when compared to readers of non-Cochrane Reviews.
Future research on the priorities of editors, authors, and especially researchers regarding the length of abstracts is needed to gain better insight into how abstracts work and how long they should be.
To the best of our knowledge, this is the first study on the reporting

A total of 15 , 1
188 CRs were indexed between 2000 and 2022, with the number of published CRs per year ranging from 455 to 999 (with n = 305 between 1 January and 18 November 2022).After the start of indexing in the year 2000 (with n = 1006), the number of CRs published in PubMed increased steadily from 509 CRs in 2001 to 999 CRs in 2013, before steadily decreasing again to 518 CRs in 2021 (TableS1).Overall, 43.4% were published between 2000 and 2010, 29.4% from 2011 through 2015, and 27.2% from 2016 through 2022.Overall, an average of 4 authors were involved.Looking at the length of abstracts over the years (Figure2), with the exception of the year 2000 (median number of words 353, IQR Flow chart.

F I G U R E 2
Number of words in the abstract by year of publication (n = 15,188).282-424), the median number of words in the abstracts was quite consistently between 387 and 401 words per year until 2010.After 2010, abstract length increased steadily to 680 (IQR 526-811) in 2016 and reached a median of 838 words (IQR 663-960) in 2022 (Figure this study on all 15,188 CRs published in PubMed until 18 November 2022, the median number of words in the abstract was 469 (IQR 389-686 words), increasing from 353 words in 2000 to 838 words in 2022.Less than one percent of the abstracts in 2022 had ≤ 300 words (in 2000: 30.7%).The second part of this study on a random sample of 437 abstracts shows a mean score of 6.1 fully reported PRISMA-A items, with wide variation in reporting between items (adherence of 0%−100%) and an interrater disagreement of 6.3%.PRISMA-A adherence has increased over time; when stratified by years of publication, an increasing word count in the abstract went along with higher PRISMA-A adherence in 2000-2010 and 2011-2015, but no longer in 2016-2022.
quality and length of abstracts of CRs.It is based on all CRs indexed in PubMed without any restrictions.Data from the first study part were extracted directly from PubMed and counted using SAS.The second part was based on a random sample with a planned sample size of 105 CRs per quartile and a total size of 440 CRs.Data extraction was performed independently by two reviewers for all items.The exclusion of empty CRs, however, meant that the planned sample size was not achieved.Furthermore, the analyses stratified by year should be interpreted with caution as they were underpowered.The number of words in abstracts of CRs has increased over the years and so has PRISMA-A compliance.On average, CRs fully reported 6 of the 12 PRISMA-A items.Adherence increased with increasing word count in 2000-2010 and 2011-2015, while in 2016-2022, no differences in PRISMA-A adherence were found based on abstract length.Nevertheless, CRs have become increasingly longer over the years and in some cases have a word count as high as 1000.This conflicts with the Cochrane Handbook, which recommended a maximum length of 400 words before being adapted to MECIR in 2019.The latter has recommended a length of < 700 words since 2012 but allows up to 1000 words.It is debatable whether such long abstracts meet the goal of being informative, accurate, attractive, and concise at the same time.To better understand the impact of abstract length on readability, future studies should examine how long an abstract needs to be in order for it to be informative and capture the reader's attention.
Assessment of PRISMA-A adherence and discrepancy between raters.PRISMA-A adherence by number of words in the abstract (quartiles), empty reviews excluded.
In contrast, in the fourth quartile (≥ 671 words), 4.0% of articles were indexed in 2000-2010, 29.3% in 2011-2015, and 66.7% in 2016-2022.Overall, there was a significant difference in PRISMA-A compliance between quartiles (p < 0.0001).With a higher number of words, theTA B L E 1a The denominator of the fractions differs (n = 399) because the item was not applicable for some Cochrane Reviews, which did not include any study.TA B L E 2mean adherence to PRISMA-A increased from 5.4 fully reported items in the first quartile (≤ 398 words) to 7.1 items in the fourth quartile (≥ 671 words).This trend can be found for most of the items, except for item 3 (eligibility criteria), which increased from 53.3% in the first quartile to 66.0% in the third quartile (502-670 words) before decreas-ing to 52.5% in the fourth quartile.Item 6 was fully reported more than twice as often (63.6%) in the fourth quartile than in the first quartile (28.8%).Items 7 (synthesis of results) and 8 (description of the effect) increased from 78.1% and 57.1% respectively, to more than 95% adherence in the fourth quartile.
TA B L E 3 PRISMA-A adherence by number of words (quartiles) for the year 2016-2022.) are longer than the average length of non-Cochrane review abstracts identified in previous studies.Thus, it is debatable whether CR abstracts meet the goal of abstracts to present the main content of the manuscript in an "informative, accurate, attractive, and concise" 8 form