Fate of manuscripts declined by the British Journal of Surgery

Abstract

Background:

The fate of papers submitted and subsequently rejected by the British Journal of Surgery (BJS) is currently unknown. The present study was designed to investigate whether, when and where these papers are published following rejection.

Methods:

All rejected manuscripts in the year 2006 were identified from the Manuscript Central electronic database. Between December 2008 and February 2009, a PubMed search was conducted spanning the period 2006-2009 using the corresponding author's last name and initials to identify whether and when manuscripts had been published elsewhere.

Results:

From the 926 manuscripts rejected by BJS, 609 (65·8 per cent) were published in 198 different journals with a mean(s.d.) time lapse of 13·8(6·5) months. Some 165 manuscripts (27·1 per cent) were published in general surgical journals, 250 (41·1 per cent) in subspecialty surgical journals and 194 (31·9 per cent) in non-surgical journals. The mean(s.d.) impact factor of the journals was 2·0(1·1). Only 14 manuscripts (2·3 per cent) were published in journals with a higher impact factor than that of BJS.

Conclusion:

Rejection of a manuscript by BJS does not preclude publication, but rejected manuscripts are published more often in surgical subspecialty journals and journals with a lower impact factor, although the occasional exception exists. Copyright © 2010 British Journal of Surgery Society Ltd. Published by John Wiley & Sons, Ltd.

Introduction

The British Journal of Surgery (BJS) receives more than 1100 article submissions each year. The acceptance rate of submitted manuscripts ranges between 15 and 18 per cent. The editors of BJS accept manuscripts for publication based on various factors, including originality of the research, clinical importance and usefulness of the findings, methodological quality, and supposed interest of the readership of the journal. Selection of manuscripts for publication is a difficult and complex process. Manuscripts undergo a peer review process that is supposed to be able to identify their strengths and weaknesses. Although recommendations made by reviewers have considerable influence on the fate of papers submitted to journals, agreement between reviewers appears to be little greater than would be expected by chance alone1, 2. Other critics have also argued that the process of peer and editorial review can be subjective, biased, and fails to prevent publication of methodologically weak studies3. All of this suggests that manuscripts rejected by BJS are not always of inferior quality and may well be published in other high-quality journals.

Hitherto it has not been known what happens to manuscripts rejected by BJS. In particular, information is lacking with respect to how many rejected articles are subsequently published elsewhere. Equally, no information is available regarding the type of journal in which such rejected papers may be published and how these journals compare with BJS in terms of impact factor. The authors believe that an answer to these questions might help researchers when choosing a journal for resubmission of a manuscript. Furthermore, this could provide more information on the quality of the BJS review process. A retrospective cohort study was therefore undertaken to track the publication fate of rejected BJS manuscripts.

Methods

All manuscripts submitted to BJS between 1 January and 31 December 2006 were identified from the Manuscript Central electronic database. This list contained manuscript number, title, corresponding author (name and initials), date of submission, handling editor and rejection notes. Abstracts of the manuscripts were also available. Manuscripts that were rejected by the editors in 2006 were included in the study; book reviews and letters (correspondence section) were excluded. To identify whether rejected manuscripts had been published in another journal, a PubMed search (http://www.ncbi.nlm.nih.gov) was performed between December 2008 and February 2009, spanning the time period 2006–2009 and using the corresponding author's last name and initials. Obviously, the search was limited to manuscripts published from the year of submission onwards. The title and abstract of the retrieved manuscripts for each author were compared with the title and abstract in the BJS 2006 submission database. In a few cases, where the title and the authors had changed considerably, some judgement had to be used to determine whether the published article was the one that had originally been submitted to BJS.

The proportion of declined manuscripts published elsewhere was calculated by dividing the number of rejected manuscripts subsequently published in other journals by the total number of rejected BJS manuscripts. The journals in which the rejected manuscripts were published were categorized into general surgical journals, subspecialty surgical and ‘surgical interest’ journals, and other (non-surgical) journals. The 2006 impact factor of the journals in which the rejected BJS manuscripts were published was retrieved from the Web of Science of the Institute for Scientific Information (ISI) (http://www.isiwebofknowledge.com) and compared with the BJS impact factor for 2006. The time interval between date of submission of the manuscript to BJS and (e-)publication elsewhere was calculated.

Papers that are rejected by BJS and subsequently published elsewhere may be of good quality and cited frequently. It is of interest to compare the number of citations to manuscripts rejected by BJS with manuscripts accepted by BJS. On 31 March 2009, the number of citations to 20 papers rejected by BJS and subsequently accepted by other journals with high impact factors was scored by using data from the ISI Web of Science, and compared with the number of citations from 20 randomly selected manuscripts accepted for publication in BJS in 2006. The time interval between publication and web search was calculated.

Statistical analysis

A database of all rejected BJS papers was generated in Microsoft^® Excel 2004 for Mac (Microsoft Corporation, Redmond, Washington, USA). Mean(s.d.) and median (range) values with standard deviation or range were calculated. Analysis of categorical data was done with the Pearson's χ² test. P ≤ 0·050 was considered statistically significant.

Results

Proportion of manuscripts published elsewhere

A total of 926 submitted manuscripts could be identified that were subsequently rejected in 2006 by BJS. The majority of rejected manuscripts were original articles (n = 780, 84·2 per cent), followed by reviews (n = 52, 5·6 per cent), randomized controlled trials (n = 37, 4·0 per cent), systematic reviews (n = 24, 2·6 per cent), leading articles (n = 20, 2·2 per cent) and meta-analyses (n = 13, 1·4 per cent). Some 496 manuscripts (53·6 per cent) were rejected after peer review, whereas 430 (46·4 per cent) were rejected after early editorial manuscript screening for appropriateness without being sent out for review.

The PubMed search revealed that, as of March 2009, 609 (65·8 per cent) of these 926 manuscripts were published in other journals and 317 (34·2 per cent) were not published. The mean(s.d.) time lapse between submission to BJS and (e-)publication elsewhere was 13·8(6·5) months. There was no relationship between the type of manuscript and likelihood of publication elsewhere. A higher proportion of manuscripts rejected by BJS after peer review were published elsewhere compared with manuscripts rejected after editorial screening alone: 71·0 per cent (352 of 496) versus 59·8 per cent (257 of 430) (P < 0·001, χ² test).

Type of journal and impact factor

The 609 manuscripts submitted to BJS, then rejected and subsequently accepted for publication by other journals, were published in 198 different journals. Some 165 manuscripts (27·1 per cent) were published in general surgical journals, 250 (41·1 per cent) in specialty surgical journals and 194 (31·9 per cent) in other journals. Table1 shows the distribution of the 165 papers published in general surgical journals, and Table2 the 250 papers published in subspecialty surgical and ‘surgical interest’ journals.

Table 1. General surgical journals in which manuscripts rejected by the British Journal of Surgery were published

Journal	No. of manuscripts	Impact factor*
Values in parentheses are percentages. * Impact factor 2006 derived from the Web of Knowledge of the Institute of Scientific Information (http://www.isiwebofknowledge.com). † Other journals in which no more than one manuscript was published. NA, not applicable (no impact factor listed for 2006).
World Journal of Surgery	46 (27·9)	1·765
American Journal of Surgery	19 (11·5)	2·101
Annals of the Royal College of Surgeons of England	15 (9·1)	0·720
ANZ Journal of Surgery	15 (9·1)	0·881
Langenbeck's Archives of Surgery	14 (8·5)	1·496
Archives of Surgery	9 (5·5)	3·058
Surgery	9 (5·5)	2·977
Journal of the American College of Surgeons	7 (4·2)	2·813
Surgeon	6 (3·6)	NA
Surgery Today	6 (3·6)	0·698
Acta Chirurgica Belgica	3 (1·8)	0·348
American Surgeon	2 (1·2)	2·101
Annals of Surgery	1 (0·6)	7·678
Other†	13 (7·9)	–
Total	165 (100)	–

Table 2. Manuscripts rejected by the British Journal of Surgery and published in subspecialty surgical and ‘surgical interest’ journals

Journal	No. of manuscripts	Impact factor*
Values in parentheses are percentages. * Impact factor 2006 derived from the Web of Knowledge of the Institute of Scientific Information (http://www.isiwebofknowledge.com). † Other journals in which no more than five manuscripts were published. NA, not applicable (no impact factor listed for 2006).
Annals of Surgical Oncology	23 (9·2)	3·329
European Journal of Surgical Oncology	22 (8·8)	NA
Surgical Endoscopy	22 (8·8)	1·969
Colorectal Disease	21 (8·4)	NA
European Journal of Vascular and Endovascular Surgery	21 (8·4)	2·156
International Journal of Colorectal Disease	18 (7·2)	2·006
Journal of Surgical Oncology	16 (6·4)	2·183
Diseases of the Colon and Rectum	13 (5·2)	2·442
Digestive Surgery	12 (4·8)	1·397
Hepatogastroenterology	11 (4·4)	0·756
Journal of Surgical Research	10 (4·0)	2·038
HPB	7 (2·8)	NA
Journal of Laparoendoscopic & Advanced Surgical Techniques	6 (2·4)	0·718
Hernia	6 (2·4)	NA
Journal of Vascular Surgery	5 (2·0)	3·311
Other†	37 (14·8)	–
Total	250 (100)	–

No impact factor was listed in the ISI Web of Knowledge for 51 (25·8 per cent) of 198 journals. The mean(s.d.) impact factor for the remaining 147 journals was 2·0(1·1). Only 14 manuscripts (2·3 per cent) were published in journals with an impact factor higher than that of BJS (Table3).

Table 3. Journals with a high impact factor in which 14 manuscripts rejected by the British Journal of Surgery were published

Journal	Type of manuscript	Impact factor*
* Impact factor 2006 derived from the Web of Knowledge of the Institute of Scientific Information (http://www.isiwebofknowledge.com). RCT, randomized controlled trial. † Manuscript rejected after editorial review without external review; ‡ manuscript rejected after external peer review.
Gut	1 original‡	9·002
Annals of Surgery	1 original†	7·678
Critical Care Medicine	1 original‡	6·599
Clinical Infectious Diseases	1 original†	6·186
Journal of Thrombosis and Haemostasis	1 RCT†, 1 review‡	5·138
Journal of Nuclear Medicine	1 original‡	4·986
Psychoneuroendocrinology	1 original†	4·850
Breast Cancer Research and Treatment	3 original†‡‡	4·671
British Journal of Cancer	2 original†‡, 1 systematic review‡	4·459

Number of citations of rejected manuscripts

The median number of citations for 20 manuscripts published in journals with a high impact factor (between 3·900 and 9·000) as of March 2009 was 3 (range 0–22). The median time elapsed since publication of these manuscripts was 22 (range 1–30) months. The median number of citations for 20 papers accepted by BJS in 2006 was 6 (range 1–24), although the time elapsed since publication was longer at 24 (range 20–31) months.

Discussion

This study investigated the fate of manuscripts rejected by BJS. Most (65·8 per cent) were eventually published just over a year (mean 13·8 months) after rejection. The proportion published elsewhere is somewhat higher than that for other journals that have reviewed the outcome of submissions declined by their editorial team. These studies have reported rates between 41 and 58 per cent4–8. No similar studies have been undertaken for general surgical journals.

The number of manuscripts accepted elsewhere is probably an underestimate, because PubMed does not cover all sources. Only about a quarter of all medical publications are listed in PubMed5. Furthermore, the time between rejection by BJS and the date the PubMed search was performed may not have allowed sufficient time for some rejected manuscripts to reach the stage of subsequent publication. However, most articles rejected by the Annals of Internal Medicine were eventually published within 2·5 years (mean latency 552 days), so it is probable that those manuscripts in the present study not published after 38 months are inactive. The editorial turnaround time of BJS (interval between submission of the manuscript and first decision) is, on average, 22 days, which helps the author to resubmit quickly to another journal. Finally, manuscripts may evolve in terms of author order or title, making the paper difficult to identify through searches. This may also lead to an underestimation of the proportion of rejected manuscripts subsequently published elsewhere.

Early editorial screening of manuscripts for appropriateness for publication is a BJS policy. In a randomized trial by Johnston and colleagues9, this method was found to be appropriate as it decreased the time between manuscript submission and publication decision, and reduced the burden on reviewers while having a minimal impact on the quality of accepted manuscripts. The manuscripts that were rejected solely after editorial review, that is, without being sent for peer review, had a small chance of being published elsewhere. This probably reflects their poor quality. It supports the current BJS policy of editorial screening.

Only 14 manuscripts (2·3 per cent) were published in journals with a higher impact factor than that of BJS (4·304 at the time of the study). Eight of these 14 papers were published in journals with an impact factor only marginally higher (range 4·459–4·986). This is lower than the 7·5 and 10 per cent rates of publication in higher-impact journals reported for other journals6, 10. Most authors probably feel that they should first attempt publication in a journal with a relatively high impact factor. Upon rejection, the manuscript is then usually sent to a journal with a lower impact factor6, 7, 11, 12. This is likely to be because authors assume that such journals have higher acceptance rates. The mean impact factor of the journals that published the 609 manuscripts rejected by BJS was 2·0, significantly lower than the BJS impact factor of 4·304 (currently 4·921).

Most manuscripts were published in surgical specialty journals. Perhaps many papers that are rejected by a general surgical journal such as BJS meet this fate because they are judged to be too specialized. The subsequent shift to a subspecialty journal may therefore be desirable and rational. The data support previous observations that manuscripts rejected by general medical journals are also published more often in journals that serve a smaller readership or research community and are cited less frequently7, 13.

It is difficult to draw firm conclusions about the preference of authors to publish their work in a particular journal, although the data in Tables1 and 2 give some insight. Among the 165 papers accepted for publication in general surgical journals, most were published in USA-based general surgical journals such as the World Journal of Surgery and the American Journal of Surgery. There seems to be no clear preference of authors for a particular subspecialty surgical and surgical interest journal. Most of these journals are gastrointestinal or surgical oncology oriented. This probably reflects the fact that the number of vascular and trauma papers submitted to BJS is relatively small.

The reasons for rejection of the 14 manuscripts that were published in journals with a higher impact factor were reviewed. It was felt by the handling editors that some papers dealt with topics that were beyond the scope of BJS and/or the topic was supposed not to be of interest to a typical BJS reader. Inferior methodological quality was another feature. This has been studied by Lee and co-workers14, who demonstrated that manuscripts of high methodological quality, such as randomized controlled trials and those with larger sample sizes, are more likely to be published. Finally, novelty and originality is deemed to be important by BJS editors, and others15. Six of the 14 manuscripts were rejected after editorial review without external review, which is roughly the same proportion as for all papers submitted to BJS. Further research into the factors that relate to publication or rejection of manuscripts might help both authors and editors to improve the scientific quality of journals.

A journal's impact factor is often regarded as associated with the scientific quality and impact of the papers published. However, scientific impact is a multidimensional construct that cannot be measured adequately by any single indicator such as impact factor16. To estimate the scientific impact of papers rejected by BJS, the number of citations from 20 papers were retrieved and compared with that of 20 papers accepted by BJS in 2006. Papers rejected by BJS were cited less frequently than those accepted by BJS, despite publication in journals with a relatively high impact factor (3·900–9·000). It is important to realize that a high citation count for an article is not necessarily a sign of quality or approval of the results or interpretation of a study, as citations may be critical of, or contradict, an article16, 17.

Authors of manuscripts that are rejected by BJS may find reason for hope in the finding that rejected articles had a good chance of being published in other peer review journals. BJS encourages authors to incorporate suggestions made by reviewers and editors into their manuscript. This may improve the quality of papers and, even if a paper is rejected, this process may add value in terms of future publication of rejected papers in different journals.

It would be interesting now to evaluate the trajectory of those papers that were published in BJS. How many of them were first submissions? How many had previously been rejected by direct competitors of BJS? Such information would allow the editors to assess the relative prestige of surgical journals. This information might be useful not only for the editors but also for the readers and potential authors.

Acknowledgements

The authors acknowledge Mrs Harriet MacLehose, Mr Gavin Stewart and Mrs Bryony Urquhart from the BJS editorial office for their help with retrieving the data and completing the database. B.P.L.W. received a BJS Editor Assistant Bursary in 2008 and expenses to permit travel to the monthly editorial meetings. The authors declare no other conflict of interest.