Purposes of peer review: A qualitative study of stakeholder expectations and perceptions

Stakeholders might have diverging or conflicting expectations about the functions that peer review should fulfil. We aimed to explore how stakeholder groups perceive peer review and what they expect from it. We conducted qualitative focus group workshops with early‐, mid‐, and senior career scholars, editors, and publishers. We recruited participants following a purposive maximum variation sampling approach. To identify purposes of peer review, we conducted a thematic analysis. Stakeholders expected peer review (1) to assess the contributions of a manuscript, (2) to conduct quality control, (3) to improve manuscripts, (4) to assess the suitability of manuscripts for a journal, (5) to provide a decision‐making tool for editors, (6) to provide feedback by peers, (7) to curate a community, and (8) to provide a seal of accreditation for published articles. Stakeholders with different roles and tasks in the peer review process differed in the value they attached to the functions of peer review. Early‐ and mid‐career researchers valued social and feedback functions of peer review, while senior career researchers and editors expected it to instead perform a technical assessment of manuscripts and serve as a decision‐making tool. Publishers expected peer review to assess the suitability of manuscripts for their journals and to provide a seal of accreditation. This revealed a potential tension between functions of peer review. Stakeholder expectations are shaped by how stakeholders perceive their own roles both in relation to the peer review process and within their scientific community.


INTRODUCTION
In scholarly communication, peer review is the accepted best practice for determining which papers are published in academic journals. Peer review describes a scientific appraisal process in which manuscripts are evaluated for quality, originality, validity, and possible impact by other experts (Tennant et al., 2017). This involves stakeholders interacting with one another. Authors submit their manuscript and potential revisions to a journal.
Reviewers evaluate the manuscript, make a recommendation to the editor regarding its publishability, and potentially request revisions (Schroter, 2006). Editors and their editorial teams assign reviewers, oversee the review process, and decide if a manuscript is publishable (Bakanic, McPhail, & Simon, 1987). Publishers support the review process by managing publishing operations and by providing resources (Medici, Pracucci, & Modugno, 2017).
A mutual understanding of what authors, including early-, mid-, and senior career researchers; reviewers; editors; and publishers expect from peer review determines their satisfaction with the process (Glonti, Boutron, Moher, & Hren, 2019). Depending upon their relationship with the process, stakeholders might have diverging or conflicting expectations about peer review. This currently presents an information gap. While there is a wealth of research on author and reviewer attitudes, little research qualitatively explores and compares stakeholders' expectations of the functions of peer review. One line of research consists of largescale and standardized surveys that gauge senior author and reviewer attitudes. These surveys revealed that senior authors and reviewers expected peer review to fulfil a variety of purposes, including manuscript improvement (Mulligan, Hall, & Raphael, 2013;Nicholas et al., 2015;Rowlands & Nicholas, 2006;VanTassell, McLemore, & Roberts, 1992;Ware, 2008b), quality control (VanTassell et al., 1992), originality assessment (Ware, 2008a), selection of the "best" manuscripts for publication in journals (Mulligan et al., 2013;VanTassell et al., 1992;Ware, 2008a), detection of fraud (Mulligan et al., 2013;Ware, 2008a), and provision of a seal of approval (Nicholas et al., 2015;Ware, 2008a). Another line of research is concerned with early-to mid-career researchers' expectations of peer review. This research indicates that early-to mid-career researchers valued peer review mainly for self-oriented reasons.
Reflecting upon their role as authors, this group expected peer review to help them improve the quality of their manuscripts (Merga, Mason, & Morris, 2018;Nicholas et al., 2018;Rodriguez, 2014). Early-to mid-career researchers further expected that acting as reviewers themselves should enhance their writing and research skills and help them identify opportunities for future collaborations (Merga et al., 2018;Merry, Jarvis, Kupoluyi, & Jomama Lual, 2017;Rodriguez, 2014). Within this context, the role of editors was perceived as crucial. Because early career researchers might not be sufficiently familiar with the review process, they expected editors to provide guidance beyond the official journal guidelines and to synthesize and filter reviewer comments (Allen et al., 2019;Merga et al., 2018).
Assuming that peer review is a social process that goes beyond technical scrutiny (Glonti et al., 2019) Researching scholars' motivations for engaging in peer review, Zaharie and Osoian (2016) conducted semi-structured interviews with early-and senior career scholars in Romania. For most of their interviewed senior researchers, reviewing meant a reciprocal duty. In contrast, early career researchers, who still had to build reputation, thought of peer review as a means of career advancement (Zaharie & Osoian, 2016). To understand the needs and practices of faculty, Harley, Acord, and King (2010) interviewed early-to senior career researchers, publishers, and librarians. This study revealed that stakeholders unanimously valued peer review as quality and relevance assessment. Peer review was believed to serve as an indicator of the quality, relevance, and likely impact of scholarly work (Harley et al., 2010). A qualitative exploration of biomedical journal editors' views regarding the tasks of reviewers by Glonti et al. (2019) indicated that editors perceived peer review as a tool that aids their publication decision. When viewed as a whole, these studies contribute to an in-depth understanding of stakeholder expectations. However, no single study comprehensively explores how stakeholder groups differ in their expectations of peer review, which would help highlight how perceived purposes of peer review might depend on the role stakeholders have in the process. Our aim was to address this gap by conducting focus groups that explored how stakeholders, including early-, mid-, and senior career scholars; reviewers; editors; and publishers across academic disciplines perceive peer review and what they expect from it. Focus group discussions focused on the purposes and challenges of peer review and how the latter could be addressed. This manuscript reports the purposes of peer review.

Peer review as a social process
A traditional belief in academic research is that all claims to knowledge should be subject to an impartial and technical scrutiny (Bedeian, 2004;Ziman, 1984). As a process of scientific appraisal that is intended to distinguish between good and bad manuscripts, peer review is often referred to as embodying this principle, and peer review has been called upon to be free of nepotism, personal influence, and affiliations to institutions or schools of thought (Hirschauer, 2010;Merton, 1974). However, a wealth of research has shown that review processes are prone to biases stemming from author or reviewer characteristics and

Key points
• In scholarly publishing, peer review is expected to fulfil a multitude of purposes, some of which are contradictory.
• Stakeholders with different roles and tasks in the peer review process differ in the value they attach to different functions of peer review.
• Peer review serves social functions that go beyond a technical scrutiny of manuscripts.
• There is a tension between formative and summative functions of peer review. suffer from limitations in consistency (Smith, 2006). Following this, some scholars have suggested suspending normative expectations towards peer review in favour of investigating the social dimensions that underlie it (Hirschauer, 2010). Taking a sociology of knowledge perspective, these scholars assume that scientific activities are inherently social activities in the sense that scientific techniques and knowledge are created and curated by human actors (Mendelsohn, 1977). With its characteristic exchange of reviewer comments and author revisions (Hirschauer, 2010), peer review is believed to provide the institutional framework in which scholars define knowledge through negotiation and eventual consensus (De Rond & Miller, 2005). Doing so, peer review fulfils a number of social functions. It is a form of symbolic interaction whereby generalized views of the academic community influence how knowledge claims are validated and presented (Hirschauer, 2010), thereby guiding the curation of research within this community. Therefore, peer review also reinforces concepts of good-quality research and adequate scholarly behaviour in a particular community, hence aiding socialization processes, relationships, and sanctions within a community.

MATERIALS AND METHODS
We conducted focus groups with stakeholders involved in journal review processes. The methodological approach is described below.

Sampling and recruitment
Based on a purposive maximum variation sampling approach (Breen, 2006), we selected information-rich cases that covered all stakeholder groups involved in the review process. Doing so, we intended to generate a comprehensive understanding of the purposes of peer review from all angles. Recruitment continued until saturation across sampling criteria was reached (Table 1). Early career researchers were defined as scholars within their PhD or the first years after the award of their PhD (including PhDs and postdocs). Mid-career researchers included scholars who had completed the first years after their PhD and were in transition to independent researchers (including lecturers and research fellows). Senior career researchers comprised experienced scholars who conduct research and related activities independently or in a leadership role (including assistant professors, professors, and emeriti professors). Editors were defined as professionals who held an editorial role with an academic journal (including assistant or associate editors, section editors, editors-in-chief, managing editors, and other editorial board members). Publishers included professionals holding a position at an academic publishing outlet (including publishers, publishing directors).
Participants were sampled from a variety of sources, including the researchers' professional networks, university staff webpages, journal editorial board webpages, and LinkedIn profiles. Potential participants were contacted via email and were sent a consent form and information sheet, which included details about the investigators, study aims, workshop processes, code of conduct, and confidentiality. Following a snowballing approach, stakeholders who agreed to take part in the study were asked if they could refer further potential participants.

Data collection
The focus groups were undertaken by AS and JC in spring 2019. All focus groups were conducted face to face and involved 2-hour long workshops with three to seven participants each.
Informed consent was obtained from all participants before each workshop. A semi-structured topic guide was used to moderate discussions (Supporting Information). The guide was informed by a review of the literature and further refined after an informal pilot. Participants were asked to discuss the purposes of peer review and what they believed to constitute "good-quality" review. All focus groups were audio-recorded. An assistant was present and took field notes. Audio files were imported to NVivo 12 and transcribed by AS. At the time of the study, AS was a PhD researcher who had completed training in conducting qualitative expert interviews. JC, who had previously been involved in peer review processes as an author and reviewer, supervised AS.

Data analysis
Transcripts were thematically analysed, which involved exploring patterns and themes, thereby allowing the identification of implicit and explicit purposes of peer review (Braun & Clarke, 2006). The thematic analysis was conducted in two steps and entailed both deductive and inductive elements. The first step was driven by the study's theoretical and analytical interest and involved creating a preliminary codebook based on the topics of the semi-structured guide (AS and JC). In a second step and supervised by JC, AS read and reread the focus group transcripts and coded their topic, that is, the purposes of peer review that The codebook was updated and revised continuously (see Supporting Information). Transcripts were coded in the order in which the respective focus groups took place. Coding was repeated until saturation across purposes of peer review was reached, defined as the point where no additional information, insights, or themes were forthcoming from repeated coding (Ando, Cousins, & Young, 2014;Merriam & Tisdell, 2015). Where codes appeared in a patterned way, they became a theme (Vaismoradi, Jones, Turunen, & Snelgrove, 2016).

RESULTS
A total of 37 participants were recruited for seven focus group workshops, including 5 early career researchers, 4 mid-career researchers, 17 senior researchers (of which 13 participants also held an editorial position), 3 editors who did not hold an academic position at a research institution, and 8 publishers ( Table 2). Of note, the groups of senior career scholars and editors are in large part overlapping in this study as most recruited senior career scholars held an editorial position with a journal and because most editors held an academic position with a research institution. For this reason and because senior scholars and editors (including those senior scholars without an editorial position and editors without an academic position) showed no differences in their expectations or experiences, we will refer to these participants as one stakeholder group. A variety of purposes was deduced in our analyses that stakeholders expected peer review to fulfil. These purposes revolved around eight key themes. Peer review was expected (i) to assess the contributions of a manuscript, (ii) to conduct quality control, (iii) to improve manuscripts, (iv) to assess the suitability of manuscripts for a journal and its readership, (v) to provide a decision-making tool for editors, (vi) to provide feedback by peers, (vii) to curate a community, and (viii) to provide a seal of accreditation for published articles. The framing and importance of these purposes varied considerably across stakeholder groups. This was indicated, first, by how strongly stakeholder groups agreed on certain themes constituting purposes of peer review (Figs. 1-3) and, second, by how often individual themes were mentioned across groups (Table 3).
Early career researchers and mid-career researchers showed consistent agreement in their expectations towards peer review. To this group, the most valued purposes of peer review were manuscript improvement, receiving feedback from their peers, and community curation. Senior researchers who held an editorial position, senior scholars who did not hold an editorial position, and editors who did not hold an academic position also were consistent in their expectations towards peer review. This group had a technical understanding of peer review. Rather than curating a community, peer review was expected to deliver a technical assessment of the quality and the contributions of a manuscript, which ultimately was expected to inform the editor's decision upon a manuscript's publishability. Publishers expected peer review to assess the suitability of manuscripts for publication in their journals and to provide a seal of accreditation to published contents.

Contributions assessment
Something that all stakeholder groups expected from peer review was that it should evaluate which contributions, if any, a paper makes to the existing research. Stakeholders perceived this to be an important purpose. Without it, stakeholders feared that researchers might be inclined to publish more and to split up their results into shorter articles as they believed promotion procedures in academia to be characterized by an increasing focus on quantity over quality in scientific publications: "To get promoted you are supposed to simply say 'I publi- shed a high number of papers in top journals'" (Professor and editor, mathematics) Stakeholders believed some of the scientific literature to be unnecessarily repetitive or irrelevant. However, there were different opinions about the role that peer review should play in this setting. Senior researchers and editors strongly agreed that peer review serves a gatekeeping function by evaluating the publishability of a paper based on its contributions. One common idea was that peer review should assess if a paper adds something novel. This was related not only to the research question but also to theoretical concepts and methods. One participant explained: "[…] a manuscript takes a problem that it addresses. Maybe someone has already done it. Maybe in a different way.
So maybe this paper's problem has been solved, so in some respect this paper is not new. But nonetheless, maybe the methods add something new." (Emeritus professor and editor-in-chief, mathematics).
It was highlighted that a paper might make different types of contribution, and this could lead to confusion. Reflecting upon their role as reviewers, senior researchers highlighted that a paper  there is a whole other research going on that they are obviously not connected to, or should not be concerned  Some senior researchers and editors postulated that reviewing the novelty or originality of a manuscript might be too narrow and that peer review should instead evaluate a manuscript's relevance or usefulness in a broader context. Publishers shared the latter view and pointed to the recent trend of journals operating on a so-called "soundness-only" review policy, meaning an evaluation of the scientific soundness or rigour of an article, not its originality (Spezi et al., 2018). Some early-and mid-career researchers also believed that peer review should assess a Figure 3 Publishers' expectations.

Suitability assessment
Publishers, as well as senior scholars and editors, repeatedly expressed that one of the key purposes of peer review is to determine if manuscripts are suitable for publication in particular journals.
"The main purpose of peer review is to test if this is something that fits to the journal. And if it is also interesting for the readership of the journal" (Senior scientist and assistant editor, biosafety).
For publishers, the importance of this purpose was tied to their self-perception as service providers to the academic community. Noting that journals have a role in curating research, publishers believed that their ability to attract readership depends on how well they understand the needs of their community and translate these into their own aims, thereby fleshing out a profile that is congruent with the community's interests. Within this context, publishers expected peer review to ensure that only contents are published that meet the aims of their journals.
"You want reviewers to be […] commenting on the significance or the likely significance of the work. Is it appropriate for the journal and hence interesting for its readership? If so, why?" (Publisher, life sciences) Publishers and editors feared that inexperienced authors might not understand or could misinterpret an editor's decision to reject their manuscript if deemed unsuitable for a particular journal.
"So authors would be surprised […] when their article is accepted or rejected based on the stated aims of the journal and then that's when you may bring up a perceived bias or lack of appropriateness of your work for a particular community." (Publisher, multi-disciplinary) "They certainly do not understand that if they get a deskrejection, the paper does not suit the journal. They think that it is bad science." (Professor and editor-in-chief, political science) Interestingly, early-and mid-career researchers, who were more likely at that point to possess experience of being an author or reviewer than experience as an editor, did not identify this theme as an important purpose of peer review.

Decision-making tool for editors
Senior researchers, editors, and publishers were explicit about expecting that peer review serve as a decision-making tool for editors. Senior scholars and editors highlighted the role of editors as decision makers, who decide if a manuscript is accepted or rejected for publication. When asked what constitutes goodquality peer review, one managing editor explained: Did it help inform the [publication] decision? Because sometimes you have only one review but that might actually be helping you but other times, its 'so, what do I do with these reviews?'" (Managing editor, life sciences) They explained that, in order to make this decision, editors would rely on an expert assessment carried out by reviewers. This was rationalized in different ways. Senior scholars and editors stressed that editors might not have the resources to conduct a detailed assessment of every manuscript submitted for publication. They also emphasized that, even though they are experts within their field, editors could not be familiar with the topic of every manuscript.
Consequently, it was concluded that they would rely on reviewers who, as experts on the topic of a given manuscript, assess its strengths and weaknesses and make a recommendation upon its publishability. Some pointed to the challenge that, for interdisciplinary submissions, editors might have to recruit a number of reviewers with different backgrounds and ask each of them to evaluate specific aspects of a manuscript. This group further shared the concern that peer review serving as their decision tool might be at odds with the expectations of other stakeholders, particularly with authors expecting that peer review improve the quality of their manuscripts.
Publishers agreed with the notion that a reviewer's assessment of a manuscript should inform publication decisions, whereas early-or mid-career researchers did not identify this theme as an important purpose. Those who did discuss peer review serving as a decision tool shared their concern about editors relying too heavily on reviewer comments: "In my experience editors always seem tired and rely a lot on whatever view the reviewers have summarized.
[…] It has been fairly rare that I have encountered an editor who seems to have given a lot of their own thoughts on a paper" (Lecturer, social sciences) Accreditation Stakeholders commonly believed that peer review awards a seal of accreditation to published contents. Peer review was understood as a vetting process by which papers are scrutinized against the publishing criteria of a journal. Even though all stakeholder groups agreed on this notion, they had diverging ideas about how and why peer review should accredit publications. safeguarding function for their own research. By scrutinizing the quality of their work prior to publication, peer review ensures that it meets the standards of the community before being publicly available. In contrast, senior researchers and editors did not only take the perspective of authors but also that of readers.
They stressed that peer review is responsible for awarding credibility for published research and for functioning as "a guarantor to outsiders", meaning that published work, which has undergone peer review, can be trusted to have been scanned by experts. Senior scholars, as well as editors, concluded that they would have doubts about the quality and relevance of publications that have not (yet) been peer reviewed, for example, preprints. This group believed the seal of accreditation to be particularly important where research findings were to be communicated to nonacademic audiences. Peer review was trusted to make a difference by filtering out poor-quality manuscripts and by improving the quality of accepted papers.
To publishers, assigning trustworthiness to published research was particularly important. When asked what the key purpose of peer review was, one publisher answered: One editor shared that, in such cases, it would be the responsibility of journals to not only provide the requirements necessary for reviewers to scrutinize manuscripts but also to verify the credibility of published contents.

Manuscript improvement
To early-and mid-career scholars, manuscript improvement was one of the main purposes of peer review. Taking the perspective of authors, they commonly expressed that peer review should provide a framework that enables peers, who are not working in the same niche but who are familiar with their research, to make an independent assessment of their paper. Reviewers should also give detailed instructions on how to improve a paper: "I mean peer review is there for improvement. It is there to assist people to do a better job. […] It is there for […] determining what is weak and what is strong and hopefully eliminating the weaknesses and keep strengthening whatever needs to be strengthened" (Lecturer, computer sciences) When being asked which aspects of their manuscript should be improved, scholars gave examples that related to intellectual efforts and less to the technical parts of their research. Peer review should ensure that recent literature was considered, enhance the understandability of argumentation, and ensure that conclusions were supported by the data.
Senior scholars and editors agreed that peer review helps improving manuscripts and expressed that they appreciate peer review as a second opinion on minor points of their work. However, also taking into account their experience as having acted as reviewers and editors, senior scholars reflected upon a potential tension between purposes of peer review. They argued that manuscript improvement was not a primary purpose but more a welcomed side effect of having peers evaluate their work, which provides the grounds for the editor's publication decision. This also appeared to influence their own reviewing behaviour. In particular, the more senior scholars in this group reported that they would only look for sufficient reasons to recommend publishability and do not perceive it as their responsibility to help authors improve their manuscripts.
"[…] I may just look for a sufficient reason to reject because the narrow purpose of reviewing is giving the editor information about whether this is publishable. If I find a fatal flaw or a serious flaw, I am not going to read the whole paper as a reviewer. I'm not about to go through the typos and the point is, it may take a lot of work to turn [the manuscript] into a good paper and that is not my job." (Emeritus professor and editor-in-chief, mathematics) Publishers acknowledged that this function is crucial for their authors.

Quality control
All stakeholder groups agreed on the general notion that peer review should perform some form of quality control by filtering out papers that do not meet the quality standards of a particular journal or academic community. Publishers, researchers, and editors expected reviewers to comment on the soundness of submitted work to ensure that only papers are published in their journals that are sound. Related to this, all groups expressed that they trusted peer review to be able to detect scientific errors and, based on this, either reject respective papers for publication or help authors in eliminating flaws in accepted papers. Relating quality control to the purpose of assigning a seal of approval, in particular, senior researchers and editors expressed that publications should only be publicly available after peer review.

Feedback by peers
Unlike publishers and senior scholars, early career and mid-career researchers believed that an important purpose of peer review is to receive feedback from their peers. With their limited experience of acting as reviewers, this group expressed that their primary concern as authors is to receive critical appraisal by other scholars. They expected peer review to enable peers to provide honest and independent criticism. One mid-career researcher explained: "It's a huge benefit to be able to have people who are independent from you to […] say that's an important piece, that needs changing […] or that's wrong. (Lecturer, computer sciences) This was rationalized in different ways. In particular early career researchers explained that, being in the early stage of their career, they need to develop a sense about the relative quality of their research. Defining the quality of one's own work relative to the work of other scholars was believed to be a trial-and-error process of submitting work to a journal and having peers evaluate it against the journal's publication criteria. In this setting, publication criteria of journals were perceived as proxies for the quality standards of the academic community. Other scholars explained that they occasionally might be in a situation where they and their co-authors have immersed so deeply in a research project that they would no longer be able to judge if their manuscript is understandable to external readers. Some scholars also shared that, often, they would feel alone or isolated and might not receive any helpful feedback from their direct colleagues or supervisors. One early career researcher explained: "Many times, we are very alone in our scientific community. Like I for example, I cannot present my work in front of anybody in my institute. And so yeah I do sometimes do that [submitting to journals speculatively to get reviews], I am just collecting anonymous reviews just because I would like to get some feedback on my work." (Postdoc, political sciences) Assuming that peer review quality correlates with journal prestige, scholars reported that they speculatively submit their papers to the most prestigious journals within their field, where there were relatively poor chances for their manuscripts to be published but where they receive helpful reviews.

Community curation
To publishers, community curation did not appear to be of great importance, even though they stressed that peer review is crucial to the academic endeavour of scientific communities. All other stakeholders agreed on the notion that peer review describes a set of practices that contribute to the creation, curation, and enhancement of their scientific community. Reviewing was perceived as a service to the community and meant contributing to a greater good, which was defined as the general scientific endeavour. It was believed that, in order to become full members of the scientific community, scholars would have to engage in peer review. Senior researchers stressed that community curation also had an educational purpose. They reported that they encourage early-and mid-career researchers in their research groups, particularly PhDs and postdocs, to write reviews because reviewing would teach them about their field, both in terms of the research as such and about the appropriate scholarly behaviour. Peer review in this sense serves a socialization function by providing a mechanism whereby junior scholars become functioning members of the scientific community and take on its values and behavioural patterns.
Most When participants were asked about training on peer review, early-, mid-, and senior researchers agreed that it would be helpful for training to take place formally and either be provided by publishers or be part of doctoral training and transferable skills courses at university. However, both groups stressed that peer review training would be most effective when delivered informally, for example, through co-reviewing. This was referred to senior scholars or supervisors asking their PhD students and postdocs to perform their peer reviews, either with or without their assistance and either with or without the identification of co-reviewers to the journal, and was perceived as helpful by both groups. Scholars reported that, because peer review is neither institutionalized nor financially compensated, it depends on scholars volunteering their time. In an academic system that is at the same time characterized by incentives that encourage selfinterest, there might be scholars exploiting the reviewing system by publishing their own research without reviewing the work of peers. Reflecting upon their own reviewing and publishing behaviour, scholars concluded that the number of reviews scholars do should be proportionate to the number of their manuscripts submitted for publication.
It was explained that, typically, every manuscript receives two to three reviews so that, as a rule of thumb, scholars should perform two to three times more reviews than the number of manuscripts they submit in a given period.

DISCUSSION AND CONCLUSION
This study provided an in-depth exploration into how stakeholders, including early-, mid-, and senior career scholars; reviewers; editors; and publishers, across academic disciplines perceive peer review and what they expect from it. It was also important to understand whether stakeholder expectations differed depending on their relationship with the process.
One key finding of this study was that peer review serves a social function. Unlike existing research into stakeholder expectations, this study identified community curation and feedback from peers as themes that stakeholders perceived as important purposes of peer review. Particularly valued by scholars in the early stages of their academic careers, these themes relate to social processes that describe a socialization function whereby junior scholars become members of their community. Previous research into stakeholder expectations has not identified these themes as purposes of peer review. Besides this, our study confirmed the findings of previous studies. This included assessing the contributions of a manuscript (Ware, 2008a(Ware, , 2008b, quality control (Mulligan et al., 2013;VanTassell et al., 1992;Ware, 2008a), improvement of a manuscript (Mulligan et al., 2013;Nicholas et al., 2015;Rowlands & Nicholas, 2006;VanTassell et al., 1992;Ware, 2008b), assessing the suitability of a manuscript for a particular journal (Mulligan et al., 2013;VanTassell et al., 1992), providing a decision-making tool for editors (Glonti et al., 2019), and providing of a seal of accreditation or approval for published contents (Nicholas et al., 2015;Ware, 2008a).
Another key finding was that stakeholders with different roles and tasks in the review process differed in their understanding of and the value they attached to functions of peer review. To early career researchers and mid-career researchers, the most valued purposes of peer review were manuscript improvement, receiving feedback from their peers, and community curation.
Being in the early stages of their career, their experiences with peer review were in large part limited to receiving reviews for their own research and some occasional reviewing of the work of their peers. Showing uncertainty about the quality and relevance of their work relative to the standards of their community, junior scholars expected peer review to serve a feedback mechanism on the basis of which they could improve their manuscript and be taught about appropriate behaviour and values of their communities. In contrast to more junior scholars, senior scholars and editors had a technical understanding of peer review. Rather than curating a community, peer review was expected to deliver a technical assessment of the quality and the contributions of a manuscript, which was ultimately expected to inform the editor's decision upon a manuscript's publishability. This expectation was tied to the self-perception of this group as the final decisionmakers who rely on the expert assessment by reviewers. Having experienced peer review from a multitude of perspectives, senior scholars and editors appeared to be aware that their expectations might be incompatible with the needs of (junior) authors. Publishers expected peer review to assess the suitability of manuscripts for publication in their journals and to provide a seal of accreditation to published contents. Perceiving themselves as service providers to the community, publishers expected peer review to serve as a mechanism which ensures that only manuscripts are accepted for publication that meet their community's quality standards and interests, thereby enabling them to build a brand, maintain readership, and serve their community's needs.
Concluding, we found that stakeholder expectations are shaped by how stakeholders perceived their own roles both in relation to the peer review process and within their scientific community.
A further key finding was the potential tension between individual purposes of peer review. Most pronounced was the tension between formative functions and summative functions.
Junior scholars expected peer review to help improve their manuscripts by providing feedback and detailed guidance on how to enforce strengths and eliminate weaknesses. In contrast, senior scholars and editors believed the purpose of peer review to be providing the editor with information about whether a manuscript is publishable. Even though this would not per se prevent reviewers from making an effort in helping authors improve their manuscript, the more senior scholars reported that they would only look for sufficient reasons to recommend acceptance or rejection instead of providing guidance to authors.
As a final key finding, this study revealed an inter-relatedness between purposes of peer review. Reviewers assessing the contributions, suitability, and methodological quality of a manuscript helps inform publication decisions by editors and publishers. At the same time, this assessment can help authors improve their manuscript if communicated in a constructive way, for example, by including examples. This procedure has been labelled as peers providing feedback to authors. On a more abstract level, the processes of peers assessing manuscripts and providing feedback to authors were believed to reinforce the identity and values of a scholarly community, thereby ensuring that junior scholars become members of that community. This study has several limitations. As for the sampling approach, there is a chance that there were unconscious selection biases in the way in which we recruited participants. We tried to address this by using a set of recruitment criteria. Further, because focus groups were conducted face to face, it is possible that those participants who lived or worked close to the workshop location were more likely to attend the focus group workshops. In an attempt to keep geographical biases in participant recruitment to a minimum, we offered participants to choose from a selection of date and time slots and reimbursed any travel expenses. Nevertheless, there remains a Global North bias, with stakeholders located in any other locations than the UK and Switzerland being underrepresented. Because academic publishing practices differ across geographic regions (Collyer, 2018), the homogeneity in our study likely limits the generalizability of our results. An analysis of review experiences that authors submitted to SciRev.sc showed that authors located in countries where English is the first language rate the peer review process less positively than authors located in other countries, indicating that authors in English-speaking countries have different expectations of the process and are critical of aspects that do not meet their expectations (Huisman & Smits, 2017).
Moreover, due to the limited resources of this study, the size of our sample was relatively small. This might have limited this study's ability to generate a comprehensive understanding of the purpose of peer review from the angles of relevant stakeholder groups. An additional limitation might be the fact that the validity of self-reported attitudes and behaviour can be subject to biases or errors in recall, inaccurate interpretations, inability to respond accurately, and socially desirable responding (O'Sullivan, 2008).
Consequently, there might be discrepancies between what focus group participants reported to expect from peer review and what they actually believed the purposes of peer review were. Finally, because the researcher's experience and judgement influence how data are collected, analysed, and interpreted, qualitative data analysis always includes some degree of subjectivity. To alleviate this, analysis and interpretation were based on a codebook and exemplary participant quotes as provided in Supporting Information. Despite these limitations, our study is the first to comprehensively explore and compare how different stakeholder groups involved in journal peer review processes perceive the purposes of peer review. Having identified patterns and trends of stakeholder expectations, this study aids a mutual understanding of stakeholder needs, which can ultimately increase their satisfaction with peer review.