The next generation of peer reviewing


Figure 1.
Figure 2.
Figure 3.

Science as we know it wouldn't work without peer review, yet this integral part of the scientific enterprise is surprisingly absent from the formal education of many graduate students. Unless given specific guidance from their academic advisor in the art of composing timely, salient, and succinct feedback, most students have to develop these skills “on the fly”. For such a crucial endeavor, there ought to be an alternative. To better understand the perspectives of peer-reviewed journal editors on this issue, the Ecological Society of America's (ESA's) Student Section recently carried out a survey among the Editors-in-Chief of nine prominent ecology-related journals (WebPanel 1).

To put the current reviewing situation in context, we estimated how many reviews are typically needed for each published paper based on results from that survey and information from ISI's Journal Citation Reports. Assuming that two reviewers are required for each submission and that submissions to a given journal may have previously undergone review elsewhere, a minimum of 6 to 8 reviews are required for each published paper. Additionally, according to ISI, the number of articles published by journals in the category “ecology” more than doubled in a 13-year period, from <7000 in 1997 to >14 000 in 2009. The linear rate of increase is somewhere around 600 additional papers published each year. Fourteen thousand papers, when multiplied by 6 to 8 reviews for each, is equivalent to between 84 000 and 112 000 reviews per year, increasing at a rate of at least 3600 reviews per year (all else remaining equal).

About 25% of ESA's 10 000 members are students. Given the continuing increase in the number of articles and their associated reviews, there is a need for more peer reviewers. Students and early-career researchers represent an untapped population of potential reviewers, ready and willing to fulfill that need. To assess young researchers' opinions on the peer-review process, we conducted a second online survey (see WebPanel 2) – this time of early-career ecologists, from undergraduates to pre-tenure faculty (n = 193). Among these respondents, the main reason given for reviewing papers was a sense of professional responsibility (83%), followed by the belief that reviewing manuscripts helped students develop both critical reviewing and writing skills (76%). Thinking specifically about what makes for a “good” manuscript – its scientific content, logical presentation, writing style, and conclusions – will provide practice in carrying out and sharing quality science. Of the students who responded to the survey, 71% had never declined a review request and 69% felt honored to review papers, but didn't often have the chance unless they were given articles by their advisor or had already published a number of papers themselves. Even then, not all advisors mentor their students on how to conduct a review.

We therefore propose a new “mentor–student” reviewing system, to be instituted at the level of the professional society, in which reviewing becomes a standard part of the graduate school experience. A central database could be developed in which students, paired with their advisors, could register as reviewing teams, each providing a list of areas they felt comfortable reviewing. This database would be available to editors of all ecology journals, who could then monitor how many reviews student–advisor pairs have completed and locate appropriate reviewing teams. We have presented this plan to a number of Editors-in-Chief, who are supportive of the idea.

While acknowledging that securing enough reviewers for a given paper can be difficult (where 1 is not difficult and 5 is very difficult, the mean response was 3), several of the editors we spoke to expressed concern that many students would not yet have enough experience or perspective to provide quality reviews. However, these editors noted that the reviews that they receive from students (which currently make up approximately 8% of assigned reviews) are timely and comprehensive, and would be happy to increase student reviewing opportunities if done in conjunction with a mentor. The proposed collaborative reviewing system would help to alleviate concerns about the quality of reviews, while instituting a formal means to teach students how to read critically, think constructively, and write clearly. The advancement of any scientific discipline is rooted in the quality of the research produced; helping students to recognize quality in the writings of other scientists and to learn how to provide constructive criticism will be to the collective benefit of members of our discipline.