Process for selecting and implementing a manuscript management system: Experiences of a new peer-reviewed journal

Key points Global Health: Science and Practice, a start‐up peer‐reviewed journal, used a fourstage process to evaluate and implement a manuscript management system: define internal requirements, explore platform options, select platform of choice, and configure and test the platform Configuration and testing of the selected platform took longer than anticipated, mainly because we had to spend considerable time developing our submission and peer review workflows. The six key features of manuscript management of most value are: automating manuscript tracking; selecting peer reviewers; assigning alterative reviewers; sending automated letters and reminders; facilitating blinded peer review, and monitoring and evaluating journal processes

themselves. The case study also highlights, in our experience, key features of such a system that make it a worthwhile investment for not only large publishing operations but also small journals.

PLATFORM EVALUATION AND SELECTION PROCESS
As a new start-up journal, GHSP had no prior experience with manuscript management systems. In order to select the best platform to meet the journal's needs, we followed a four-stage process: • Stage 1: Define internal requirements From this formative research, we developed a number of requirements of a manuscript management platform, including the following: • Fee structure based on expected submission of 80-100 articles each year, with the potential for growth over time • Short set-up time and low start-up fees • Flexible length of contract • Ability to handle a range of article types • Ability to manage different review processes depending on article type (i.e. peer review vs. in-house editorial review) • Flexibility to customize evolving workflow solutions • Ability to deliver routine metrics, such as number of submissions and time to process articles through different points in the workflow • Ability to integrate a database of peer reviewers and match reviewers to articles through keywords • Ability for editorial staff to generate and modify email messages for authors and peer reviewers, to keep track of due dates, and to send out automatic reminders when appropriate Stage 2: Explore platform options Our discussions with colleagues in the publishing field directed us to a number of manuscript management systems to consider. In total, we considered 10 systems (Table 1). Starting around February 2012, representatives from each of the 10 vendors provided our staff an overview of their respective systems' key features and capabilities via teleconference. Some provided live demos of the systems, which helped us become familiar with the interfaces and gauge which interface the staff (as well as authors and peer reviewers) would be most comfortable with.
To more easily compare the features and offerings of the platforms, we created a spreadsheet in Microsoft Excel to collect information acquired about the 10 platforms. With this information, we discovered that many of the manuscript management systems provided similar types of core features and functionalityfor example, customization of the workflow per editorial office and per article type, delivery of routine metrics, and generation of automated reminders. We also realized, however, that the industry-leading systems provided the added value of continual investments in their systems both to ensure stability and to offer upgrades. They also offered more advanced functionality that could be particularly useful for journals as they grow their operations, for example, integration with data standards such as ORCID (registry of unique identifiers for authors).
Thus, we narrowed the options down from 10 to 3: Editorial Manager, EJPress, and ScholarOne. These were among the industryleading manuscript management systems, had similar features and cost structures, and were deemed affordable by GHSP.

Stage 3: Select platform of choice
One of the key differences between manuscript management systems is the design and usability of the interfacethat is, how different elements of the interface, including input controls (e.g. buttons, text fields, checkboxes, and dropdown lists), navigational components (e.g. pagination and search field), and informational components (e.g. tooltips, progress bar, and notifications), act to help authors, peer reviewers, and editorial staff accomplish their respective tasks. In fact, many of our colleagues in the publishing field advised us that the final decision of which system to select often comes down to the comfort level of the editorial staff with a particular interface. In addition, we appreciated that Aries offers free, regular upgrades to their clients. Furthermore, Aries representatives had proved to be exceptionally helpful to GHSP staff throughout the evaluation and implementation process. In

Stage 4: Configure and test platform
The first of four trainings on Editorial Manager took place on 17 July 2012, and manuscript upload began on 26 November 2012a period of more than 4 months (Fig. 1). On average, journals take 2 to 3 months to conceptualize, configure, and test the system. Our ambitious goal had been to complete set-up within 45 days, but we had a steep learning curve to overcome; although we had a long history in the global health field, we had limited formal experience in the scholarly publishing field (from the publishers' perspective). In particular, not only did the managing editors have to learn how to configure and use Editorial Manager but they also had to develop new workflows for receiving and reviewing manuscripts that fit the needs of the editorial staff and the goals of the journal.
In addition, we were simultaneously evaluating and implementing other needed systems, such as online hosting platforms and composition vendors. Established journals with workflows already in place could likely complete configuration of a new manuscript management system in a much shorter time frame.
As we were configuring Editorial Manager, we also were simultaneously managing and tracking new article submissions manually, using Microsoft Excel spreadsheets and personal e-mail accounts. The system's content load of this existing manuscript pipeline was complete in November 2012, and the system was operational for manuscript submission by month's end.

FEATURES OF MANUSCRIPT MANAGEMENT SYSTEMS CONSIDERED VALUABLE BY GLOBAL HEALTH: SCIENCE AND PRACTICE
While we used manual processes for only a few months prior to publishing the inaugural issue, and thus quantification of beforeand-after improvements are not feasible or practical, the benefits of using an online system to manage manuscripts and peer reviews were immediately apparent to us, as the journal moved to more automated processes. In particular, six features of such a system have been key to helping us manage the journal's processes, particularly with respect to peer review: (1) automated tracking of manuscripts through the peer review workflow; (2) selecting appropriate peer reviewers; (3) assigning alternative reviewers; (4) sending letters and reminder notifications; (5) facilitating the blinded review process; and (6) monitoring and evaluating journal processes. These core features are generally included in the standard set-up of any manuscript management system, but we refer specifically to Editorial Manager as that is the specific system with which we have experience.
FIGURE 1 Timeline of Global Health: Science and Practice's manuscript management system evaluation and implementation process.  Because GHSP's taxonomy is broad, the ability to search for reviewers using more detailed keywords used in the CVs facilitates and expands our ability to search for appropriate reviewers. An additional and potentially useful feature of the internal database is the ability to review the past performance of existing peer reviewers in terms of number of recent reviews, overall turnaround time, and quality of peer reviews. This feature can facilitate choosing reviewers who will provide high-quality reviews in the requested time frame. As a relatively new journal, we have limited data on our pool of peer reviewers, but as we continue to grow, we anticipate using this feature more fully.

Assigning alternative reviewers
With Editorial Manager, a number of main and 'back-up' peer reviewers can be assigned to a manuscript so that when a main reviewer does not respond to, or declines, an invitation to review within a defined period of time, an alternative reviewer can be invited automatically.
Given there are many manuscripts under consideration at different points throughout the peer review process (e.g. initial screening, peer review, and undergoing revision), automating peer reviewer assignments to the extent possible helps to ensure papers advance through the workflow as quickly as possible.

Sending letters and reminder notifications
Editorial Managers gives us the ability to send letters to authors and reviewers automatically with minor customization to existing templates.
The system can also be configured to send out reminders to authors, peer reviewers, and editors of upcoming due dates for assigned tasks at predefined intervals (e.g. 5 days before the due date). We find the reminder letters to peer reviewers, in particular, serve as helpful triggers to peer reviewers to complete the assigned task (or to request an extension). Because GHSP reviewers are volunteers who have many demands on their time, it is understandable for such tasks to get pushed back until prompted with a reminder.
Facilitating the blinded review process

Monitoring and evaluating journal processes
We strive to make the submission, peer review, and publishing process as efficient as possible, with the goal of initially screening submitted manuscript by the editorial team within 2 weeks of submission, completing peer review within 2-4 weeks, and issuing a final accept/reject decision on all manuscripts within 12 weeks of submission.
Editorial Manager provides many reporting tools to measure such information, including the length of time from submission to first accept decision, from submission to first revise decision, and from submission to publication. With this information, we can identify bottlenecks in the workflow and develop appropriate solutions.
Of course, no system comes without its challenges. There are some instances in which the functionality is not as intuitive or seamless as we would prefer, for example, cancelling out of editing a submission or registration and confirmation of new co-authors. However, Aries continually makes improvements to the Editorial Manager system, taking into account user feedback and requests. Furthermore, Editorial Manager provides a number of ways for journal publishers to get ongoing support, including a built-in, searchable help system, a third-party facilitated listserv, annual user group meetings in Boston, Tokyo, and London, and, most importantly, direct contact with a nominated account coordinator.