Scoping reviews in medical education: A scoping review

Abstract Objectives Over the last two decades, the number of scoping reviews in core medical education journals has increased by 4200%. Despite this growth, research on scoping reviews provides limited information about their nature, including how they are conducted or why medical educators undertake this knowledge synthesis type. This gap makes it difficult to know where the field stands and may hamper attempts to improve the conduct, reporting and utility of scoping reviews. Thus, this review characterises the nature of medical education scoping reviews to identify areas for improvement and highlight future research opportunities. Method The authors searched PubMed for scoping reviews published between 1/1999 and 4/2020 in 14 medical education journals. The authors extracted and summarised key bibliometric data, the rationales given for conducting a scoping review, the research questions and key reporting elements as described in the PRISMA‐ScR. Rationales and research questions were mapped to Arksey and O'Malley's reasons for conducting a scoping review. Results One hundred and one scoping reviews were included. On average, 10.1 scoping reviews (SD = 13.1, median = 4) were published annually with the most reviews published in 2019 (n = 42). Authors described multiple reasons for undertaking scoping reviews; the most prevalent being to summarise and disseminate research findings (n = 77). In 11 reviews, the rationales for the scoping review and the research questions aligned. No review addressed all elements of the PRISMA‐ScR, with few authors publishing a protocol (n = 2) or including stakeholders (n = 20). Authors identified shortcomings of scoping reviews, including lack of critical appraisal. Conclusions Scoping reviews are increasingly conducted in medical education and published by most core journals. Scoping reviews aim to map the depth and breadth of emerging topics; as such, they have the potential to play a critical role in the practice, policy and research of medical education. However, these results suggest improvements are needed for this role to be fully realised.


| INTRODUC TI ON
Over the last two decades, the number of knowledge syntheses published in core medical education journals has increased by 2620%. 1 Among these knowledge syntheses, there has been an even steeper rise in the number of scoping reviews published, with that number increasing by 4200%. The growth of scoping reviews is prompting discussions among scholars regarding the role of scoping reviews in the field and their potential to influence educational practices, policies and future research. However, despite this growth, the extant research on scoping reviews provides limited information about their nature, including how they are conducted, if they are funded, or why medical educators decide to undertake this type of knowledge synthesis in the first place. This lack of direct insight makes it difficult to know where the field stands and may hamper attempts to take evidence-informed steps to improve the conduct, reporting and utility of scoping reviews in medical education.
Scoping reviews are often cast as publications that 'map' the depth and breadth of the literature in a field. 2,3 Through such synthetic mapping, authors describe the main concepts that underpin a topic and can illuminate gaps in the literature. Scoping reviews are generally driven by broad, exploratory research questions and typically incorporate studies that employ a variety of research designs. 4,5 In their seminal article outlining a model for scoping reviews, Arksey Similar to medical education, the number of scoping reviews published in the health sciences is also on the rise. 9,10 To characterise these scoping reviews, researchers have recently authored discipline-specific 11 and cross-disciplinary 3,12 scoping reviews of scoping reviews. Collectively, these scoping reviews have identified methodological shortcomings and a need for improved scoping review reporting. While these studies are valuable, two are now several years old and the most recent review focuses solely on rehabilitation medicine, thus providing limited information on current approaches specific to medical education. What is more, the multi-disciplinary nature of medical education research suggests there may be variability in how researchers approach scoping reviews in our field, the topics they choose to review, and their purposes for using a scoping review. These differences warranted further exploration, which we undertook in the present scoping review.
As a relatively new field that includes researchers from a variety of backgrounds and research traditions, we believe that medical education scoping reviews are not immune to the methodological and reporting concerns found in other disciplines. 13 Thus, we propose that there is value in specifically examining scoping reviews in medical education and assert that in light of their exponential growth rate, the time is now to undertake such an analysis. In doing so, we hope to identify areas for improvement in the conduct and reporting of scoping reviews in medical education, thereby helping to ensure that those produced are relevant to and practical for application in the field (eg, useful for mapping the literature of a topic or identifying gaps in the literature). We do not aim to revisit the usefulness of scoping reviews or their methodological and epistemological considerations, as others have done in previous papers. 2,13 Rather, in the study reported here, we aim to characterise the extent, range and nature of scoping reviews published in medical education journals in order to identify areas for improving their conduct and reporting, and to highlight future research opportunities.

| ME THODS
Guided by the framework presented by Arksey and O'Malley 2 as updated by Levac et al, 5 we conducted a scoping review of medical education scoping reviews to examine and characterise the extent, range and nature of scoping reviews in core medical education journals.

| Identifying the research question
This scoping review is a component of a larger bibliometric analysis conducted by members of the author team. In the larger analysis, we broadly characterised knowledge syntheses in a core set of medical education journals and observed exponential growth in the number of scoping reviews. 1 This observation prompted three follow-on questions: (a) What are the characteristics of the scoping reviews and how can they be improved?; (b) What rationales do authors provide for undertaking a scoping review?; and (c) How do authors report the details of their scoping reviews?

| Identifying relevant studies
In the present study, we identified scoping reviews published during the time frame of the original study (1999-2019) 1 plus those reviews published in the first four months of 2020. On 26 March 2020 JC, an information scientist, queried PubMed using a combination of keywords and controlled vocabulary terms (See Appendix S1 for complete searches). 1 He reran this search on 27 April 2020 to capture any new citations. All retrieved citations and their metadata (eg, abstract, author names) were managed in GoogleSheets. 14 On 21 May, 2020, we obtained from Web of Science the number of times each review had been cited.
Searches were limited to 14 journals previously identified as core medical education titles, 15 15,16 we selected these titles because they are indexed in the Web of Science. This indexing enabled us to obtain citation data for individual reviews. Additionally, we erred on the side of search strategy accuracy by targeting this curated set of education-focused publications. Thus, we did not search PubMed broadly using keywords, which would have opened our search to all biomedical journals. As such, our search strategy ensured that retrieved studies were focused on medical education, especially because the indexing of medical education content is not comprehensive. Lastly, due to the nature of our research questions, our search was restricted only to scoping reviews.
We assembled a research team with expertise in knowledge synthesis methodology, information science, and medical education to guide the overall conduct of the review and our interpretation of the results. All team members had conducted previous scoping reviews.

| Selecting the studies to be included
To select studies for inclusion, we used an iterative approach. LM and JC independently reviewed the titles and abstracts of all citations. To facilitate calibration, they met four times during the process with the first meeting focused on creating a shared understanding of the criteria and then in the three subsequent meetings comparing selected citations and discussing any discrepancies. AA was available to facilitate any coding disagreements. Articles were included if they described the conduct of a scoping review. This initial determination was generally made based on the presence of the word 'scoping' or variants thereof and the mention of Arksey and O'Malley 2 and Levac 5 ; however, articles in which authors discussed scoping reviews as a methodological approach, but that did not describe undertaking an actual scoping review, were excluded.

| Charting the data
We created a data extraction tool that included and expanded upon the 22 items from the PRISMA-ScR checklist. 8 The extraction tool also included, but was not limited to, questions about the review's population, authors' rationale for undertaking the review, and the LM and JC piloted the data extraction tool by independently reviewing seven reviews and then comparing results. The extraction tool was modified based on the pilot and then used to extract data from the remaining full texts of articles. LM and JC independently extracted data from all included articles and met three times to discuss any discrepancies. AA was again available to discuss any coding disagreements and serve as tiebreaker.

| Collating, summarising and reporting results
We calculated descriptive statistics using GoogleSheets 14 to describe review characteristics. To describe the authors' rationales for undertaking a scoping review and their research questions, we conducted a thematic analysis. 17

| Reporting in alignment with PRISMA-ScR
Studies reported items from the PRISMA-ScR to varying degrees, and none included all of the items. Thirteen reviews (12.9%) 29   Additionally, because so many authors receive public funding for their reviews, and authors dedicate a substantial amount of internal resources (eg, faculty time, research assistant effort) to their conduct, it is critical that authors be good stewards of these resources by rigorously conducting and clearly reporting their work.

| D ISCUSS I ON
With this in mind, and to move the field forward, we focus our discussion on areas we feel are ripe for improvement in the conduct and reporting of scoping reviews.
Researchers have highlighted the importance of authors linking their rationale to their research questions. Doing so helps to guide the scoping review's overall conduct, especially to inform the inclusion/exclusion of evidence and data extraction processes. 5, 13 We observed some alignment in authors' rationales and their research questions/aims, but also noted room for improvement. For example, most authors' rationales and research questions mapped to those described by Arksey and O'Malley. 2 However, Levac criticised these rationales as being applicable broadly to a variety of knowledge synthesis methodologies and not necessarily specific to scoping reviews. 5 Thus, it is possible that this lack of specificity contributed to the suboptimal alignment between the reported rationales and Levac has declared it as essential. 5 On the other hand, stakeholder consultation is absent from the PRISMA-ScR. 8 Despite this lack of clarity, several scoping review authors appear to be leveraging stakeholders in creative and critical ways. For example, one scoping review, which addressed education to reduce health gaps between Aboriginal and non-Aboriginal peoples, integrated Aboriginal stakeholders throughout the entire conduct of the review. 106 We suspect that this review would have suffered without stakeholder involvement. As we are unaware of any firm guidance on stakeholder inclusion in scoping reviews within medical education, we propose an important step forward would be for the field come to some consensus on best practice guidelines regarding the role of stakeholders. At the very least, review teams should make explicit why the stakeholders were involved and describe the ways in which the review was strengthened as a result of their input. Doing so could help to ensure scoping reviews are optimised for medical education. As noted in Table 2, only two scoping reviews were registered and provided links to a submitted protocol. 35

| Limitations of this scoping review
This study has several limitations. First, scoping reviews on other health professions education topics may have been missed because we focused only on a core set of medical education journals. For example, it is possible we missed scoping reviews focused on medical education topics, but that were published in clinical journals. Future research might consider expanding the sample of journals to more broadly survey the field. Additionally, because we directed our inclusion/exclusion criteria on authors' use of the term 'scoping' in the title or abstract, it is possible that we inadvertently excluded relevant scoping reviews in cases where the authors did not use that specific terminology. This limitation may have implications for earlier scoping reviews in which authors may have been less familiar with the term. Third, to guide our data extraction, we used the PRISMA-ScR reporting guidelines, which was published in 2018. It is possible that authors publishing prior to 2018 were unaware of the importance of reporting many of the items in this reporting guideline, and thus did not include them (even if those data had been collected).
However, PRISMA, 119 which is the basis for the PRISMA-ScR, was published in 2009 and contains many of the same items. This suggests that while the PRISMA is not specific to scoping reviews, authors should have had familiarity with most of these items. Lastly, we did not register our scoping review protocol. Although we do not view this oversight as a study limitation, per se, protocol registration is considered a best practice. 124

| CON CLUS ION
Scoping reviews are increasingly conducted in medical education and published by almost all of the core journals examined here. Scoping reviews aim to map the depth and breadth of emerging topics; as such, they have the potential to play a critical role in the practice, policy and research of medical education. Although scoping reviews are not designed to result inaction-oriented recommendations, per se, the results from the present study suggest that improvements are needed for this role to be fully realised. These findings suggest room for improvement in the conduct and reporting of scoping reviews, including the alignment of research questions with rationales for undertaking a review, the publishing of protocols and the inclusion of external stakeholders in published works.

ACK N OWLED G EM ENTS
We acknowledge the hard work and dedication of all the included scoping review authors.

CO N FLI C T O F I NTE R E S T
The authors declare that they have no conflict of interest.

AUTH O R CO NTR I B UTI O N S
Lauren Maggio, Kelsey Larsen, Aliki Thomas, Joseph Costello, Anthony Artino made substantial contributions to the conception or design of the work; the acquisition, analysis, or interpretation of data for the work; the drafting of the work; its critical revision for important intellectual content; provided final approval of the version to be published; and agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

E TH I C A L A PPROVA L
This research does not contain any human subjects.

D I SCL A I M ER
The views expressed in this article are those of the authors and do not necessarily reflect the official policy or position of the Uniformed Services University of the Health Sciences, the Department of Defense, or the US Government.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data charting for all studies and the data charting tool has been