Learning health systems: A review of key topic areas and bibliometric trends

Abstract Introduction The emergent field of learning health systems (LHSs) has been rapidly evolving as the concept continues to be embraced by researchers, managers, and clinicians. This paper reports on a scoping review and bibliometric analysis of the LHS literature to identify key topic areas and examine the influence and spread of recent research. Methods We conducted a scoping review of LHS literature published between January 2016 and May 2020. The authors extracted publication data (eg, journal, country, authors, citation count, keywords) and reviewed full‐texts to identify: type of study (empirical, non‐empirical, or review), degree of focus (general or specific), and the reference used when defining LHSs. Results A total of 272 publications were included in this review. Almost two thirds (65.1%) of the included articles were non‐empirical and over two‐thirds (68.4%) were from authors in the United States. More than half of the publications focused on specific areas, for example: oncology, cardiovascular care, and genomic medicine. Other key topic areas included: ethics, research, quality improvement, and electronic health records. We identified that definitions of the LHS concept are converging; however, many papers focused on data platforms and analytical processes rather than organisational and behavioural factors to support change and learning activities. Conclusions The literature on LHSs remains largely theoretical with definitions of LHSs focusing on technical processes to reuse data collected during the clinical process and embedding analysed data back into the system. A shift in the literature to empirical LHS studies with consideration of organisational and human factors is warranted.


| Review methods and eligibility criteria
We carried out a scoping review of LHS literature published between January 2016 and May 2020. A scoping review synthesises and maps the research on a particular topic to identify key concepts and gaps in the literature. 14 Searches were carried out in two online databases (PubMed and Scopus) using the term "learning health* system*". Publications were downloaded into EndNote and duplicates were removed.
The review team (CP, ZM, AV, LAE, GK, CLS) screened retrieved publications in full to determine their inclusion. Publications were included if they were (a) in the English-language, (b) peer-reviewed publications (journal articles, review articles, journal commentaries, editorials, books, or book chapters), and (c) had a key focus on LHS. Publications containing no substantive discussion of the LHS concept (eg, only superficially used the term in the abstract, conclusion, or among the keywords) were excluded. Five percent of retrieved publications were independently screened by the entire review team to ensure consistent inclusion. Disagreements about the eligibility of publications for inclusion were resolved through review team consultation, with YZ and JB resolving any outstanding disagreements.

| Data extraction
Publications were reviewed in full to identify publication data (the journal in which the paper was published, keywords, author names, and the country of residence of the corresponding author), the type of study (empirical, non-empirical, or review), degree of focus (general or specific), and, if a definition was included. This information was extracted and stored in Microsoft Excel 365. The number of citations for each publication was also recorded, as reported by Web of Science in September 2020, and was used as a measure of impact or influence, rather than as an indicator of quality.

| Analysis
Publication data were collected to determine the most active journals, authors, and countries publishing in the LHS field. For degree of focus, a publication was coded as "general" when LHS concepts were discussed without discussing a specific program, health condition, or topic area. Publications coded as having a specific degree of focus were later aggregated to create counts per key topic area (ie, to delineate how many included publications focused on ethics). Key topic areas were also examined through an analysis of keywords. Keywords of publications were identified in EndNote X9, then cleaned and checked for consistency. Derivatives (eg, health system, healthcare systems, health systems) were amalgamated. The keyword data were analysed for frequency counts and co-occurrence using VOSviewer v.16.14 (https://www.vosviewer.com/). The citation average for the field was calculated by citation count per paper divided by the number of years elapsed since publication.
Where a definition of LHS was used, we extracted the definition and referenced citation. Definitions were analysed using text analysis to determine word use frequency (ie, similar terms were grouped together, such as improvement, improve, improving). LHS definition references were graphically presented using Gephi, version 0.9.2. The nodes in the network were the publications in the review, as well as any output that was cited as a definition. Ties were LHS definition citations (eg, publication X cited publication Y to define an LHS). The most influential publications in the network were assessed using in-degree calculation 15 (ie, the number of times a reference was cited by different publications when defining an LHS).

| RESULTS
The search yielded 430 publications. After the removal of duplicates, there were 420 publications, 148 of which did not meet the inclusion criteria. This resulted in 272 publications on LHSs included in our analysis.

| Defining learning health systems
Most publications (n = 193, 70.4%) provided a definition of an LHS.
Ninety-five different citations were used when providing a definition of an LHS. More than half of the definition references were used more than once (n = 56, 58.9%). Seven definitions did not have citations and the wording could not be identified as belonging to a previous paper; these were classified as definitions developed by the author(s). Common terms used when defining LHSs are listed in Table 3. See Table 4 for the most-cited sources of LHS definitions.

| Key topic areas
Among the 272 included publications, 1067 unique keywords were identified. Of these, 69 were used at least five times. Co-occurrence of keywords (ie, keywords used together on a publication) is visually depicted in the network of co-occurring keywords (Figure 2). In this network, each circle represents a keyword and each line indicates co- Oncology's CancerLinQ initiative. 22 CancerLinQ is a large database that collects information from electronic health records of cancer patients to foster an oncology learning community across the United States; it has been described as a rapid LHS. 23 This example demonstrates the inherent need for integrated data platforms to support the adoption of LHSs and the importance of rapid implementation and implementation science in this sphere. 24 As to strengths and limitations, this study highlighted key topic reviews. There is a need to capture research written in languages other than English to obtain a more widespread view of the LHS concept and its potential. This is particularly important given the divide between high-, middle-, and low-income countries when it comes to big data and personalised medicine, 25 which are key components of LHSs.

| CONCLUSION
Despite ongoing interest in the concept of LHS as demonstrated by the growing body of literature, this study shows that considerations of LHS concepts are largely focused on technical processes rather than the organisational and human factors necessary to facilitate an LHS. The LHS literature warrants a shift in focus to move the field from the conceptual LHS to take-up and adoption, and from technical processes to emphasising the complexity of human factors in LHSs.

ACKNOWLEDGEMENT
This study was funded by NHMRC grant nos. 9100002 and APP1176620.