A framework for analysing learning health systems: Are we removing the most impactful barriers?

Abstract Introduction Learning health systems (LHS) are one of the major computing advances in health care. However, no prior research has systematically analysed barriers and facilitators for LHS. This paper presents an investigation into the barriers, benefits, and facilitating factors for LHS in order to create a basis for their successful implementation and adoption. Methods First, the ITPOSMO‐BBF framework was developed based on the established ITPOSMO (information, technology, processes, objectives, staffing, management, and other factors) framework, extending it for analysing barriers, benefits, and facilitators. Second, the new framework was applied to LHS. Results We found that LHS shares similar barriers and facilitators with electronic health records (EHR); in particular, most facilitator effort in implementing EHR and LHS goes towards barriers categorised as human factors, even though they were seen to carry fewer benefits. Barriers whose resolution would bring significant benefits in safety, quality, and health outcomes remain. LHS envisage constant generation of new clinical knowledge and practice based on the central role of collections of EHR. Once LHS are constructed and operational, they trigger new data streams into the EHR. So LHS and EHR have a symbiotic relationship. The implementation and adoption of EHRs have proved and continues to prove challenging, and there are many lessons for LHS arising from these challenges. Conclusions Successful adoption of LHS should take account of the framework proposed in this paper, especially with respect to its focus on removing barriers that have the most impact.


| INTRODUCTION
Learning health systems (LHS) were developed as a vehicle to advance clinical safety and health research and improve patient-centred care, with the added goal to more fully realise the benefits and potential of electronic health records (EHR). [1][2][3] The learning component of LHS can occur at multiple levels, including the personal level for individual actors (when educating doctors, patients, caregivers, etc), the team and organisational level (when revising work practice and care pathways), and at the whole system level (when the LHS demonstrates holistic learning). This work primarily focuses on learning that uses knowledge derived from collections of EHR as a digital support system to introduce programmed improvements (when introducing new workflow or decision support) or as automated learning (as promised by the introduction of AI within clinical systems).
Our exploration of LHS began by discovering that much work describing LHS is not actually identified as such within the LHS domain: Something we described as the research community awareness challenge. 4 To aid researchers in appropriately identifying their works within the domain, this research group provided a taxonomy describing the nine types of LHS commonly observed in the literature and a unifying framework showing how each type positions within the larger learning health organisation. 1 In this work, we focus on the notion of barriers, benefits, and facilitators, their identification, impact, and application. Barriers are those things that inhibit implementation and use of a particular technology or system, such as health information systems (HIS) and LHS. Benefits are the positive outcomes realised by resolving a barrier through engaging a facilitator. Facilitators are those interventions described as easing the burden of implementation and use of a technology such as EHR. A facilitator is targeted towards resolving one or more related barriers.
EHR are the enabling technology for LHS. Considerable research has consolidated knowledge on barriers, benefits, and facilitators for EHR implementations. [5][6][7] However, in the LHS domain, the picture is considerably different. We believe a gap exists in the research literature in that with the exception of passing reference to LHS barriers, no such consolidation of knowledge on barriers and benefits for LHS adoption could be found. There is reference to a link between EHR barriers and LHS barriers 8 albeit without any analytical framework. It is necessary to develop approaches for analysis and mitigation of barriers in order to facilitate LHS, just as LHS should benefit patients through the conduct of more precise, individualised medical practice. 1 This paper presents a literature review used to close the research gap and develop such an analysis framework. The main focus of this work is therefore the development and application of that framework for use in analysing and consolidating barriers and facilitators that may be encountered in LHS implementations and determining whether benefits already identified from implementations of EHR are similar to those that may result from implementing LHS.
In the information systems literature, one often observed and widely used framework for evaluating implementation challenges is ITPOSMO. ITPOSMO identifies seven dimensions for exploring the gap between a system's design and the reality of its implementation: information, technology, processes, objectives, staffing, management, and other factors. 9 The ITPOSMO framework was originally proposed for evaluating e-government projects, but it has since been used to evaluate EHR projects and help explain why health information systems succeed or fail. 10 Our work extends the ITPOSMO framework to support the comparative analysis of barriers, benefits, and facilitators for both EHR and LHS. This led to a new framework, which we call ITPOSMO-BBF, that was then used to explore the literature on barriers, benefits, and facilitators for LHS.
Hence, this paper presents results of an investigation into those things that hinder or enable technology use in health care environments. In particular, it presents a framework for classifying and analysing barriers and facilitators and contrasting facilitators with the degree of benefit authors have ascribed to them. One of our key objectives is to help those implementing LHS to identify their own major barriers and optimise their efforts with facilitators that will not only address those barriers but whose impact will maximise implementation success.
The rest of this paper is organised as follows: Section 2 introduces the concept of LHS and presents the background and context of the research problem. Section 3 presents the methodology, particularly the framework developed for addressing the problem. The results of applying that method to the literature are presented in Section 4, along with discussion, before we conclude the paper.

| LEARNING HEALTH SYSTEMS
EHR are repositories of retrospective, current, and prospective patient data stored in digital form, with the intention of supporting efficient, quality health care service delivery. 11 Those implementing EHR have long complained of slow adoption and limited implementation success rates. [12][13][14][15] LHS represents a vision to transform health care. 16,17 This vision includes leveraging recent and ongoing developments in EHRs by developing new knowledge from the ever-increasing amounts of digital routine health data accumulating within them 17,18 and innovating learning from the slower population-based processes of evidencebased medicine (EBM), using rapid identification of new knowledge to deliver precision medicine. 17,19 Large collections of EHR are the fuel for LHS, giving statistical power to population level insights, and the EHR also provides the delivery mechanism for decision support tools that allow clinicians to diagnose and tailor treatment decisions using patient-level data in real time. 18 While EHR have existed in some form for more than 40 years, LHS have existed for less than one quarter of that, and the bulk of LHS research and development has only occurred since 2011. 1 EHR have become almost ubiquitous in health care, yet many hospitals and clinics in these countries still employ a mixture of paper and electronic records. [20][21][22][23] Among those EHR implementations in the hospital setting, many offer only limited functionality and occur as isolated islands of information, with separate EHRs tied to a particular ward, medical specialty, or care pathway. 24 There have been some spectacular failures to realise the initial promise of EHRs, as with the UK's National Program for IT wherein the NHS failed to deliver effective national hospital EHRs. 25 The design of many EHRs, particularly those from the United States, comes from health service billing software that can conflict with the needs of clinicians leading to both workflow and information presentation challenges in clinical use. 26 The data within medical records typically exist within a specific context, and there are some limitations on transferring it to another context, or work needed to support that transfer. 27 Many commercial EHR solutions have proprietary and cost barriers to integration with other systems and sources of health data relevant to the individual patient. 28,29 With the best intentions, many health care organisations have self-inflicted these issues by layering inflexible new technology over existing processes and procedures in the belief that EHR implementation meant simply replacing paper records with electronic systems. 23,30 The result too often has been implementations of EHR that have failed to improve quality of care, increase efficiency, or reduce health care costs. 31,32 Behind this is a lack of understanding about how clinicians interact with computers and disagreement as to whether such interaction enables or inhibits patient-centred care. 33 Successful implementation of new EHRs requires clinician-and userled processes that re-evaluate practices and procedures, with a requisite period of adaption and training for those who will use the resulting combination of new IT systems, documentation procedures, and clinical workflows. 34,35 There is growing recognition that LHS can exist at different scales from department to organisation and across multiple organisational boundaries, 1,2 but in all cases, they are limited by the functionality, quality, and interoperability of their underlying EHRs. 1,[36][37][38] Our research is motivated by the belief that an understanding of EHR barriers, benefits, and facilitators is essential for those implementing LHS because a successful EHR is a pre-requisite and because the challenges faced in EHR implementation are symptomatic of a range of deeper issues that impact on any major innovation in digital health.

| METHOD
Our literature review followed the systematic method for identifying benefits and barriers as described in Yao et al. 39 The method is divided into three phases: Search and selection, Categorisation, and Analysis.

| Search and selection
There were two parts to the literature search. For EHRs, an initial search used the search terms ("Electronic Healthcare Record" + "Barriers" + "Benefits" + "Facilitating Factors"). A second search replaced "Healthcare" with "Health" while preserving the remaining search terms.
We used a combined search engine drawing on the following repositories: Scopia, Science Direct, PubMed, EBSCOhost, DOAJ, and Elsevier.
Articles were included where they presented a scoping or systematic review of EHR implementations, providing analysis and discussion of all three elements: barriers, benefits, and facilitators. Those not meeting these requirements were rejected. For alignment with our LHS literature review, 1 articles more than 10 years old were also rejected.
For the literature on LHS, we used the search and inclusion criteria described in our previous paper, 1 which used the plain language search terms ("LHS" and "learning healthcare systems") to identify works that presented or proposed a solution self-identified within the LHS domain.

| Categorisation
Collected literature was divided into two sets: those providing statistical metrics for barriers and benefits and those that did not. The analysis framework was adapted from the method used in Greenhalgh et al 24 and is shown in Figure 1. Content Analysis and Thematic Analysis 40 and Formal Concept Analysis 41 were used to identify and classify the barriers and benefits described from implementing EHR and LHS.

| Analysis
Finally, we contrasted, compared, and analysed the barriers and benefits described in the EHR and LHS literature sets using an extension of the ITPOSMO methodology. This effort sought to identify similarities between facilitating factors from established EHR and LHS implementations. Precedents exist for expanding ITPOSMO to enable additional scope and functionality, including service quality analysis, 42 survey-based study of consumer and public perceptions, 43 and Socratic analysis of local e-Government. 44 Our ITPOSMO-BBF model used in this work adopts the aspects and dimensions of the original ITPOSMO framework 9 and combines these with a framework for analysing barriers and facilitators from Hassan et al. 45 The ITPOSMO framework is

| Thematic analysis of barriers and facilitators
Content and thematic analysis 40 was used to identify barrier, benefit, and facilitator themes, while also examining the context in which authors described themes and the overall frequency of their use within the collected literature. It was possible to categorise many of the themes identified within an overall summary or key theme: a grouping of related like themes. These key themes are described in Table 1, which lists the LHS and EHR literature from which that theme was identified, and whether it was discussed by the authors in the context of a barrier inhibiting implementation or use, or if appropriately engaged, would serve as a facilitator for new implementations. For example, in Table 1, key theme 3 concerns concepts of data standardisation, integration, and interoperability has been encountered as a barrier with respect to implementation of LHS and EHR in 14 and 11 papers, respectively. Resolving the issues of key theme 3 is also described as a potential facilitator of LHS and EHR in eight and five papers, respectively.  Table 1 maps the literature to the key themes that are significant with respect to barriers, benefits, and facilitators. Figure 6 then maps these key themes to the ITPOSMO elements, or domains, which form the basis of ITPOSMO-BBF that this paper uses as its analytical framework.

| Information and technology
A number of information and technology barriers were described by authors as shown in Table 1. These were most often issues that arose resulting from the stand-alone and bespoke nature of health systems, coupled with a lack of ability to combine systems or data in any simple, inexpensive, or meaningful way. Little effort has been expended in devising facilitators to resolving these barriers, even though there are important benefits that could be realised. Figure 7 shows that the most frequently discussed facilitator that authors considered would realise a number of the listed benefits was the seemingly simple act of integrating historic patient data so that any new system presented a complete picture of the patient.

| Skills and management
Most effort aimed at facilitating EHR has gone towards resolving "human factor" barriers, even though the literature only makes one reference to a benefit as shown in the skills and management (SM) element of ITPOSMO in Figure 9. Even in the case of technical support and training, these were described by authors in the context of developing skills and managing staff resistance, yet no single author reported that any of these facilitators was actually reducing staff resistance to technology or improving adoption rates for EHR. While there is strong interest directed towards resolving adoption issues, the facilitators presently being employed do not appear to have substantially resolved these issues, as the EHR adoption problem persists.

| Other resources and constraints
The other resources and constraints (O) element shown in Figure 10 reviews those attributes not falling within the first three ITPOSMO elements, including finance, maintenance, and user and systems support. While financial incentives that had been enshrined in the laws of countries like the United States was discussed by more than half of all EHR papers, only three mentioned the potential for penalties for non-adoption contained in the same legislation to be a facilitator.
Note, however, that none of the three led to the belief that the threat of penalties had helped an implementation of HIS.

| Barriers
Almost every author spoke of barriers or challenges in implementing EHR and LHS. Some present as barriers to entry, such as the high financial burden to implement and support HIS technology 6,54,70 and issues relating to complicated and inconsistent legislation. 7,55,56 Others present as barriers to success, including the need for data and systems standardisation 7,54,73,109 and issues with interoperability 5,7,53 or integration. 53,71 Many also describe barriers to organisational culture, 132 such as clinical users and patients expressing negative attitudes towards, and reservations with, the use of computing systems, resistance to potential changes in workflow that it was believed could disrupt the flow of patient care, 6,53,71 the impact on time management, 5,73,89 and opposition to the need for training and staff development in HIS use. 6,70,71 The potential for issues of data accuracy and integrity was said to arise from the use of HIS, with lasting impacts for patient care and health outcomes. 7,54,73,89 Concerns were expressed that systems producing, storing, exchanging, or amalgamating health data would negatively affect patient privacy and confidentiality, 5,53,70,109 with adequate data security seen as an unresolved primary challenge. 70,73,111 We also identified barriers that arose from overpromised HIS technology and budget overruns that add to negative perceptions of EHR and LHS. Barriers were discussed consistently for both EHR and LHS.

| Facilitators
Overcoming barriers has represented a significant challenge for EHR implementations, which is why organisations invest in efforts to identify and engage facilitators that will lead to successful implementations. 53,70,71 Some problems are seen as endemic and specific to the IT industry, 71 while others require change in health policy and legislation. 6,53,71 The literature demonstrates that patience, 71

| Benefits
Organisations contemplating HIS implementation do so with some intention of realising one or more benefits. 136 While it could be argued that any benefit improves the health of patients, even indirectly, this research found that benefits fall broadly into two categories. First, there are those that have a direct positive effect on health outcomes for patients. These include any that increase patient safety, 7,70 reduce harm from treatment or medication errors, 70,73 or improve the overall quality of health care. 6,54,73 Second, those seeking to improve some metric of health care delivery by increasing efficiencies and accountability, 6,54,70 or reducing waste and over-consumption of resources which accordingly increases overall capacity of the health care system. 6,70,109 A key point was that while LHS are seen to significantly benefit the conduct of many types of clinical trials, EHR were not discussed by any author as doing so to any similar degree. This, in spite of the fact that EHRs are the constituent components of all LHS.

| DISCUSSION
While the benefits of LHS build and significantly expand on those put forward for EHR, the barriers described for both are similar. This confirms LHS are inheriting unresolved challenges from EHR. For this reason, we chose to also investigate the factors identified as facilitating

FIGURE 6
Linking key themes to ITPOSMO (numbers shown identify themes listed in Table 1) EHR implementation to assess whether it is possible that these may aid in resolving LHS implementation challenges.

| Acceptance of EHR
A novel causal factor receiving attention is digital disruption. Digital disruption is a catch-all term for a range of related issues, described as the changes facilitated by the introduction of digital technologies that occur at a pace and magnitude that disrupt established ways of value creation, social interactions, doing business, and, more generally, our way of thinking. 137 One group in Australia have attributed the failure of more than half of all EHR system implementations to poor understanding and management of digital disruption, failure to understand and manage disruption to clinical workflows, the anxiety this engenders in staff, staff dissatisfaction, and the concerns for the quality and safety of care being delivered during the digital transformation. 138 Elements of digital disruption are seen in almost all of the barriers identified in this work. Facilitators, such as those that stipulate early staff involvement, staff training, and user championing, have been promoted for many years as mitigants for these barriers.

| The legal position for eHealth technologies
It is common for the hospitals implementing HIS to not even be party to the contract. 142 This is certainly the case when health departments and state organisations use centrally negotiated contracts (CNC). 142 CNC use impacts communication, placing multiple layers of organisations between clinical user and developer. 142 CNCs prevent HIS users from having proactive roles in negotiating terms in HIS vendor contracts, 26 exacerbating issues when they do occur as vendors cannot always be expected to make decisions in the best interests of the delivery of health care. 26 All health care procedures, tools, products, and services come with inherent risks, along with patient-harboured expectations of the level of quality and the standards of care. 143 Legislation on general liability in most countries makes reference to standards a patient may reasonably expect. 143 The questions that are much harder to answer are as follows: whether a duty of care is owed when issues arise out of the use of EHR and LHS; and who owes that duty of care to the patient?
While legislation and the common law Bolam Test in countries following the English legal tradition deal with situations where treating physicians breach a duty of care, it seems that no current legislation adequately contemplates or addresses general liability or duty of care issues arising from use of the multitude of eHealth products, from the seemingly simple EHR through to the multitude of complicated diagnostic medical devices, implantable technologies, software products, and prescribable mHealth apps. 143,144

| Enabling LHS
The barriers identified by this study represent the substantive issues impeding implementation and adoption of LHS. However, few authors are asking the right questions, such as how can health departments achieve subject matter expertise in all technology, legal, compliance, and privacy aspects? Nor are they recognising that these issues must be resolved in order to achieve a secure data repository to support LHS. 3,48 The literature contains abundant discussion of requirements or elements of a solution to one or more of the barriers, mostly revolving around calls for a new and common set of standards. 48,93 However, we conclude that the absence of universal and effective LHS shows these barriers remain unresolved. While LHS have been developed that are intended to learn from evidence-based literatures, patents, genomics, and other non-patient data, LHS can only be Data sharing to create large-scale data warehouses will only occur when clinicians and patients can trust that methods and systems used are protecting their privacy. Even then, ethics review processes may still impede the realisation of knowledge that can come from LHS.
It became clear during this study that the majority of facilitation efforts is focused on human factors and more specifically the mitigation of negative aspects arising from a general resistance to change.
More facilitators fell within the skills and management aspect of ITPOSMO-BBF than any other: an aspect domain that primarily deals with staff, the skills they possess, whether these are sufficient to using the HIS being implemented, and the structure and style of management within the health care organisation. 9,42 While many of these facilitators should lead towards a smoother and more successful implementation, only one benefit was directly ascribed by authors to this aspect domain: staff retention. As a result, the greatest mitigation effort has been focused in an area that on review has least amount of tangible benefit. Seeking input from all stakeholders who will impact and be impacted by the HIS is a significant factor in reducing resistance and increasing adoption of technology that has the potential to help many. 145  believe the primary goal in researching and designing new HIS is that they be used to make more precise diagnosis, select personalised treatment options, and improve overall health outcomes for patients.
We argue that of all the potential facilitators discussed, ensuring the widest and most appropriate group of stakeholders, including patients, may be the most significant factor for ensuring success in the implementation of any new health technology.
Learning health systems have potential to be one of the major computing technological advances in health care. The objective of this paper was to present an analysis from an investigation of the barriers, benefits, and facilitating factors identified from the literature of EHR and LHS. This was undertaken in order to create a basis for discussion on how best to expedite successful implementation and adoption of LHS for clinical application. In the methodology for this paper, we used an extension of the ITPOSMO methodology, we termed ITPOSMO-

BBF.
A key result is that although no prior research has analysed barriers and facilitators for LHS, EHR and LHS are seen to share similar barriers and facilitators. We also found that most facilitator effort in implementing both EHR and LHS involve addressing barriers that are best described as human factors, even though they carried the least number of author-identified benefits. The process and objectives barriers that would appear to bring the greatest number of patient outcome, safety, and quality benefits remain unresolved, requiring significantly more attention in order to ensure the goals of LHS can be realised. ing from these challenges for LHS. This paper is unique in presenting a framework for, and a systematic analysis using, a new framework based on extending ITPOSMO with consideration of benefits and consolidation of overall knowledge relating to barriers, benefits, and facilitators. This framework simply and succinctly relates barriers to facilitators and aids those implementing LHS to understand where the significant or important benefits can be realised. ITPOSMO-BBF will aid those implementing LHS to ensure their facilitation efforts can be focused in commensurate amounts to the degree of benefit that comes from resolving each set of barriers.