CFIR simplified: Pragmatic application of and adaptations to the Consolidated Framework for Implementation Research (CFIR) for evaluation of a patient‐centered care transformation within a learning health system

Abstract Introduction The Consolidated Framework for Implementation Research (CFIR) is a commonly used implementation science framework to facilitate design, evaluation, and implementation of evidence‐based interventions. Its comprehensiveness is an asset for considering facilitators and barriers to implementation and also makes the framework cumbersome to use. We describe adaptations we made to CFIR to simplify its pragmatic application, for use in a learning health system context, in the evaluation of a complex patient‐centered care transformation. Methods We conducted a qualitative study and structured our evaluation questions, data collection methods, analysis, and reporting around CFIR. We collected qualitative data via semi‐structured interviews and observations with key stakeholders throughout. We identified and documented adaptations to CFIR throughout the evaluation process. Results We analyzed semi‐structured interviews with key stakeholders (n = 23) from clinical observations (n = 5). We made three key adaptations to CFIR: (a) promoted “patient needs and resources,” a subconstruct of the outer setting, to its own domain within CFIR during data analysis; (b) divided the “inner setting” domain into three layers that account for the hierarchy of health care systems (i. pilot clinic, ii. peer clinics, and iii. overarching health care system); and (c) tailored several construct definitions to fit a patient‐centered, primary care setting. Analysis yielded qualitative findings concentrated in the CFIR domains “intervention characteristics” and “outer setting,” with a robust number of findings in the new domain “patient needs and resources.” Conclusions To make CFIR more accessible and relevant for wider use in the context of patient‐centered care transformations within a learning health system, a few adaptations are key. Specifically, we found success by teasing apart interactions across the inner layers of a health system, tailoring construct definitions, and placing additional focus on patient needs.


| The Consolidated Framework for Implementation Research
The Consolidated Framework for Implementation Research (CFIR) is a comprehensive implementation science framework compiled from 20 sources spanning 13 scientific disciplines. 1,2 It was developed to guide effective implementation of evidence-based practices from design to evaluation and is most commonly cited in evaluations of single interventions as opposed to complex transformation initiatives.
The CFIR comprised 39 constructs divided into the following five domains: 1. Intervention characteristics: aspects of an intervention that may impact implementation success, including its perceived internal or external origin, evidence quality and strength, relative advantage, adaptability, trialability, complexity, design quality and presentation, and cost. additionally, CFIR's combined breadth and depth is not always feasible for implementation in rapid time frames. Reported use of CFIR in the context of evaluation of multifaceted, patient-centered care transformations is rare; we know of only a handful of instances, recently reported. 2,[4][5][6] Here, we report adaptations we made to the CFIR in the evaluation of a multicomponent, patient-centered care transformation, designed to address the Quadruple Aim. 7,8 The Quadruple Aim of health care expands the Triple Aim's goals 7 of quality of care, patient experience, and cost savings by adding a provider/staff satisfaction component. We believe our adaptations to CFIR are generalizable to any patient-facing intervention implemented in an outpatient health care system. We describe our application and adaptation of the framework at each stage of our evaluation, along with the resulting outcomes derived from our approach.

| QUESTIONS OF INTEREST
• Is the Consolidated Framework for Implementation Research (CFIR) a useful, accessible tool to use when evaluating a complex intervention within a learning health system?
• Which adaptations are needed to make CFIR better suited to evaluating complex care transformations within a learning health system context?
• How might our experience of and lessons learned from using CFIR to evaluate a complex primary care transformation within a learning health system inform future evaluations of similar interventions?

| Setting
Primary Care 2.0 is a patient-centered care model developed by Stanford Primary Care leaders, with input from numerous stakeholders, including patients and families. 8,9 Table 1 lists the six modules and their key components which enable focus on health (rather than disease), flexibility in types of appointments, and provision of services beyond traditional primary care.
After extensive staff and provider training and planning, Stanford launched Primary Care 2.0 in June 2016 as a pilot in a new academic primary care clinic located in a community setting. The patient panel was initially about 1700 with a 3-year goal of 10 000. The clinic serves a patient population that is diverse both ethnically and socioeconomically, with 72% non-White patients and 14% publicly insured.

| Pragmatic CFIR application
We divided the evaluation of Primary Care 2.0 into three stages: evaluation design, qualitative data collection and analysis, and assessment of spread. Here, we report on our utilization of CFIR in the first two stages, as stage 3 data collection and analysis of spread are currently in process. At each stage, CFIR served as a skeleton around which we structured our evaluation questions, methods, and reporting.

| Stage 1: Application of CFIR in evaluation design (November 2015-May 2016)
In the first stage of our evaluation, we designed a plan appropriate for assessing key implementation outcomes in the context of a rapidly changing health system. We chose to frame our evaluation plan around CFIR as we expected it to be sensitive to most aspects of the complex implementation process. We used CFIR's five broad domains (intervention characteristics, outer setting, inner setting, characteristics of individuals, and process of implementation) to focus our evaluation measures by pairing them with the following implementation science outcomes: acceptability, adoption, appropriateness, feasibility, and adaptation. We used this combination of CFIR domains and implementation outcomes for each of the six modules of Primary Care 2.0 to drive our data collection, analysis, and reporting.

| Stage 2: Application of CFIR in qualitative data collection and coding (May 2016-May 2018)
For the second stage of our application of CFIR, we collected and coded qualitative data. Table 2 summarizes the data collection methods and their relationship to the CFIR domains and implementation of the six modules of Primary Care 2.0. We used semi-structured interviews and observations, as well as quantitative surveys, to collect data to assess implementation outcomes at the pilot site clinic. An additional file outlines the qualitative methods used in more detail.
We structured interviews and observations around the five CFIR domains and included questions and observation guides drawn from online CFIR references. 2 Table 3 includes sample questions from our semi-structured interview guides, by stakeholder group.
Two qualitatively trained researchers (C.B.J. and N.S.) conducted five site visits over 2 years, utilizing rapid ethnography principles to embed into the pilot site clinic and observe implementation. Rapid ethnography draws upon multiple related data collection methods (eg, observations and semi-structured interviews) in a short time frame while retaining a patient-centered focus. 10 We captured staff and prodefinitions for each construct to fit the Primary Care 2.0 model. Coders analyzed each data source for fit within each Primary Care 2.0 module and CFIR construct; a single data point could be, and often was, categorized to multiple constructs and modules. At periodic intervals, the researchers coded a transcript together to ensure intercoder agreement and reliability.
We coded transcripts using all 39 of CFIR's constructs and chose to explore each construct using deductive content analysis, an approach that utilizes a framework for analysis based on previous knowledge, 11 with each of CFIR's 39 constructs as primary nodes.
Additionally, we included six modules of the Primary Care 2.0 model in the coding structure. We analyzed coded data using the "queries" function of NVivo, which generates counts of code incidence across all data. We used an analytic matrix, juxtaposing Primary Care 2.0 modules and relevant CFIR constructs, to identify overlap between T A B L E 1 Initial Primary Care 2.0 modules and definitions Including, but not limited to, continuous quality improvement, daily "huddles," case conferences, and data monitoring the two frameworks. This process identified several CFIR constructs that did not yield any data, for example, trialability, external policies and incentives, and organizational incentives.

| RESULTS
We made three key adaptations to CFIR at different stages throughout our evaluation: 1) promoted the "patient needs and resources" construct from the "outer setting" domain to its own sixth domain in the framework; 2) divided the "inner setting" domain into three layers to better reflect the complexity and hierarchy of implementation within a health care system; and 3) tailored CFIR construct definitions to fit the intervention's patient-centered, team-based care context and support consistency of data collection and coding.

| A sixth domain: Patient needs and resources
The most significant change to CFIR structure arose in the process of coding of our qualitative data. Following our initial coding, we conducted a matrix analysis juxtaposing the five CFIR domains and each of the Primary Care 2.0 modules, as shown in Table 4. Reviewing the data at the intersection of these two categories sparked further analysis where high and low frequencies emerged, specifically within the outer setting domain which yielded the highest number of data points.
A deeper dive revealed that a large portion of the "outer setting" data pertained to themes around patient satisfaction, needs, and resources.
Our team promoted the CFIR construct "patient needs and resources" to sixth domain during analysis, due to the high frequency with which patients were referenced (a total of 46 times), as well as the T A B L E 2 Data collection methods for each Primary Care 2.0 module, by CFIR domain

| Inner setting
The CFIR framework draws particular attention to the intervention setting, which it divides into inner and outer settings, in order to capture contextual forces on implementation. The "inner setting" refers to implementation factors as they exist within an "organization," while the outer setting mostly considers influences external to the organization. Distinct from the original application of CFIR and as part of our data collection planning process, we further divided the inner setting into three distinct subparts, defined in Figure 1. These levels were adapted from organizational theory, which posits that effective efforts to impact quality consider four levels of change within a health system: the individual, the group or team, the organization, and the larger system and environment. 12 We redefined CFIR's use of "organization"

| Tailored CFIR construct definitions
The first barrier our team encountered in applying CFIR was agreement for each construct's meaning and application in our specific intervention and clinical context as we developed our data collection materials. We found some constructs repetitive and others too broad in the context of an academic medical center. To address these issues, the primary researchers C.B.J. and N.S. reviewed each construct and codeveloped tailored definitions that were clear to both and a better fit for the context of a patient-centered care transformation within a learning health system. Table 5 provides examples of these definitions.
For example, we addressed the overlap between the constructs "goals and feedback" and "reflecting and evaluating" by tailoring each to exist exclusively in different settings. We redefined "goals and feedback," originally categorized to the "inner setting" domain, as communication and activities around implementation goals and progress that is led by the pilot clinic leadership. We characterized "reflecting and evaluating," a construct of the "process" domain, as an exercise led by Stanford Health Care leadership with support from evaluation team efforts, as part of a partnered research approach.
Although they originally exist within two different domains, the overlap between these constructs in the context of this transformation necessitated a shift of "goals and feedback" to the "process" domain since the new definition articulated a process to support implementation.
Much like the process for codebook development in preparation for qualitative data analysis, which emphasizes clear exclusion and inclusion criteria to support inter-coder reliability, 13 we aligned our working CFIR definitions at the start of the evaluation to ensure interobserver concordance.

| DISCUSSION
This study describes the pragmatic adaptations we made to CFIR to The addition of a sixth domain may also ensure that future evaluations prioritize patient needs during evaluation design, data collection, and analysis.
Dividing the "inner setting" domain into three layers to better reflect the complexity and hierarchy of our health care system (ie, a primary care network within a broader academic health system that has strong tertiary and quaternary-level enterprise) facilitated our identification of drivers of decisional and operational change.
Whereas original use of CFIR would have us consider the inner setting for an organization overall, we harnessed lessons learned from organizational theory to pay attention to differences in implementation factors at various levels throughout. These factors were vital to understanding the evolution of Primary Care 2.0, especially when the Health Care leadership, but we also were attentive to the need to reach theme saturation, which is a standard approach in rapid qualitative analysis that we achieved in this study. 10 Second, we were not able to include patient interviews and observations we conducted in this analysis due to insufficient sample size. Both limitations are artifacts of project resourcing.
While others have also used CFIR in evaluating care transformations and found the framework valuable in their efforts, [4][5][6] we are the first to add depth to the "inner setting" domain and recognize the need to elevate patient-facing elements of redesign by making "patient needs and resources" its own domain rather than a construct within a domain. Supported by community-engaged research and other user-centered approaches, we believe the prioritization of and attention to the patient in this adaptation is essential and generalizable to implementation of any human-centered intervention in a health care setting. A patient-centered CFIR is also wellpositioned to complement the growing body of clinical effectiveness and implementation hybrid research studies. 18 As a whole, these adaptations to CFIR facilitated robust data collection and analysis across multiple qualitative researchers in a patient-centered, outpatient primary care health setting. We also hypothesize that they are applicable to other patient-centered, outpatient contexts. Our adaptations may also be transferrable to a wider array of audiences and fields, including user-centered design, academia, and community organizations.

| CONCLUSION
We believe this example of how to tailor CFIR will make CFIR more accessible and relevant for wider use in the context of patientcentered health care interventions, especially within the context of multi-part health care systems. Incorporating the adapted CFIR into our evaluation allowed us to assess a complex, dynamic primary care initiative while capturing context and culture-specific factors that influenced implementation across each level of the organization. We found CFIR to be a powerful longitudinal evaluation tool that facilitated capture of thoughtful nuances and key voices throughout the implementation process. Adaptation of the framework, however, was key for successful pragmatic application.
Placing additional emphasis on patient needs and resources; adapting the "inner setting" domain to reflect the complexity of health care organizations; and developing tailored, agreed-upon definitions for CFIR constructs at the start of the evaluation supported our team in capturing real-time changes within the implementation process and filled significant gaps in the framework. We believe our adaptations will be helpful to others and encourage wider application of CFIR and these strategies to move forward the science of evaluating patientcentered health care redesign. Dr Mahoney is currently a practicing provider in Stanford Health Care's system.

CONFLICTS OF INTEREST
Author Mahoney is currently a practicing provider in Stanford Health Care's system. We have no other conflicts of interest to report.