A pilot study to assess the learning environment and use of reliability enhancing work practices in VHA cardiac catheterization laboratories

Abstract Introduction A learning health system (LHS) harnesses data and analytics to learn from clinical encounters to implement the best care with high reliability. The 81 Veterans Health Administration (VHA) cardiac catheterization laboratories (cath lab) are a model LHS. The quality and safety of coronary procedures are monitored and reported by the Clinical Assessment, Reporting and Tracking (CART) Program, which has identified variation in care across cath labs. This variation may be due to underappreciated aspects of LHSs, the learning environment and reliability enhancing work practices (REWPs). Learning environments are the educational approaches, context, and settings in which learning occurs. REWPs are the organizational practices found in high reliability organizations. High learning environments and use of REWPs are associated with improved outcomes. This study assessed the learning environments and use of REWPs in VHA cath labs to examine factors supportive of learning and high reliability. Methods In 2018, the learning organization survey‐27 and the REWP survey were administered to 732 cath lab staff. Factor analysis and linear models were computed. Unit‐level analyses and site ranking (high, low) were conducted on cath labs with >40% response rate using Bayesian methods. Results Surveys from 40% of cath lab staff (n = 294) at 84% of cath labs (n = 68) were included. Learning environment and REWP strengths across cath labs include the presence of training programs, openness to new ideas, and respectful interaction. Learning environment and REWP gaps include lack of structured knowledge transfer (eg, checklists) and low use of forums for improvement. Survey dimensions matched established factor structures and demonstrated high reliability (Cronbach's alpha >.76). Unit‐level analyses were conducted for 29 cath labs. One ranked as high and four as low learning environments. Conclusions This work demonstrates an approach to assess local learning environments and use of REWPs, providing insights for systems working to become a LHS.


| INTRODUCTION
A learning health system (LHS) harnesses data and analytics to learn from clinical encounters to implement the best care with high reliability. The foundations of LHSs are both technical and cultural. Technical aspects include an infrastructure for capturing clinical data and analytic tools to process the data to produce the best possible evidence to inform decision making. Cultural elements include incentive systems that reward continuous learning, leadership support for transparency and the use of research to influence practice and, in turn, using this practice to influence subsequent research. 1 LHSs are designed to systematically and continuously collect, analyze, and deliver data to the point of care so each patient encounter is informed by the one before it. 2 This continuous learning cycle requires an environment supportive of learning and work practices that support high reliability.
Learning environments are the educational approaches, cultural context, and settings in which teaching and learning occurs. Reliability enhancing work practices (REWPs) are a bundle of work practices developed in high reliability organizations (HROs) to ensure consistently error free performance in complex settings. REWPs include hiring and training for interpersonal skills, forums for sharing expertise and making recommendations for improvement, along with providing opportunities for local adaptations and front-line control over work processes. 3 The learning environment and REWPs are underappreciated aspect of LHSs that have been associated with enhanced workforce and patient outcomes. [3][4][5][6][7] Many healthcare organizations are working to become LHSs and HROs in order to improve clinical quality and outcomes, including the Veterans Health Administration (VHA). 8 The VHA is the largest integrated health care system in the United States, providing care at 1250 health care facilities and serving nine million enrolled Veterans each year. The VHA's LHS and HRO journey began through large-scale implementation of an electronic medical record and data-warehousing infrastructure to capture clinical data from every patient encounter.
The data are then analyzed to inform decision making at the point of care. LHSs can be found at the department, program, unit, or facility level. A model LHS within the VHA is the Clinical Assessment, Reporting and Tracking (CART) Program.
CART is a national quality and safety program for invasive cardiovascular procedures, including those performed in cardiac catheterization laboratories (cath labs). This program harnesses real-time clinical data to support Veteran care and quality monitoring. Integrated within the VHA electronic medical record, the CART Program uses a specialized software platform to collect real-time patient and procedural data for all VHA patients undergoing coronary procedures to drive the LHS continuous learning cycle. 9 The CART Program has detected variation in clinical care across VHA cath labs. This includes variation in implementation of appropriate use criteria for elective percutaneous coronary interventions (PCI) 10 and unnecessary hospitalizations after PCI and costs of care. 11 This variation has been attributed, in part, to differences in local cath lab learning environments. 10 To understand the relationship between variation in care and the role of learning environments and REWPs in the VHA, assessment of the state of learning environments, and use of REWPs across the 81 VHA cath labs is necessary.
The aims of this pilot study are to assess the learning environments and use of REWPs of VHA cath labs and identify factors supportive of learning and high reliability. The findings will contribute to the scientific knowledge about how to build supportive learning environments and high reliability.

| QUESTIONS OF INTEREST
• What is the state of learning environments and use of REWPs in VHA cath labs?
• Are VHA cath labs supportive of learning and high reliability?

| METHODS
To identify the state of VHA cath lab learning environments and use of REWPs, we identified two existing surveys that were guided by learning organization 12 and high reliability 3 theories.

| Learning organization survey-27
The learning organization survey-27 12 (LOS-27) is a 27-item reliable and valid measure of organizational learning designed to pinpoint areas needing improvement. The tool was originally developed by Garvin et al 13 to examine the three building blocks of organizational learning: supportive learning environments (eg, psychological safety, appreciation of differences, time for reflection), concrete learning processes and practices (eg, activities for knowledge sharing), and leadership that reinforces learning (eg, bi-directional communication with employees, prioritization of issues). 13 Each building block contributes to teams' ability to learn. Together the building blocks produce supportive learning environments, which are foundational to LHSs and HROs. 12 The LOS-27 assesses perceptions of organizational learning using a five-point frequency scale (never to always) for the leadership items and a seven-point accuracy scale (highly inaccurate to highly accurate) for all other items. The psychometric properties of the LOS-27 were established in 2012 by Singer et al. 12 The LOS-27 was selected for this pilot work to assess learning environments, identify practices supportive of learning and to allow for comparison of learning environments across cath labs. A limitation of the LOS-27 is the lack of items that query the use of high reliability practices which can provide a deeper understanding of the learning practices building block.

| The REWPs survey
The REWPs survey 3 developed by Vogus and Iacobucci, is a valid and reliable instrument that measures high reliability practices in teams. The 31-item survey assesses five factors that positively influence patient safety through fewer medication errors and patient falls. 3  values. We also considered the consistency of the empirically derived factor structure with the theoretically determined scales. 3,12 To explore the questions of interest, we calculated descriptive and correlational statistics and reliability estimates. The leadership items of the LOS-27 were rescaled to a 1 to 7 ascending Likert to be consistent with the other items in the survey. To assess the association between an overall learning environment and REWP score and respondent/facility traits, a generalized estimating equation (GEE) model, clustered by site, was fit on the entire cohort of 294 respondents. The model used the average survey factor score as the response variable and an identity link function. The average survey score was based on the average of the individual factor scores. Predictors in the model included respondent age, gender, race, supervisory status (ie, yes/no), role (ie, interventional cardiologist, nurse, technician, other), years in VHA cath lab, and years in a cath lab (both dichotomized as ≤3 years vs >3 years), and the 2018 Star rating of the respondent's hospital. Star ratings are a comprehensive metric, calculated annually in the VHA, that include nine quality domains and one efficiency and capacity domain. 16 To identify site-level variation and rank sites on their average learning environment and REWP factor scores, Bayesian methods were used. [17][18][19] Bayesian profiling of the average factor score for sites was modeled using a Markov Chain Monte Carlo method with a burnin of 500 000 iterations, 400 000 estimation iterations, keeping every 200th estimation iteration for a total of 2000 samples used in calculation of site-level estimates and 95% credible intervals. The model had mean factor score for each respondent as the outcome variable, included a random intercept for each site, used an identity link function, and adjusted for the same covariates used in the GEE model described previously. The dataset for this model was restricted to cath labs with at least four responses and a 40% or greater response rate, as recommended by the VHA National Center of Organization Development for unit-level representativeness. High and low ranking was determined by identifying cath labs with credible intervals that did not overlap the system-wide average for mean factor score.
To assess the degree of variation between respondents within a cath lab, a coefficient of variation was calculated for the LOS-27 and REWP survey factors. The coefficient of variation was calculated as a ratio of the SD for a given factor divided by its site-level mean and then multiplied by 100 for a given site. The site-level mean was shifted to a 0 to 6 scale prior to calculation of the mean to provide a meaningful zero point. Pearson's r was computed to assess the relationship between survey factors.

| RESULTS
We received responses from 68 (84%) of the 81 cath labs. In total 294 of 732 eligible employees completed surveys (40% response rate). Of those, 65% (n = 190) were nurses; 11% (n = 31) were interventional cardiologists; 18% (n = 52) were technicians; 7% (n = 21) were "other"; and 27% (n = 80) held a supervisory role. The median age was 49 years with a median of 22 years in healthcare, 7 years in the VHA, and 5 years in their current cath lab ( High: cath labs with factor scores above the system-wide average (dotted line) and with credible intervals that did not include the system-wide average. Low: Cath labs with factor scores below the system-wide average and with credible intervals that did not  REWPs. This may be attributable to the complexity of publicly reported, facility-level quality metrics and noted challenges in connecting these outcomes directly to practices within a single unit. 26 In summary, this pilot work established the state of learning environments and use of REWPs in VHA cath labs. The results provide system and unit-level insights to guide the selection of evidencebased interventions for leaders and healthcare systems working toward becoming LHSs and HROs. The survey identified positive deviant cath labs (eg, high learning environments) that support learning through the creation of training programs, information sharing processes, opportunities for reflection, and practicing mindful organizing, affective commitment, and respectful interaction. These unit-level findings provide an opportunity for exploration into how and why these environments were created and sustained. The authors will be conducting a mixed-methods study in this population to understand how learning environments develop and if the findings apply across all VHA cath labs. This would fill gaps in the literature 3,12 and inform the spread and scale-up of best practices to benefit employees and Veteran care.

ACKNOWLEDGMENTS
We would like to thank the VHA staff that responded to this survey and Dr Eric Campbell for his wisdom during the writing and revising of this manuscript. Dr Gilmartin is supported by VHA Career Development Award IKHX002567 from the US Department of Veterans Affairs.