Unpacking the Cinderella black box of complex intervention development through the Partners at Care Transitions (PACT) programme of research

Abstract Introduction Complex intervention development has been described as the ‘Cinderella’ black box in health services research. Greater transparency in the intervention development process is urgently needed to help reduce research waste. Methods We applied a new consensus‐based framework for complex intervention development to our programme of research, in which we developed an intervention to improve the safety and experience of care transitions for older people. Through this process, we aimed to reflect on the framework's utility for intervention development and identify any important gaps within it to support its continued development. Findings The framework was a useful tool for transparent reporting of the process of complex intervention development. We identified potential ‘action’ gaps in the framework including ‘consolidation of evidence’ and ‘development of principles’ that could bracket and steer decision‐making in the process. Conclusions We consider that the level of transparency demonstrated in this report, aided through use of the framework, is essential in the quest for reducing research waste. Patient or Public Contribution We have involved our dedicated patient and public involvement group in all work packages of this programme of research. Specifically, they attended and contributed to co‐design workshops and contributed to finalizing the intervention for the pilot evaluation. Staff also participated by attending co‐design workshops, helping us to prioritize content ideas for the intervention and supporting the development of intervention components outside of the workshops.


| INTRODUCTION
Literature describing complex intervention development has lagged behind evaluation and implementation sciences and has been described as the 'Cinderella' black box in health services research. 1 Despite the publication of guidance and frameworks on the development of complex interventions over many years, [2][3][4][5][6] replicable and detailed reporting of the collective actions taken during the development of complex healthcare interventions is somewhat limited. Reasons for this lag are likely multifactorial and interlinked, including, for example, lack of evidence linking use of intervention development guidance frameworks to positive outcomes 7 ; a lack of clarity about which actions within the guidance contribute to clinically effective interventions [8][9][10] ; adoption of varying and uncertain approaches 11 and a lack of welcome 'space' (through journals) to host 'messy' research. 1 From our own observations, we would add to that list: an unchallenged acceptance that snapshots of, for example, co-design steps sufficiently represent the main process of development; a scientific narrative that has predominantly focused on evaluation, specifically randomised controlled trials (RCT) and an underappreciation of the imperative to broaden the narrative and share the learning to progress the science on complex intervention development. Research waste (i.e., research that is, among other things, poorly designed or unnecessary) is of ongoing concern, 12,13 and it is thought that greater transparency about the intervention development process will support the learning needed to shift trends from negative and inconclusive trials towards more positive outcomes. 7,14 Recently, O'Cathain et al. 14 published a consensus-based framework to guide decision-making in complex intervention development. This framework lends itself to more systematic and transparent reporting. 15 The framework suggests a set of 11 key actions and 5 principles, collectively forming a logic model. The actions include planning the process through to intervention refinement ( Table 1). The principles include being dynamic, iterative, creative, open to change and looking towards evaluation ( Table 2).
The framework is critical for the field of complex intervention development in that it both acknowledges the challenges and nuances of the process and grants permission to researchers to be open about these. There is now an opportunity to start unpacking this major black box within health services research, to contribute to the emerging science on complex healthcare intervention development and to support efforts to reduce research waste.
Here, we report the process of developing a complex healthcare intervention to improve the safety and experience of older people (aged 75 years and older) transitioning from hospital to home (Table 3) and the extent to which the steps that we took address the actions within the reported framework. In line with O'Cathain et al.'s 14 recommendations, we report the work that preceded, and was fully embedded in, the active designing of the intervention itself. Throughout, we highlight the challenges encountered and the nuanced decisions that were made in line with enacting the principles of complex intervention development. Finally, we explore important gaps within the framework and offer suggestions for its enhancement.

| THE PROCESS OF INTERVENTION DEVELOPMENT
The main steps undertaken in developing the intervention are shown in Figure 1. Each of the three phases is described below.

| Phase 1: Early preparatory work
Transitional care for older people was raised as a concern by a local quality and safety stakeholder group. The group, which represents a diverse local community, comprises 25 local community members. At this stage, they had been working regularly with the research team, based at the local NHS Trust, for 2 years. Work to maintain the group and its diversity through connecting with local community groups continues to this day through dedicated patient and public involvement (PPI) leads connected with the research team. Amongst other things, the group's role involves highlighting priorities for research and one of these areas was patient participation in care with an expressed concern for the safety of older people. On exploring the literature, we learned that the transitional period from hospital to home is a risky time for older people, exacerbated by reduced lengths of stay and ongoing complex care needs. 28 As the study moved towards active intervention development, the team were increasingly supported by experienced research nurses providing a future focus on service context.
The team expanded to include two intervention designers with expertise in co-design. Before and between codesign workshops, the operational and design team met to plan, create a shared language and understanding of our programme theory and project boundaries and to develop the intervention ideas into a prototype.

IV Review published research evidence
The original proposal outlines the policy context, the evidence in relation to transitions and related interventions and opportunities for improvement from a theoretical perspective.
Gaps in the evidence base were identified in relation to both understanding the concept of patient involvement in care and in ascertaining the active ingredients of existing transitions interventions. A systematic review of patient involvement clarified this concept, 16 whilst a review of reviews of transitions intervention provided little clarity on the active components.
More broadly, the evidence that we gathered included literature reviews, scoping reviews, evidence from our own WPs, stakeholder events and patient panel feedback throughout the programme.
V Drawing on existing theories Our proposal outlined our plans to apply a novel theoretical approach to transitional care. This shifted the focus from a traditional safety management approach with control and standardization of work (with fixed intervention components) to a resilience-based approach allowing for flexibility according to system requirements. 17 In consolidating the evidence from our first two work packages, we applied FRAM, 18 which allowed us to fully explore the transitional system components as functions. 19 We modified our original principles and identified functional activities of intervention from WP1 and 2, as part of WP4 involving stakeholder engagement.
with the group and exploration of the literature, we learned that greater patient involvement in care was desirable and potentially amenable to change [30][31][32] but that there was some reluctance to adopt specific activities as part of a safety role, [33][34][35][36] for example, when challenging staff or monitoring whether particular practices occur. [34][35][36] These findings led us to prepare a research proposal that, in line with emerging safety science, was underpinned by a Safety II approach. 17 With this, there is a shift towards understanding everyday performance and strengthening the system to enhance the number of things that go right. This is opposed to the traditional approach that exclusively focuses on, and seeks to prevent, adverse events (now termed Safety I). 17 For the purposes of this programme, the aim was to create opportunities to support patients and staff to respond to, monitor, learn from and anticipate potential threats. 17 On application, funders required us to outline the format of the intervention. Based on existing evidence, we anticipated that we would design a patient-owned record and a predischarge meeting delivered in the hospital and a postdischarge telephone call to patients. Documenting this requirement here is important as it allows readers to consider how we have responded to the framework principle of being 'open to change' (see Table 4). The effectiveness of the trial of the intervention also required an important and readily measurable patient safety outcome and reviewers advised targeting emergency readmissions as the primary outcome. At the outset of the study, we defined our own set of guiding principles for intervention development (see Table 4). One such principle (to develop a flexible intervention that was adaptable to different care systems) demonstrates early consideration to future implementation, in line with existing evidence 37 (see Action IX in Table 3). The need for flexibility additionally guided us to develop an intervention around functional aims rather than purely fixed components. [38][39][40] 2.2 | Phase 2: Gathering the evidence to develop a theory of change Four main evidence-gathering activities that contributed to informing intervention development included two literature reviews (a systematic review on patient involvement in care and a review of reviews on the transitions intervention literature) and primary data collection (WPs 1 and 2) (see Actions IV and VII in Table 1). The literature reviews clarified unchallenged assumptions about the meaning of patient involvement and its enactment in a hospital setting 16 and showed that multicomponent and often condition-specific transitions interventions articulated patient involvement as patient education or self-management that differed from how we conceived patient involvement, as a nuanced and dynamic interactional process. This factor indicated our approach to be novel, with the intervention literature offering limited guidance.
Numerous early team discussions, including regular PPI, enabled us to start exploring how patient involvement in hospital for this population could happen in healthcare. We observed the role of patients in plugging gaps in care delivery and error checking as a way of supporting system resilience 41,42 but noted that some of these safety activities necessitated patients having to express concerns to staff.
This was considered to be particularly challenging for many patients, particularly older, non-English-speaking or more unwell patients. [43][44][45] T A B L E 2 Summary of how the PACT programme of research addresse the principles of the O'Cathain et al. 14

framework
Framework principles How each priniciple was addressed in the PACT research programme Dynamic An ambitious programme of research with the first three work packages overlapping and informing each other. Subsequent work packages were sequential, with rapid turnaround of results to feed into the next stage.

Iterative
One formal iteration of the intervention planned within WP4 with opportunity to understand some of the requirements for implementation. Opportunities to refine the intervention and further develop the implementation package were available during feasibility testing.

Creative
Use of Safety II theory 17 as an underpinning concept for intervention development. Fully exploring the meaning of patient involvement. Using FRAM 18 to consolidate our evidence and inform our theory of change and approaching transitions interventions in a novel way.
Open to change An original planned review on older people's experiences of transitions was replaced (because of the identification of an existing recent review) with a review on how patients enacted involvement. 16 The creation of an intervention that was, to some extent, different to that envisaged at the outset including the intended location for delivery. Going forward, being open to feedback from pilot and feasibility testing requiring modification to the intervention.
Looking forward to evaluation Adopting from the outset an approach that focused on building an intervention based on functions rather than components to allow for some local adaptation.

WP3
Developing a measure (PACT-M) to assess the quality and experience of transitions for older people. This measure was to be considered as a secondary outcome in the full cluster RCT. Not described in the current paper as this work was not central to intervention development. 24,25 WP4 Co-design the PACT intervention and perform a formative evaluation of the prototype intervention. 26

WP5
Testing the feasibility of delivering the new PACT intervention within a cluster RCT with a nested qualitative evaluation to inform the implementation strategy. 27

WP6
Cluster RCTto assess the effectiveness and cost-effectiveness of the PACT intervention.
Abbreviation: PACT, Partners at Care Transitions.
Additional challenges to enhancing patient involvement became apparent through our early work within the study (see Action II in Table 1). These included, for example, staff pressures that prioritize rapid discharges over planning for a transition, risk 'management' practices in hospital that shifted risk to patients after discharge, 23 care processes that are often not seen by patients and carers and patients' capacity or preferences for involvement in care. 21 We reflected on concerns that an intervention targeting patient involvement to close system gaps may represent a 'sticking plaster' and that the onus of responsibility for enacting care should not be placed unduly on vulnerable patients. Our discussions also helped us to clarify the basis for what would be our theory of change. Specifically, we understood that patients needed to 'reach in' to a system and that staff within the system needed to 'reach out' to patients but that critically, many interacting contextual factors would make this unlikely without the support and without changing mindsets. 41 This new understanding fitted with our considered original plans of having patient-facing core intervention components to support involvement, scaffolded by flexible staff-facing components.
To establish key areas to target for our intervention, to guide the delivery location and to inform our theory of change, we mapped data gathered from work packages 1 and 2 (specifically in relation to care activities) using the Functional Resonance Analysis Method (FRAM). 18 We identified four key functional activities that patients undertake after discharge: managing medicines; managing health and well-being; managing activities of daily living and appropriately escalating care to acute hospitals. 19 Our theory of change suggested that the ability of patients to carry out these activities successfully

Early intervention principles
Revised intervention principles (before the active intervention development phase) To help patients/carers better navigate and manage the transition from hospital to home in partnership with formal carers. Doing this with patients and those caring for them. Being flexible so fitting within different systems of care.
Creating resources that enable patients/carers to reach in to the system such that they can practice for going home. These resources will support patients to (1) ask questions and (2) actively 'do care'. Supporting patients/carers to be knowledge brokers, making themselves visible to the system, reducing safety gaps. Making the care system more visible to support patients/carers to better navigate and interrogate it. Creating a gentle scaffold to systematize patients reaching in. This will be in the form of reciprocal staff/service involvement work through staff facing flexible intervention components.

MURRAY ET AL.
| 1483 would be partly determined by the extent of their involvement in preparing for these during the hospital stay 19 (see Action VI in Table 1). Consequently, our intervention needed to support patient involvement in targeted specific hospital-based activities to improve outcomes after discharge. Locating delivery of the intervention in the acute setting represented a change from our original plans that envisaged delivery of an intervention split between secondary and primary care. We did, however, envisage that the intervention could be carried through to the community by patients, akin to the 'red book' used in UK maternity, health visiting and community services.

| Phase 3: Active phase of intervention design
Having clarified our theory of change, we revisited our earlier intervention principles to provide a more prescriptive guide for the active phase of intervention design (Table 4). We also created statements of functional aims for each of our five functional activities (Table 5). Both the intervention principles and functional aims were important benchmarks for judging potential content and format ideas to be taken forward for the intervention.
Four key overlapping but distinct activities during the active phase of intervention development occurred throughout 2018 ( Figure 1) and included co-design workshops; consolidation of evidence; formation of content and format for the intervention and the prototype. These are described below. Challenges to the co-design process and our responses to these included the following:

| Co-design workshops
(1) The first two workshops were designed to orientate participants using visual aids and creative approaches such as LEGO ® Serious Play ® and Persona's to attune them to our intentions for the intervention. This included ensuring that they understood the boundaries of the research in relation to the intervention principles (Table 4) and functional aims (  To resource patients to correctly take their medications 1 , effectively transition between TTOs b and their ongoing medications 2 and to identify and overcome any medication problems 3 that they may encounter. Understanding and managing health and well-being To resource patients to understand their health and well-being and the care therein 1 , their role in facilitating recovery 2 and to be ready to resume responsibility 3 (as appropriate) for their health and well-being Managing activities of daily living To resource patients to retain their autonomy and minimize the effects of deconditioning.
Knowing what to expect and how to escalate care appropriately after discharge.
To resource patients to seek help in an appropriate and timely manner for when any aspect of their health and well-being deteriorates acutely.  Table 1). | 1485 an about me card, a leaflet, a red bag, passports, a communication board and video and lunch time and locker mats. Numerous discussions within the co-design workshops and between the PACT team and the design team took place about how to take forward format ideas related to cost, the extent to which it supported patient involvement and met with funders' aims in terms of scope, the extent to which it could meet all of the intended functional aims, practical implementation (including the capacity of staff to deliver the intervention and generalizability) and infection control. For the purposes of developing a prototype, we were primarily concerned with taking forward suggestions that would support patients to 'reach in' and support system resilience.

| Reviewing the prototype
We created the 'Getting Home, Staying Home' intervention, which comprised a patient-held booklet, a ward induction leaflet, a stand-up question card and a 'your hospital record' sheet and a patient-friendly discharge letter. These were housed in a purpose-designed envelope. within the hospital stay was also discussed. The prototype intervention was taken forward to a formative evaluation to explore its initial acceptability, to learn about its implementation in practice and to inform further development. At this stage, we had not fully explored how staff could respond flexibly to the intervention functions to scaffold the intervention and enhance patients reaching in. We envisaged that this would be explored during the pilot and trial feasibility studies.

| DISCUSSION
We have developed a novel transitions intervention for older people based on theoretical principles of resilient healthcare 17 and patient involvement. 16 This intervention aims to support patients to reach  49 Depending on the framework adopted, 'optimization' of the intervention may only take place within one formal iteration, with subsequent feasibility work often focusing on assessing trial processes and informing implementation plans.
While the most recent MRC guidance on the development of complex interventions advocates repeating phases 'where uncertainty remains', there is little guidance on how this might be achieved within predefined timelines and funding constraints. 6 Inflexibility in the funding and research process often prevents researchers from being able to truly iterate and thus truly achieve an optimal intervention that is more likely to be deliverable and effective.
Adopting more agile approaches (as in software engineering) or integrating an action research phase into the development may therefore ensure greater iteration and also allow for earlier termination of projects that are unlikely to be effective (Table 7). We identified what we consider to be two additional actions for the framework: creation of either functions or intervention principles and consolidation of evidence (Table 7). From the outset, we defined our own principles for guiding the intervention development through a set of statements that set the parameters for the intervention. We revisited and enhanced these so that they collectively became a decision-making tool to guide the content for the intervention.
We considered this to be a distinct and essential action within the research process and therefore a worthwhile addition to the framework. The evidence that we amassed during the first two work packages, the multiple stakeholder engagement approaches and the various reviews had the potential to be unwieldy.

Iterative and dynamic
Be open to how about what is realistically achievable within a time-bound study and/or at the time of planning, build in sufficient time and evaluative steps to ensure that uncertainty during intervention development is minimized. Constant change through being dynamic also needs to recognize time to pause and reflect.

Creative
Expand the meaning to include delving into creative practices for the evolution and testing of ideas and using novel methodologies that facilitate new ways to explore problems and solutions.
Missing action 'Creation of functions or principles' for the future intervention to bracket and guide intervention development. This further allows for transparent reporting of intervention ideas that were suggested but not taken forward.
Missing action 'Consolidation of evidence' to capture how the evidence from disparate sources (including the co-design stage) is brought together.
MURRAY ET AL.
| 1487 feedback and discuss implementation approaches. This would have allowed a greater opportunity for small-scale, less burdensome (for patients and staff) iteration cycles.
We found the co-design workshops both rewarding and challenging. For us, entering into the co-design phase with prescribed principles and functions was essential but we recognized that these might stymy the creativity of the group. Participants did, however, make valuable contributions to both the intervention format and supported decision-making about the content. Some format suggestions were challenging to reconcile within the scope defined by our principles, theory of change and the practical constraints of the project resources (time and budget). This is clearly an area that requires further debate; however, the sheer scale of this topic places it beyond the scope of this report. In a future article, we intend to reflect on this from a number of case studies. On balance, our principles and functions enabled us to demonstrate the provenance of the intervention, guide the participants and report a coherent and clear account of its development.

| CONCLUSIONS
The new MRC guidance advocates bolder, deliberative and flexible approaches to complex healthcare intervention development and evaluation. We agree with this as long as it is aligned with funders who are fully supportive of, in particular, more flexible approaches.
We consider that without transparent and full reporting of the