Refining a capability development framework for building successful consumer and staff partnerships in healthcare quality improvement: A coproduced eDelphi study

Abstract Background The capability of consumers and staff may be critical for authentic and effective partnerships in healthcare quality improvement (QI). Capability frameworks describe core knowledge, skills, values, attitudes, and behaviours and guide learning and development at individual and organizational levels. Objective To refine a capability framework for successful partnerships in healthcare QI which was coproduced from a scoping review. Design A two‐round eDelphi design was used. The International Expert Panel rated the importance of framework items in supporting successful QI partnerships, and suggested improvements. They also rated implementation options and commented on the influence of context. Participants Seven Research Advisory Group members were recruited to support the research team. The eDelphi panel included 53 people, with 44 (83%) and 42 (77. 8%) participating in rounds 1 and 2, respectively. They were from eight countries and had diverse backgrounds. Results The Research Advisory Group and panel endorsed the framework and summary diagram as valuable resources to support the growth of authentic and meaningful partnerships in QI across healthcare contexts, conditions, and countries. A consensus was established on content and structure. Substantial rewording included a stronger emphasis on growth, trust, respect, inclusivity, diversity, and challenging the status quo. The final capability development framework included three domains: Personal Attributes, Relationships and Communication, and Principles and Practices. The Equalizing Decision Making, Power, and Leadership capability was foundational and positioned across all domains. Ten capabilities with twenty‐seven capability descriptions were also included. The Principles and Practices domain, Equalizing Decision Making, Power, and Leadership capability, and almost half (44.4%) of the capability descriptions were rated as more important for staff than consumers (p < .01). However, only the QI processes and practices capability description did not meet the inclusion threshold for consumers. Thus, the framework was applicable to staff and consumers. Conclusion The refined capability development framework provides direction for planning and provision of learning and development regarding QI partnerships. Patient or Public Contribution Two consumers were full members of the research team and are coauthors. A Research Advisory Group, inclusive of consumers, guided study execution and translation planning. More than half of the panel were consumers.


| INTRODUCTION
Globally, patient and family engagement is gaining momentum as a strategy to improve quality and safety across the continuum of health services. 1,2 The language used internationally to encompass the concepts of patient and family engagement varies and includes patient and public participation, patient and public involvement and engagement, stakeholder engagement, and consumer and community involvement (CCI). CCI will be used throughout this paper as the research originated in Australia where that terminology is widely used. The term consumer is inclusive of past, present, and future health service users (patients), family members, and the public or community. 3 Including consumer partnerships in service planning, delivery, and evaluation aims to improve person-centred care and is linked with the paradigms of consumerism, democracy, human rights, recovery, and empowerment. 4 A systematic review reported benefits from CCI in healthcare quality improvement (QI) including improved service delivery, enhanced governance, and better-informed policies and planning. 5 Despite these benefits, authentic partnerships in QI are not the norm and changes are needed at the individual, organizational, and system levels. [6][7][8] In particular, both individual consumers and healthcare staff need the requisite attitudes, skills, and knowledge to successfully partner in QI. 9 The provision of education for partnership capabilities has been recognized as an enabler of effective CCI with health professionals noting that this is not taught in their health degrees. 6 Furthermore, the World Health Organization has recommended that ongoing learning for effective involvement of patients in quality and safety should be a requirement for professional registration. 1 Additionally, training for both consumers and staff is included in policy and accreditation requirements in many health jurisdictions, 3,10,11 and research has also highlighted the need for training and development. 6,[12][13][14][15] However, despite these calls for staff and consumer education, little research provides clear direction for the planning and provision of learning initiatives to enhance partnership capabilities.
Capability frameworks may be an effective strategy to address this need as they guide learning and development planning and implementation at the individual and organizational level by describing core knowledge, skills, values, attitudes, and behaviours. [16][17][18][19] They have been used across many areas of healthcare including mental health, 17 e-health, 19 osteoarthritis, 16 interprofessional practice, 20 and frail older persons' care. 18 Given the complexity and rapid evolution of CCI, a capability perspective is required as it emphasizes integration of knowledge, skills, values, and attitudes to enable adaptation to change, continuous growth and improvement. 16,21 Of interest, only one of the aforementioned capability frameworks was developed in consultation with consumers. 16 Additionally, with the exception of the framework presented in this paper, the research team was not aware of any published health consumer-focussed capability frameworks.
Given the utility of capability frameworks to guide learning and development and the apparent lack of literature addressing capabilities for successful staff and consumer partnerships in QI, the current research team, inclusive of two consumers, coproduced a scoping review on the topic. 22 This led to the development of a Capability framework for successful partnerships in healthcare QI. The papers in that review originated from nine countries and spanned the healthcare continuum, different health conditions, and a diversity of CCI stakeholders. However, because few papers explicitly discussed capabilities, the framework was based on inferred content. Additionally, the research team members were all from Australia. Hence, the framework needed to be subjected to further scrutiny from a more diverse group to assess its international acceptability and enhance its relevance. This led to the current study which aimed to refine the Capability framework for successful partnerships in healthcare QI. Study objectives were: (i) To develop international expert consensus regarding the framework items and structure; and (ii) to explore whether different capabilities are required across diverse healthcare contexts.

| METHODS
An eDelphi design enabled geographically diverse participants to iteratively contribute to consensus development in an anonymous, convenient, and reflexive manner. 23,24 Furthermore, an eDelphi study is participatory and inclusive, and explores not just 'what is' but 'what could/should be'. 24 Although the research team included two consumers, the expertize of a Research Advisory Group, was integral to the study design to maximize the diversity of consumer and other stakeholder perspectives for study decision making and results interpretation. The Research Advisory Group met four times over 4 months. Figure 1 summarizes the study processes including Research Advisory Group meetings and topics, and eDelphi rounds. Recruitment and data collection occurred from June to September 2021.
There was no remuneration for any of the consumers involved in this study due to a lack of funding.

| Data collection: eDelphi rounds
Two eDelphi rounds were implemented online (see Figure 1) and were open for three weeks each. Two rounds, rather than three were justified because an initial exploratory round was not required as this eDelphi was based on the previously co-produced capability framework. 22 There were approximately three weeks between survey rounds. A systematic review of 100 Delphi studies reported a median consensus level of 75%. 27 Therefore, this consensus level was adopted.

| Round 1
To prepare the panel for involvement, eligible participants were

| Round 2
A summary of round 1 results, including changes to the capability framework item wording, was emailed to all panel members with the round 2 survey hyperlink. Round 2 included an education level F I G U R E 1 eDelphi study design flow chart question as requested by the Research Advisory Group. Additionally, the panel was asked to rate their level of agreement that the updated title, new purpose statement and new underpinning principles adequately described framework intent and content. Separate importance ratings of the updated domains and capability descriptions were sought for consumers and healthcare staff, as more than 50% of the panel requested this in round 1. Free text comments and suggestions for additional items were again requested. The panel was also asked to rate the overall usefulness of the updated framework and updated summary diagram. Additionally, they were asked to indicate how they thought the framework could be used to support healthcare QI partnerships. Comments were also sought regarding contextual issues which may limit or promote framework use.

| Data analysis
All data were initially analyzed by the first author and then reviewed by the full research team with reference to the raw data. Once an agreement was reached, preliminary results were discussed with the Research Advisory Group. Quantitative data were analyzed descriptively and presented in tables and graphs. Additionally, for round 2, the Kruskal-Wallis H test was applied to determine the statistical significance of any differences in ratings of 'usefulness' and 'importance' for all questions regarding the framework items for (i) participants in Australia compared to those outside Australia (country); and (ii) participants identifying as a consumer and/or carer and nonconsumers (role). Furthermore, the χ 2 test was used to determine the statistical significance of any differences regarding the choice of potential implementation options by country or role as above. Wilcoxon-signed ranks were utilized to test whether there was a statistically significant difference in all participants' importance ratings of the capability framework items for consumers versus staff.
The p-value was set at .01 for statistical power given there were a large number of comparisons across a relatively small sample size.
Statistical analyses were completed using PSPP (Version 1.4.1). 30 Qualitative descriptive analysis included identification of common positive feedback ideas, suggestions for improvement, challenges to items and queries regarding wording, as recommended elsewhere. 29 3 | RESULTS

| eDelphi Round 1
In round 1, all responses (n = 44) were complete, and 95.5% of the panel rated the framework as 'very useful' (56.8%) or 'useful' (38.6%) in supporting successful partnerships in QI. Table 2  The word building was suggested as an addition to the title, consistent with the house imagery of the framework summary diagram.
There were also several requests for a new section at the beginning of the framework to clarify intent and scope with the strong suggestion that growth, development, respect, accountability, and trust be explicitly stated. An emphasis on diversity and the impact of colonization on First Nations peoples was also requested. The Research Advisory Group supported the inclusion of these concepts in the principles. The house diagram was endorsed due to the metaphor of strong foundations in power-sharing. It was suggested that 'knowledge, skills and attitudes' be removed to simplify the image and for colours be added to focus the eye on the vertical capability domains.

| eDelphi Round 2
Of the 42 Round 2 responses, five were incomplete. All available data were included in the analyses. The number of respondents for each question is stated below. All participants (100%, n = 39) rated the framework as 'very useful' (84.6%) or 'somewhat useful' (15.4%) in supporting successful partnerships in QI.

| Title, purpose statement, principles and diagram
There was consensus that the updated title adequately described the intent and contents of the framework with 97.6% of participants indicating 'strongly agree' or 'agree' (n = 42). Some concern was expressed that including all iterations of patient/consumer made the title too long. intent and contents of the framework (n = 42). Some concerns were expressed that the statements were too detailed and lengthy. However, this was contrasted with proposals suggesting additional content. Table 3 includes the final reworded framework title, purpose statement and principles. The updated colour summary diagram was supported by 100% (n = 39) of participants as 'very useful' or 'somewhat useful' with comments that it had been improved (see Figure 2).

| Separate importance ratings for staff and consumers
The capability domains and all capability descriptions were strongly supported as 'very important' or 'important' for healthcare staff due to their organizational knowledge and power (see Table 2 with all being rated as less important for consumers (see Table 2).
Qualitative data also highlighted panel perceptions that staff and consumer responsibilities were different for capability development. 1. To describe the key capabilities needed for building successful partnerships in healthcare quality improvement; and 2. To promote reflection, growth, learning and development regarding these capabilities at individual, team, and organizational levels. Principles: 1. Everyone is on a learning journey and this framework intends to support life-long learning and development for all partners. It is not intended to imply that all partners will begin with all capabilities. 2. Successful partnerships happen in organizational and social contexts, and it is essential that everyone feels welcome, empowered, responsible, trusted, and accountable. 3. Capabilities include knowledge, skills, attitudes, and values which influence behaviour and go beyond competence to include a focus on personal growth and adaptation to change. 4. Organizational leaders have a key role in fostering, resourcing, and promoting a supportive, respectful culture for successful partnerships.

Partnerships must occur with diverse individuals and communities across the lifespan including Australian Aboriginal and Torres Strait
Islander peoples, and other indigenous peoples internationally; people with a disability; people who identify as LGBTIQ+; people from culturally and linguistically diverse backgrounds; people from rural and remote areas; and all people who experience health inequities. 6. Knowledge and understanding of the history of colonization and the current impact on indigenous peoples lays a foundation for moving forward. 7. Service users, patients, consumers, citizens, family members, carers, friends, community, clinical, and nonclinical health service staff and consumer organization staff, volunteers and consumer advisors are all a focus for this framework. It is also inclusive of current, past or potential users of health services. 8. There is no 'one size fits all' method of successful engagement.
Appropriate strategies will depend on many factors including improvement goals and available resources.
| 1573 rated the framework items similarly to participants from other countries and that consumers and carers rated the framework items similarly to other panel members.

| Implementation strategies
The most frequently endorsed strategies for implementing the framework were: QI team or committee (inclusive of consumers and healthcare staff) reflection and development planning (92.3%, n = 39); and, individual reflection and development planning for healthcare staff (alone or with a supervisor or mentor) (84.6%, n = 39) (see Table 4). Other implementation suggestions included: using the framework to guide a community of practice; aligning QI activity streams with capability domains for learning events and planning fora; and government system-level review of QI policies and practice.
The current lack of formal structures in most organizations to promote individual consumer development planning was noted.
Several panel members suggested that the framework should be validated for research partnerships.
There was no statistically significant difference regarding the choice of potential framework implementation options by country or role. Many

| DISCUSSION
This eDelphi study advances CCI practice by refining a capability framework for successful partnerships in QI which was based on a coproduced scoping review. 22 International consensus has been established on the content and structure of the framework which is now entitled: Building successful partnerships in healthcare QI: A capability development framework for service users, families, communities, and staff.  45 The nuanced applicability of the framework for these roles is a direction for further research.

| Context and implementation
The eDelphi panel and Research Advisory Group advised that the capability development framework content may be universal. The context was perceived as influencing implementation strategies rather than framework structure or items. The fact that there was no statistically significant difference in ratings for any aspect of the framework between participants from Australia compared to other countries supports the notion that it may not be country-specific.
However, this requires further investigation as, for example, culturally specific strategies for CCI in research in Asia have been discussed. 25 Smaller organizations were identified as potentially less able to implement the framework. However, this conflicts with research that identified that smaller, nonteaching hospitals displayed a higher organizational capacity for CCI than some large teaching hospitals pointing to the critical influence of organizational culture and leadership. 2 Nongovernment organizations, primary care, and residential care were also identified as settings where framework implementation may be challenging. This requires research as there are many examples of highly effective CCI for QI in primary care, in particular. 46,47 Consumer renumeration was highlighted by some panel members and the Research Advisory Group as critical to successful implementation and is supported by some literature. 12,32 However, nonmonetary recognition may be just as important. 24,38 Utilization of the capability development framework for the whole QI team or committee reflection and development planning was the most frequently endorsed implementation strategy. Initiation of a community of practice guided by the framework had traction with the Research Advisory Group. Both of these activities would reinforce the colearning capability which may create a common language, enhance relationships, and reduce power differentials. 5,36 The capability for staff and consumer self-reflection is also incorporated in these initiatives which would further reinforce the framework.

| Limitations
The average age of participants was 51.11 years, more than 70% identified as female, all had a high level of English language proficiency, and over 55% had a Masters's degree or higher, which may limit study generalizability. However, as called for previously, 9,34