'It just wasn’t going to be heard’: A mixed methods study to compare different ways of involving people with diabetes and health‐care professionals in health intervention research

Abstract Background Guidelines recommend involving intervention users in the intervention development process. However, there is limited guidance on how to involve users in a meaningful and effective way. Objective The aim of this Study within a trial was to compare participants’ experiences of taking part in one of three types of consensus meetings—people with diabetes‐only, combined people with diabetes and health‐care professionals (HCPs) or HCP‐only meeting. Design The study used a mixed methods convergent design. Quantitative (questionnaire) and qualitative (observation notes and semi‐structured telephone interviews) data were collected to explore participants’ experiences. A triangulation protocol was used to compare quantitative and qualitative findings. Participants People with diabetes (recruited via multiple strategies) were randomly assigned to attend the people with diabetes or combined meeting. HCPs (recruited through professional networks) attended the HCP or combined meeting based on their availability. Results Sixteen people with diabetes and 15 HCPs attended meetings, of whom 18 participated in a telephone interview. Participants’ questionnaire responses suggested similar positive experiences across the three meetings. Observation and semi‐structured interviews highlighted differences experienced by participants in the combined meeting relating to: perceived lack of common ground; feeling empowered versus undervalued; needing to feel safe and going off task to fill the void. Conclusions The qualitative theme ‘needing to feel safe’ may explain the dissonance (disagreement) between quantitative and qualitative data. In this study, involving patients and HCPs simultaneously in a consensus process was not found to be as suitable as involving each stakeholder group separately.


| INTRODUC TI ON
For interventions to be successfully implemented in practice, they need to be acceptable, engaging and feasible to implement. 1 Intervention development guidelines recommend involving all appropriate intervention users to maximize the chances of successful implementation. 2 User involvement is a broad term that includes (but is not limited to) those receiving, eg patients and members of the public and delivering the intervention, eg healthcare professionals (HCPs).
Consensus methods are a way of involving multiple users simultaneously in the intervention development process. [3][4][5] Different users may have different priorities and preferences when making decisions about the content and delivery of an intervention. 6,7 For example, patients and members of the public may be concerned about how an intervention will be received by the target population, whereas HCPs may be more concerned about the cost involved (both time and money). 7 Group dynamics are complex, and some user groups may find it more difficult to voice their priorities and perspectives compared with others. 8 Despite increasing emphasis on user involvement, limited guidance exists on how to involve users in a meaningful and effective way. To our knowledge, no research has been conducted on patients and HCPs experiences of being involved in consensus methods and whether their experiences differ according to group composition.
The aim of this Study Within A Trial was to compare participants' experiences of taking part in one of three types of consensus meetings-people with diabetes-only, combined people with diabetes and HCPs or HCP-only meeting.

This Study Within A Trial (SWAT) was conducted within the on-going
Improving Diabetes Eye-screening Attendance (IDEAs) study. IDEAs is a feasibility study of a multifaceted intervention in general practice targeting HCPs and people with diabetes to improve the uptake of retinopathy screening. As part of the development phase of IDEAs, three separate consensus meetings were held to discuss the acceptability and feasibility of the proposed intervention content and suitable modes of delivery. Recommendations from each meeting were used to refine intervention components that could be delivered in general practice. The first consensus meeting consisted of people with diabetes only; the second meeting consisted of a combination of people with diabetes and HCPs and the third meeting consisted of HCPs only.

| Study design
The SWAT used a mixed methods convergent design to understand and compare participants' experiences of taking part in the consensus meetings (Figure 1). A one-phase design was used, where quantitative (experience survey) and qualitative (observation notes and semi-structured interviews) methods were used during the same timeframe and were given equal weight in the analysis. 9 Quantitative and qualitative data were collected and analysed separately. Results were merged during interpretation (mixed methods phase). A triangulation protocol was used in this phase to compare key concepts identified in each dataset that related to participants' experiences of taking part in the meetings. 9,10 The Good Reporting of A Mixed Methods

Study (GRAMMS) framework and the Consolidated Criteria for
Reporting Qualitative Studies (COREQ) were used to guide reporting of the findings. 11,12

| Recruitment of participants
People with diabetes were recruited using an information flyer de- HCPs were recruited through professional networks known to the SWAT and IDEAs study teams. HCPs were initially sent an email or letter inviting them to take part in the consensus meeting. This was followed by a phone call to confirm their attendance. HCPs were either allocated to the HCP-only or combined meeting based on their availability to attend.

| Semi-structured consensus meetings
Before the meetings, the IDEAs study team (FR, SMH) developed (a) a short summary of existing evidence on barriers to and enablers of attendance at diabetic retinopathy screening, and interventions to address non-attendance and (b) a survey asking participants to rate intervention components according to acceptability (like it, think it makes sense) and feasibility (think it can be done).

| Experience questionnaire
At the end of each meeting, all participants were asked to complete a questionnaire about their experience of the meeting. The objective of the questionnaire was to understand individual experiences of taking part in the meeting, asking them to rate how they felt about their participation and the participation of other group members; how decisions were made by the group; and the potential impact of the decisions that were made. We were unable to find a suitable validated instrument that was appropriate for our questionnaire objective and context (one-off participatory research process). Therefore, we developed our own ques-

| Quantitative data analysis
Questionnaire responses were entered into SPSS software (version 24) and analysed using descriptive statistics. The five response categories were collapsed into three categories-'Agree', 'Neither agree nor disagree' and 'Disagree'.

| Semi-structured interviews
Within 2 weeks of the consensus meetings, semi-structured telephone interviews were conducted with the consensus meeting participants who agreed to take part in an interview in the experience questionnaire. The objective of the interviews was to gain insights into individual experiences of taking part in the meeting in terms of: how comfortable they felt in the meeting; how they felt members of the group interacted with each other and how they felt they worked together to make decisions (ie, whether there was agreement, conflict, synergy). Interviews were audiorecorded (see Supplementary File 6 for Interview Topic Guide).
Interviews were conducted by ER, a young female PhD candidate.
All participants were familiar with ER as she facilitated the briefing session prior to the consensus meetings. At the beginning of each interview, the SWAT lead researcher (ER) stressed to participants that she was independent to the trial study team that 1 An in-group is a social group to which a person psychologically identifies as being a member. An out-group is a social group with which a person does not identify.
were running the consensus meetings and therefore would not be offended if they described negative experiences.

| Qualitative data analysis
Field notes were collated, and audio recordings were transcribed verbatim. All qualitative data were managed using NVivo software (version 12). Thematic analysis was carried out following Braun and Clarke guidelines. 21 Firstly, an extensive familiarization process was conducted by two researchers (ER, EP), where notes and transcripts were read and re-read multiple times. ER open coded all the observation notes and transcripts (using semantic and latent codes) and developed three separate sets of codes-one set for each meeting. The pattern and meanings of codes were then examined across the three meetings to identify one set of candidate or potential themes relating to participants' experiences and group dynamics. Themes were developed using a conventional or 'bottom-up' approach, whereby themes were developed directly from the data. 21 ER discussed each theme with EP to revise, refine and define themes.

| Mixed methods phase
After separate analysis of quantitative and qualitative data (as described above), the data were compared using a triangulation protocol. Triangulation provides a visual and tabular representation of the findings from qualitative and quantitative data, allowing for a clearer comparison and broader interpretation. 22 The steps taken to create the triangulation protocol are outlined in Table 1 below.

| Patient and Public Involvement (PPI) component
A PPI partner (GF) was involved in the SWAT from the outset. The PPI partner is a person with diabetes, previously known to the lead author (ER). She contributed to the initial discussions about the study which ultimately informed the SWAT grant application, reviewed the application and made changes to its content. GF was also involved in the development of materials used to recruit PPI contributors and assisted the research team with recruitment by posting recruitment flyers online via social media networks. In addition, she contributed to and reviewed each draft of this manuscript and is a co-author on this publication.

| Ethics
The study received ethical approval from the Social Research Ethics Committee (SREC) at University College Cork. Written informed consent was obtained from all participants prior to taking part in the consensus meetings and completing the questionnaire. Telephone consent was obtained from participants prior to taking part in the interviews.

| Participants
A total of 36 people contacted the research team expressing an interest in the SWAT. Of these, 20 completed the recruitment survey Step Activity

1.
Collate key findings from each dataset This was done by examining the original data, interpretation and reports of analysis. For quantitative data, each questionnaire item was deemed as a separate key finding. For qualitative data, multiple key findings were identified within each theme, as themes were too broad in their descriptions to compare directly to quantitative findings

2.
Group key findings into concepts Key quantitative and qualitative findings were grouped together into concepts according to how they related to participants' experiences and group dynamics (eg freedom of expression, balance of participation)

Explore intermethod discrepancies
This was done by examining the methodological rigour of each method and re-examining the data in light of the discrepancy 47

TA B L E 1
Steps taken to create triangulation protocol (see Supplementary File 3 for recruitment survey results). These 20 people were randomly assigned to either the people with diabetesonly meeting (4 with type 1 diabetes and 6 with type 2 diabetes) or the combined meeting (6 with type 1 diabetes, 3 with type 2 diabetes and 1 carer). All 10 people attended the people with diabetesonly meeting (attendance rate 100%) and 6 people with diabetes attended the combined meeting (attendance rate 60%). An invitation to attend was sent out to 50 HCPs (practice nurses, diabetes nurse specialists, general practitioners and specialist physicians), of whom 8 attended the combined meeting and 7 attended the HCP-only meeting (attendance rate 30%). Further details on the recruitment and response rates for each stage of the data collection are shown in Figure 2 below.

| Quantitative results
All consensus meeting participants (n = 31) completed the experience questionnaire (response rate 100%). Table 2 shows the results of the questionnaire stratified by meeting type (people with diabetes only, combined and HCP only). The descriptive statistics presented in Table 2

| Qualitative results
In total, 18 questionnaire respondents agreed to be contacted for a follow-up interview. Interviews were conducted with participants from the people with diabetes-only (n = 6), combined (n = 7) and HCP-only (n = 5) meetings. Interviews were, on average, 34 minutes in duration (range 18-56 minutes).
Four themes were developed from the qualitative data relating to participants' experiences and group dynamics: perceived lack of common ground; feeling empowered versus undervalued; needing to feel safe and going off task to fill the void.

| Perceived lack of common ground
In the people with diabetes-only meeting, there were differences between participants in terms of diabetes type, length of diagnosis and education level. In the HCP-only meeting, differences included profession (eg medical doctor, practice nurse, diabetes nurse specialist), experience of working with people with diabetes, and size, location and nature of their practices. During the interviews, participants from these two meetings described these demographic, geographical and clinical differences as 'small' differences, which they welcomed as they felt it allowed them to bring different perspectives to the topics they were discussing. They focused on the common ground they shared with other meeting participants and identified with one another based on the shared experience of liv- In addition, the HCPs indicated that they did not feel comfortable talking about the service that they worked in as they felt anxious that people with diabetes would confront them on the long waiting times or other issues they had with that particular service.

| Going off task to fill the void
Analysis of interview data indicated that participants across all three meetings felt they were able to work together. They reported that the content for discussion was relevant to them as users and providers of health services.

| Mixed methods results
The results of the mixed methods analysis are presented in Table 3.
Six key concepts relating to participants' experiences and group dy-

| Summary of key findings
The aim of this study was to compare participants' experiences

Interviews Interviews
Group cohesion In all three meetings, participants reported they were part of the group (like they belonged to the group) In the combined meeting, it was evident that there was a division between both stakeholder groups (eg both groups spoke at   each other across the each table as opposed  to with each other around each table).
In the people with diabetes-only and HCP-only meetings, participants reported that there were some 'small' differences between meeting members, but added that this was a good thing as it allowed them to bring different perspectives to the topics they were discussing In the combined meeting, people with diabetes reported that there was a 'complete clash of perspectives' between people with diabetes and HCPs; HCPs reported that people with diabetes and HCPs were 'two different sides of the divide'

TA B L E 3 (Continued)
result participants in the combined meeting went off task and made recommendations which were not entirely relevant for the intervention. In this study, involving patients and HCPs simultaneously in a consensus process was not found to be as suitable as involving each stakeholder group separately.

| Links to existing literature
In the people with diabetes-only and the HCP-only meetings, participants welcomed their diversity as it allowed them to hear different perspectives on the topics they were discussing. This finding is consistent with existing literature, with many theorists arguing that knowledge diversity can improve group performance by enhancing a group's ability to be creative and to discover novel solutions. [23][24][25] In these meetings, participants focused on their common ground and described being able to come together to make decisions that incorporated a range of perspectives. Previous research suggests that congruent groups-ie when group members are socially tied and share the same information-are more likely to be productive and successful. 26 The perceived lack of common ground between people with di-

| Strengths and limitations
One of the strengths of this study was the use of a mixed methods, convergent design which produced a more complete understanding of participants' experiences and group dynamics. It also allowed for the cross-validation of findings from each method resulting in more substantiated findings than sequential designs or quantitative or qualitative approaches alone. 9 The qualitative theme 'needing to feel safe' may explain the instances of dissonance between quantitative and qualitative data as participants completed the questionnaire at the end of each meeting while they were still sitting close to other participants. Some small groups even filled out the questionnaire to- This study is not without limitations. First, the questionnaire that was used to understand participants' experiences was based on non-validated questionnaire items. We were unable to conduct exploratory factor analysis to validate our questionnaire as our sample size did not meet the minimum criteria of 10 participants per questionnaire item. 33 However, given the increasing importance of evaluating PPI and other participatory research activities, 34 the questionnaire could be a useful tool in future studies which aim to understand stakeholders' experiences in similar participatory research contexts. Use of the questionnaire in future studies may allow for reliability testing and validation to be carried out. 35,36 Second, although the experience questionnaire suggested that there were no differences in participants' experiences between the three meetings, due to the number of participants, there was limited power to detect a difference (n = 31). Thus, the comparison of participants' questionnaire responses between the groups is used as only an indicator of participants' experiences. Given the small sample, we cannot rule out the possibility that differences between the groups could be detected had a larger sample size been used.
Despite using a range of strategies to recruit a representative sample of people with diabetes, another potential limitation of this study was the absence of people with type 2 diabetes in the combined meeting. As the attendance rate of people with diabetes at the combined meeting (60%) was much lower than the people with diabetes-only meeting (100%), it is plausible that people with type 2 diabetes did not attend because they knew there would be HCPs attending. Existing research has established that people with type 1 and type 2 diabetes have different experiences when managing their condition and engaging with HCPs. [37][38][39] Therefore, the involvement of people with type 2 diabetes in the combined meeting could have potentially changed the nature of the relationship between patients and HCPs and led to different participant experiences and group dynamics.
Finally, participants were given a choice to participate in an in-person or telephone interview. All participants chose telephone interviews due to time constraints and location convenience. This could be another potential limitation as researchers have previously expressed concerns about whether telephone interviews are appropriate for qualitative research. 40,41 These concerns are largely due to the absence of visual cues which may result in the loss of informal communication and contextual information, the inability to develop rapport or to probe and the misinterpretation of responses. 41 In this study, the quality of telephone data cannot be compared with in-person data as no in-person interviews were conducted. However, the researcher had considerable experience conducting phone interviews, maintained a friendly and engaging tone throughout and as mentioned previously, participants were found to be open and frank about their experiences.

| Implications
The results of this study provide much needed evidence on how different ways of involving patients and health-care professionals can lead to differing participant experiences and group dynamics. Patient and public involvement (PPI) in research is increasingly becoming a requirement in health research and for many research funders. INVOLVE, a national advisory body funded by the National Institute for Health Research (NIHR) in the UK, defines public involvement as research being carried out 'with' or 'by' members of the public rather than 'to', 'about' or 'for' them. 42 In this study, the lines between research participation and involvement were blurred, as is often the case with PPI. 43 People with diabetes were research participants in the consensus meetings, experience questionnaire and semi-structured interviews. However, their role in the consensus meeting was to discuss and make decisions about the intervention content and mode of delivery which could be viewed as PPI. 44,45 This study shows that the context and nature of involve-

| CON CLUS ION
Although the results of the experience questionnaire showed no differences in participants' experiences across the three meetings, the results of the observation and interviews highlighted that participants in the combined meeting had different experiences. In this study, involving patients and HCPs simultaneously in a consensus process was not found to be as suitable as involving each stakeholder group separately. The study provides much needed evidence on how different ways of involving patients and healthcare professionals can lead to differing participant experiences and group dynamics.

ACK N OWLED G EM ENTS
The authors would like to thank all the people who took part in this research.

CO N FLI C T O F I NTE R E S T
The authors have no conflicts of interest.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available on request from the corresponding author. The data are not publicly available due to privacy or ethical restrictions.