Patient involvement in developing a patient‐targeted feedback intervention after depression screening in primary care within the randomized controlled trial GET.FEEDBACK.GP

Abstract Background Patient and public involvement (PPI) is increasingly required in mental health services research. To empower patients to actively address depression, the GET.FEEDBACK.GP study evaluates a patient‐targeted feedback intervention after depression screening using the Patient Health Questionnaire (PHQ‐9). Objective To refine the patient‐targeted feedback from a previous study within a participatory research team (PRT) by conducting workshops to investigate patients' needs and preferences for feedback. To evaluate the process and outcome of PPI. Design Patient and public involvement was carried out on the levels of collaboration and consultation. A PRT of patient partners and researchers planned and conducted three workshops with patients. Patients' needs were investigated using a focus group. Participants prioritized needs, discussed feedback drafts and evaluated two drafts using cognitive debriefings. Researchers of the PRT communicated the results at project level. PPI was evaluated using the Public and Patient Engagement Evaluation Tools (PPEET). Setting and Participants A purposeful sampling of N = 12 patients with experiences of depression participated in at least one workshop. Results Relevant content‐related needs about feedback (eg no distinction between severe and moderate symptoms), recommendations for action and patient‐relevant information were considered. Needs for comprehensible, valuing, nonstigmatizing language and design elements (eg dimensional bar) were implemented. Workshops and PRT were positively evaluated. Discussion and Conclusions Patient and public involvement influenced the content, wording and design of the feedback. Strengths include two levels of PPI, methodical diversity and purposeful sampling. Limitations include the lack of inclusion of patients who are unaware of their depression. The evaluated PPI concept can be useful for future studies.


| INTRODUC TI ON
In the field of research and development of patient-relevant healthcare interventions, there is a traditional dominance of researchers' perspectives. 1 Researchers often try to anticipate patients' needs and preferences from an outside perspective based on the literature and known evidence instead of including the target group in the development of interventions, technologies or patient information relevant to their care. [2][3][4] Therefore, this paper describes the patient and public involvement (PPI) within the randomized controlled trial (RCT) Get.Feedback.GP (registered at ClinicalTrials.gov NCT03988985) 5 during the intervention development phase as well as its evaluation from the perspectives of involved parties (patient partners, participants and researchers) regarding process, impacts and challenges of the PPI.
Patient and public involvement is of increasing importance in health services research to deliver health services that meet patients' needs. The United Kingdom can be seen as a model for advancing and implementing PPI in the research process. 6 For example, the National Institute for Health Research (NIHR) is only funding research including PPI. 7 Supported by the NIHR, INVOLVE 6 and further initiatives, such as the Patient-Centered Outcomes Research Institute (PCORI) 8 in the United States and the Strategy for Patient-Oriented Research (SPOR) 9 in Canada, have been established to enhance PPI. Commonly, PPI is considered desirable to obtain findings and to focus on outcomes more relevant for the targeted patient group. [10][11][12] In Germany, funders and researchers recognize the relevance of involving patients in health services research. 13,14 Currently, however, initiatives as internationally implemented do not exist, and systematic PPI in German health services research is scarcely practised. 15,16 While research is often about, on or for patients, PPI is defined as research with or by patients and members of the public. 17,18 Contributing on different levels, patients can provide their valuable knowledge and expertise in single research phases (consultation), collaborate continuously in a research team as patient partners (collaboration), or even initiate and conduct research independently (user-controlled research). 18 Patients' perspectives can be used by researchers to improve the quality and practical relevance of their research. 1,19 PPI can help to increase patient participation in studies by making study materials and recruitment procedures suitable for the investigated target group. 1 In addition, PPI can help to make patient information material more relevant, readable and understandable to patients. 20 Regarding implementation and dissemination of study results, positive findings of PPI were reported (eg by presenting research findings in a lay user-friendly and poignant manner). 1 Currently, there are some recommendations and suggestions about how to carry out and report PPI. 17,21,22 However, it remains unclear how PPI, when indicated, should be conducted and evaluated systematically. 23,24 Several reviews criticize the poor and inconsistent reporting of PPI in studies (eg reporting on the representation of the involved sample 25 ) and its lack of systematic evaluation, 1,21,26,27 which results in a weak evidence base and makes it difficult to understand how and under what circumstances PPI has an impact on research. 21,28 Furthermore, it is criticized that researchers tend to perform PPI in a tokenistic manner 1,22,28 (eg to comply with study and funders' policies 1 ), thus impairing the contribution and impacts of PPI on research. Studies show that the extent of PPI impact depends, for example, on chief investigators' support 28 and preparedness to include PPI in their research, 17

on researchers'
understanding regarding the purpose of PPI 29 and on the quality of the relationship between the researchers and involved patients. 17,28 Since PPI is a complex intervention, its evaluation is also complex and should include process, outcome and impact. 1 Evaluating the benefits and cost-effectiveness of PPI in research is crucial to achieving effective PPI and to identifying factors affecting the impact and thereby overcoming tokenism. 21,23 Within the field of mental health services, PPI has increased over time 30 and can be advantageous. 31 PPI showed positive impacts on recruitment success 30 and on prioritization of clinically relevant outcomes on a systematic review. 32 Gillard and colleagues 33 found that, compared with conventional researchers, there were differences in how patient partners carried out interviews with patients and especially how they analysed the transcripts (eg with a stronger focus on patients' experiences and feelings). Additionally, in the development of technology-based interventions for people with mental illnesses, 34 there is evidence that involving potential users helps to design them to be more responsive and user-friendly. 35 Therefore, GET.FEEDBACK.GP was designed by involving patients using mental health services during the intervention development phase.

| Get.Feedback.GP
Although recommended, [36][37][38] there is a lack of international high-quality studies proving the efficiency of a depression screening 39 coupled with accurate diagnosis, appropriate treatment and follow-up. 38 The diversity and purposeful sampling. Limitations include the lack of inclusion of patients who are unaware of their depression. The evaluated PPI concept can be useful for future studies.

K E Y W O R D S
depression, feedback, patient and public involvement, patient health questionnaire (PHQ-9), patient participation, primary health care, qualitative research DEPSCREEN-INFO RCT demonstrated that a patient-targeted, written feedback intervention after depression screening with the Patient Health Questionnaire (PHQ-9) 40 had a positive effect on the severity of depression 41 and cost-effectiveness 42 after six months in patients with coronary heart disease. The suspected underlying mechanism was that patient-targeted feedback of the screening result directed at patients empowers them to actively manage depression. In addition to the written feedback of the screening result, the feedback also included graphical elements indicating patients' depression severity as well as depicting it in relation to the general population, recommendations for action, information about depression and health-care services, and contact information for the local university psychosomatic outpatient clinic. Overall, these findings underline the potential for the dissemination of the feedback intervention.
Therefore, the generalizability of these results with respect to primary care will be tested in the multicentre RCT GET.FEEDBACK.
GP. The RCT will test the effect of a patient-targeted feedback intervention on depression severity. Full details of the trial can be found in the registered study protocol. 5 In summary, PPI in mental health research can have an impact on research process, interventions, data analysis strategies and results.
Overall, it is recommended to involve targeted patients as early as possible. 18

| AIMS OF PPI
By involving patients with experiences of depression, we investigated their needs and preferences for a feedback intervention after depression screening. The feedback should be comprehensible and acceptable and encourage patients to actively deal with their depression. The process, impact and outcome of PPI on patients and researchers should be satisfactory.

| Design
A participatory research design involving patients with experiences of depression on the level of collaboration through the establishment of a participatory research team (PRT) and on the level of consultation using focus groups, prioritization tasks, group discussions and cognitive debriefing was chosen ( Figure 1). Researchers of the PRT referred to available recommendations for PPI to prepare the PRT and the workshops. 18

| Participatory research team
Three patient partners with experiences of depression were recruited to the PRT to assist at all stages in the development process of the patient-targeted feedback intervention (Figure 1). Former study participants who agreed to be contacted again for study purposes were contacted by the researchers via e-mail and selected based on their experiences with depression and their gender.
Patient partners were experienced in peer counselling (n = 1), leading support groups (n = 1) or contributing in participatory research (n = 1). The three researchers of the PRT (psychologists who work in academia for more than 10, 3 and 1 year) were experienced in PPI and patients with depressive disorders. During the planning phase, one patient partner dropped out after two months for health reasons. Patient partners continuously participated in 13 study meetings to plan the development of the feedback intervention as well as to comment on ethical factors and study course. They prepared PRT meetings (eg suggested methods and guiding questions for the workshops), with the project team (PT) deciding and specifying the workshops. In addition to planning, they were involved in conducting the three workshops and in interpreting the results. They also helped to recruit participants for the workshops. They checked comprehension of structure, content and material of the workshops from the patients' perspective. Furthermore, they contributed to the preparation of this manuscript. Patient partners received an expense allowance of 20€ per hour.

| Workshops
The PRT conducted a series of three three-hour workshops.
Results from each workshop were included in the next workshop.

| Participants
Participants were included if they had recent or past experiences with a depression diagnosis. The PRT aimed to include patients' differing with respect to age, gender and date of first depression diagnosis to include different experiences (purposeful sampling 51 ).
Furthermore, we aimed to include patients from different care settings. Researchers recruited the participants through different approaches such as a primary health-care centre, an outpatient clinic for psychosomatic medicine and psychotherapy and an outpatient clinic for geriatric psychiatry via an information sheet. Patient partners recruited participants through peer support contacts, support groups and private contacts. All participants provided written informed consent. An expense allowance of 50 € per workshop was paid to participants. The study was conducted in accordance with the Helsinki Declaration and was approved by the Ethics Committee of the Medical Chamber of Hamburg (reference number: PV5975).

| Data collection and analyses
To investigate patients' need for a feedback intervention, participants were asked about information and layout needs during a focus  a standardized documentation sheet including the questions mentioned above. Participants' rankings were analysed descriptively.
The content of the discussions and comments in the cognitive debriefings were organized within a matrix and analysed by the PRT.
Participants evaluated format and structure, characteristics of the moderators as well as the scope and meaning of all workshops with an adapted standardized tool for teaching evaluation 54

| Participants
Twelve patients with experiences of depression participated in the workshops (first workshop, n = 6; second workshop, n = 7; third workshop, n = 9). They were between 24 and 81 years old (median 60); seven were female, and five were male; and three were born abroad. Four held an A level ('Abitur/Fachabitur'), four had graduated from middle school ('Realschule'), one had not graduated, and three were missings. Participants' first diagnosis of depression was one to 49 years ago (median 11). Eight participants reported currently suffering from depression, and ten reported suffering from comorbidities (n = 5, mental illness; n = 9, physical illness). Participants had sought treatment from general practitioners (GPs, n = 9) and had undergone psychotherapy (n = 10) or psychopharmacotherapy (n = 5). They were recruited through patient partners (n = 5) as well as through researchers via an outpatient clinic for psychosomatic medicine and psychotherapy (n = 1), an outpatient clinic for geriatric psychiatry (n = 4) and a primary health-care centre (n = 2).

| Recommendations for the feedback intervention (workshop 1 and 2)
Recommendations for the patient-targeted feedback intervention included content-related aspects, language and design elements.

| Content-related recommendations
Participants mentioned content-related recommendations for the feedback of the PHQ-9 results and recommendations for action and ranked them according to their importance (Table 1).

| Language-related recommendations
Participants agreed with language-related recommendations from the literature about short sentences 58 and simple, concrete and inclusive language (eg related to gender 45 ). They emphasized the importance of a comprehensible, valuing, and nonclinical and nonstigmatizing wording focused on the patient.

| Comprehensibility and acceptance of two feedback drafts (workshop 3)
Participants understood the message of both drafts (Appendix 1).  Figure 2 shows the impacts of PPI on the development of the feedback intervention.

| Impacts
Nevertheless, some needs (eg preference for personal feedback, inclusion of telephone helpline) could not be realized because of the study design or lack of evidence. The final feedback is available from the authors upon request.

| Workshop evaluation
Participants rated the workshops' format and structure, characteristics of the moderators as well as the scope and meaning of all three workshops on a five-point Likert-scale from 'strongly disagree' to 'strongly agree' (Appendix 2). They agreed and strongly agreed with the workshops' clear structures (item 2), moderators' openness to criticism (item 5), friendliness (item 7) and helpfulness (item 8).
They moderately to strongly agreed with the illustrations used in the workshops (item 1), moderators fostering debates about the topics (item 6) and the meaning of the topics (item 9). One participant in workshops 2 and 3 disagreed that the workshops had an adequate pace (item 10), and approximately half of the participants rated the number of topics covered in the workshops as too many (item 11).
Regarding process and outcome evaluation, participants as well as patient partners rated the communication and support for participation as well as impacts and influence of the workshops, whereas researchers of the PRT rated the integrity of the design and process (PPEET; Figure 3, Appendices 3-5).
Participants appreciated the exchange, the respectful conversation and the moderation through patient partners (eg felt more accepted, good atmosphere). However, participants saw needs for improvement regarding possibilities to prepare themselves, moderation and organizational aspects (eg regional catering). Patient partners emphasized that they were integrated in the workshop group and that they had enough space to represent their own position.
From their point of view, workshops were helpful for the development of the feedback and characterized by creativity and integration of patients' different perspectives.

| Participatory research team evaluation
Patient partners and researchers of the PRT were satisfied with the team and thought that it was valuable for the project (Appendices 4 and 5). From the patient partners' perspective, the communication 4. … contain the information that the PHQ-9-result does not replace any diagnosis and that it is necessary to clarify the depressive symptoms.      11. … contain concrete next steps instead of a list of treatment recommendations. Using the short form of the information was a compromise between methodological reasons from the project team and patients' needs for patient-relevant information.

Internet addresses with trustworthy information Including internet addresses should enable access to trustworthy information. Information about study
Inclusion of study information (eg 'You receive 10 € per telephone interview.') Researchers decided to add study information to inform patients and to increase the motivation to participate. From the perspective of patient partners, this information was not necessary in the feedback. and support for participation as well as impacts and influence of the workshops regarding the feedback were positive (Appendix 4).
Researchers rated the integrity of the design and process of the PRT mainly positively (Appendix 5). Patient partners reported their initial experience of being confused because of 'several unknown terms'. They rated teamwork as 'on an equal footing' and emphasized the strength of the team to take into account different aspects and perspectives. They identified a need of a written contract and of more time and structure for the preparation of the workshops.
Researchers were interested in developing PPI-related skills regarding PPI during proposal writing, realistic expectations of patients' resources and competencies, and guidelines and templates for organizational aspects of PPI (contracts, payment, data protection, authorships, acknowledgement, rights and obligations).

| Strengths and limitations
We have reported all relevant aspects of process, outcome and impact of PPI following the recommendations of the GRIPP2 reporting checklists. 21 Our PPI process was in accordance with the recommendations of the INVOLVE initiative for implementation. 18 Our PPI study met the following six conditions identified by the RAPPORT study, 65  Even though PPI is demanded by some funders, 13,69 there is a lack of essential strategic and infrastructural support of PPI 10 in German in mental health research. 62 While consultative patient participation is practised sporadically, we have no knowledge of involvement initiatives in German mental health research that systematically apply the concept of collaborative involvement of patients in the research process. 15 Further changes in the funding structures are necessary, which should meet the requirements (eg establish participatory advisory group) and facilitating factors of participatory research.

| Implications
It is known that PPI can increase the practical relevance of outcomes, 1 ecological validity, 19  The effectiveness of the participatory-developed feedback intervention after depression screening will be tested by the multicentre RCT GET.FEEDBACK.GP. The data collection started in July 2019 and is expected to be completed in July 2021. Patient partners and participants will be involved at the end of the RCT to create a report of the results that is comprehensible to patients.

ACK N OWLED G EM ENTS
GET.FEEDBACK.GP is a multicentre randomized controlled trial, and its success is from good interdisciplinary collaboration. Applicants

CO N FLI C T O F I NTE R E S T
There are no conflicts of interest.

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request. The data are not publicly available due to ethical or privacy restrictions.

O RCI D
The workshop was clearly structured.
The learning goals of the workshop were clearly defined.
4. Additional helpful resources were given (eg, handset, literature, internet access etc).
Characteristics of the lecturers 5.
The lecturers were open to criticism.
The lecturers encouraged to critically deal with the topics covered.
The lecturers were friendly and open in dealing with the group.

Scope and meaning
9. The meaning of the topics covered was high (eg, for peer counseling etc).
10. The content of the workshop was covered in an adequate pace.
11. The amount of topics covered in this workshop was too comprehensive.
The supports I needed to participate were available (eg, travel, childcare).
3. I had enough information to contribute to the topic being discussed.
I have enough information to be able to carry out my role. -

Sharing your views and perspectives
4. I was able to express my views freely.
5. I feel that my views were heard.
A wide range of views on the topics discussed was shared.
The individuals participating in the workshops represent a broad range of perspectives.
Impacts and influence of the workshops 8. I think that the workshops achieved their objectives.
The workshops are achieving their stated objectives.
-PP ---1 1 -9. I am confident the input provided through the workshops will be used by the research group. -PP ---1 1 -10. I think the input provided through the workshops will make a difference to the work of the research group.
I think that the work of the workshops makes a difference to the work of the research group.

Final thoughts
11. As a result of my participation in the workshops, I am better informed about the recognition of depression.   2. The supports I needed to participate were available (eg, travel, childcare).
---1 1 - 3. I had enough information to contribute to the topic being discussed. ----2 -Impacts and influence of the participatory research team 8. I think that the participatory research team achieved their objectives.
---1 1 -9. I am confident the input provided through the participatory research team will be used by the research group.
---1 1 -10. I think the input provided through the participatory research team will make a difference to the work of the research group.

Strongly agree Missings
Integrity of design and process 1. The perspectives of those who will be most affected by the outputs of this project were reflected through those who participated in the engagement.