Patient and public involvement in doctoral research: Impact, resources and recommendations

Abstract Background and aim Patient and public involvement (PPI) has potential to enhance health‐care research and is increasingly an expectation, particularly for many funding bodies. However, PPI can be tokenistic, which may limit this potential. Furthermore, few studies report PPI processes and impact, particularly in doctoral research studies, which are seldom reported in peer‐reviewed papers. The aim of this paper was to explore the impact of PPI on two health‐related doctoral research studies and identify how PPI could be used meaningfully at this level. Method The PPI processes included (a) involvement of two ‘Research Buddies’ who informed the research design and ensured implementation of a booklet intervention was feasible for family carers, (b) data analysis workshops with ‘Research Buddies’ to identify emerging themes from practitioner interviews, (c) public and stakeholder involvement who informed data collection tool design, and the design of an intervention to help people with obesity who attend weight loss groups. Findings The application of PPI enhanced both doctoral studies by assisting data analysis; problem solving and improving recruitment rates; improving the usability and appeal of data collection tools and interventions; and developing implementation strategies. Patient and public involvement was considered a rewarding experience for both researchers and PPI contributors. Conclusion This paper demonstrates the value of PPI in doctoral research in relation to its impact on research processes, researchers and contributors. We also present recommendations on how PPI could be incorporated into future doctoral research, including resources required, planning PPI processes and involving PPI contributors in all stages of research.


| Background to patient and public involvement in research
Patient and public involvement (PPI) is defined as 'research being carried out "with" or "by" members of the public rather than "to", "about" or "for" them'. 1 For over two decades, researchers have been encouraged to consider patients at all stages of the research process. 2 A recent concept analysis of PPI in health and social care identified five operational definitions of public involvement, which were undefined involvement; targeted consultation; embedded consultation; co-production; and user-led research (Table 1). 3  across England, which focuses on applied health research and its implementation through partnerships with local organizations.

CLAHRC Greater Manchester (GM) research includes themes
such as end-of-life, wound care and kidney health, and, as an NIHR funded organization, is expected to include and report on PPI (see 5 for an example of a PPI evaluation within CLAHRC).
However, whilst this requirement has been seen positively more generally, it may lead to tokenism, with some research reporting little impact of PPI on the research, and instead paying 'lip service' to requirements for funding bodies. 6

| Impact of patient and public involvement
Consensus has emerged regarding some benefits of PPI, such as creation of user-friendly information and data collection tools, appropriate and effective recruitment strategies, and enhanced implementation and dissemination of findings. 6 Recommendations are available to help with planning, conducting, reporting and evaluating PPI, [6][7][8][9] and frameworks have been developed for this purpose such as the GRIPP checklist, superseded by GRIPP2. 10,11 A recent systematic review identified 65 such frameworks, yet the majority of these were used only by their developers, suggesting limited transferability. 12 Few studies have evaluated the impact of PPI, potentially due to the lack of robust tools, 13 and the wide diversity of study/contextspecific PPI aims and approaches. 14 Whilst it may not be possible to measure the impact of PPI quantitatively, the evidence base could be enhanced by reporting contextual details, aims and available resources. 15 Furthermore, Greenhalgh et al 12 question the utility of a 'one size fits all' framework for supporting, evaluation and reporting PPI in research, and suggest an adaptable 'menu of evidence based resources' (p.1).

| Patient and public involvement in doctoral research
Despite the potential benefits of PPI, researchers may be discouraged by the resources required to carry out 'best practice' involvement, such as those outlined by Harrison et al 16 . Whilst the recommendations are comprehensive, some activities may be particularly challenging for doctoral researchers. For example, 'securing funds for compensation' and 'appointing an engagement coordinator' are obstacles for students where funding is limited, and if they are not working as part of a team. In some instances, PhD topics and possibly research questions have been decided prior to the doctoral researcher becoming involved, and they may feel that public involvement is not 'worth it', given recommendations to start PPI prior to this point. Having 'regular face-to-face meetings' and 'allowing time for relationships to build' are also challenging within the time constraints of a PhD.
Though there are clear obstacles, PPI is possible at this level, as different levels of involvement are possible at all stages. Indeed, PPI is being carried out to a high level within doctoral research, but with notable exception, 17 the PPI processes and impact in doctoral research studies are seldom reported in peer-reviewed papers.
Moreover, PPI is sometimes not reported at all because of word count restrictions in journals. 6 This paper therefore aimed to address a gap in the literature, by presenting different PPI approaches taken in two doctoral research studies and outlining impact on research processes and outcomes.
This paper also sheds light on how PPI may be used meaningfully in future doctoral studies in relation to available resources, and instils a good working knowledge of PPI practices at the early career stage.
Finally, this paper aimed to answer calls for more 'user-friendly tools' for stakeholder recruitment and evaluation 18 to encourage researchers -particularly doctoral researchers -to engage with PPI.

| ME THODS OF PPI
Two different approaches to PPI were used in the two studies. Whilst the paper's aims are presented above in relation to providing some suggestions and tools for conducting meaningful PPI at doctoral and early career level, we have included the study-specific PPI aims and approaches to demonstrate impact. The approaches are framed around the conceptual definitions presented by Hughes and Duffy 3 (Table 1) and reported using the GRIPP2 Checklist 11 (Appendix D).

| Designing a dietary contract intervention for adults with overweight and obesity in low socioeconomic status areas (Study A)
The overall aim of Study A was to design an intervention to improve outcomes for adults with overweight or obesity attending weight loss groups in a low socio-economic status (SES) area. This population tends to drop out of research earlier, 19 engage less with 20 and have poorer adherence rates to weight loss interventions. 21 The first stage of designing the intervention was a qualitative study, which aimed to identify the particular barriers to lifestyle behaviour change for this population, reported elsewhere. 22 Following this, the findings were framed within the constructs of the COM-B model and the intervention was designed following the principles of the Behaviour Change Wheel, 23 described elsewhere. 24 The final intervention was a goal-setting and behavioural contracting booklet, focusing on common problematic dietary behaviours.

| Aims of PPI
1. To obtain feedback on the appropriateness of overall PhD research questions and aims.
2. To check readability and suitability of data collection tools throughout the study.
3. To finalize the intervention booklet design, checking appropriateness of content and usability.

| Method
The overall method employed in this study is an example of both Targeted Consultation (aim 1) and Embedded Consultation (aim 2) 3 (Table 1). To meet the first two aims, a CLAHRC team member invited members from their university-based PPI panel to be involved in the PhD. The already established panel was contacted first as feedback could be sought immediately, thus meeting recommendations to involve contributors as soon as possible. One member expressed an interest by contacting the researcher directly, and both the researcher and members felt they were suitable as they self-reported as overweight and lived in the same city as the study site. The panel member had also attended training provided by Citizen Scientist. A brief overview of the PhD and what would be required had been circulated during the invitation stage, but no person specification was developed given the panel was already established. The researcher contacted the contributor at various points throughout the doctoral research study, and documents (eg protocols, topic guides) and feedback were shared TA B L E 1 Overview of PPI within both studies in relation to PPI definitions and stage of research To meet the third aim, contributions were sought from members of the public using the service implementing the intervention (service users) and from staff involved in running the service (staff). Involving both groups in intervention development is suggested, given they are ultimately the end-users, 9 and are best placed to say if the inter- given the diversity in knowledge and abilities identified in the first qualitative study, 22 and a more participatory approach was not used given the time limitations. The researcher approached staff already involved in the research, and through the staff identified a suitable service user group to involve. No person specification was advertised.
The intervention booklet itself was informed by qualitative research and the wider literature, and groups were arranged once it was developed enough to distribute to the groups. Involving contributors at this stage meant there was something tangible on which people could have an opinion, which was potentially more time efficient. Also at this pre-feasibility stage, there was still time to make significant changes if required. Two separate meetings took place with these two groups, largely due to the practical difficulties of arranging joint groups:

Service user group
The researcher attended an hour-long weight loss group, run by the local authority within a low SES area. On that particular week, the usual staff member was absent, and the service invited the researcher to use the session to gather informal feedback on the first version of the intervention booklet. The booklet was introduced and distributed to ten service users, followed by some time allowing them to use the booklet. The remainder of the session was spent looking through responses and collecting feedback in small groups.
The researcher used open prompts relating to usability and suggestions for improvements. Those who contributed were given a £20 Love2shop voucher for their time, which was provided by the researcher's funder (NIHR CLAHRC GM). As no contact details were taken, it was not possible to provide feedback to this group regarding changes made as a result of the meeting. However, a lay summary of the PhD as a whole is planned to be made available to the service, which will describe the PPI and acknowledge its impact on the research.

Staff group
To gain feedback from the deliverers' perspective, a one-off meeting with staff members who would be delivering the booklet was arranged. Three weeks prior to the meeting, the booklet was circulated by email to eight staff members who had been involved in the initial research study. Three members of staff attended, and the remaining five, who could not attend, were invited to provide feedback by phone or email. In the meeting, staff members were asked to provide their initial impressions of the booklet. Each member was then asked to provide any further suggestions to improve the booklet's content, usability and appeal. The meeting, conducted in a central office within the local authority, lasted one hour. Staff contributors were sent the finalized booklet by email to see the changes made as a result of the meeting. Payments in exchange for staff's time were not permitted given staff were employed by the local authority. Staff were invited to provide feedback on their experience of PPI by email following final data collection (Appendix A).
The booklet was also emailed to the PPI panel member for any final points, to include input from someone outside of the service in case there was too much assumed knowledge. The PPI panel member was sent a £20 Love2shop voucher. As with the service user group, no feedback was given regarding changes made, but a lay summary will be sent.
The researcher took written notes throughout both sessions and from the email to ensure any suggestions could be incorporated into the next iteration of the booklet. Notes were recorded using Appendix C.

| Supporting family carers in providing care at home (Study B)
The overall aim of this study was to explore and evaluate the processes involved in the implementation and adoption of an evidencebased intervention for family carers (the 'Caring for Someone with Cancer' booklet) 25

| Method
The overall method employed in this study was Embedded Consultation given there was regular consultation throughout the research process. 3

| FINDING S
Our findings are framed around the two main impacts the PPI had on the studies: impact on research process and impact on researcher and contributor. We have included some tools in the Appendices, which researchers can adapt to use to assist recruitment and evaluation of public contribution in their studies. The authors found these tools helpful in their doctoral studies and are thus supported by an evidence base.

| Impact on research processes and outcomes
An overview of the changes made as a direct consequence of PPI feedback within both studies is presented in Table 2, alongside which aims these addressed.

| Study A
Overall, the researcher felt the PPI activities had contributed to increased skills and confidence during the doctorate. For example, from organizing PPI activities, leading meetings and making changes following feedback, the researcher felt they increased their skills in organization, communication, partnership building and use of lay language. The latter was particularly useful in developing an intervention that needed to be mindful of language and literacy.
No feedback was obtained from service users following the group session, which was an oversight. Feedback would have contributed to the overall evaluation of the PPI, but evaluation had not been planned at this stage, and no contact details were taken from these contributors for follow-up. Inviting staff contributors for feedback was easier given the on-going contact with them throughout the doctorate. As such, two of the three staff contributors who provided evaluation feedback by email. Both stated they would be happy to be involved again and enjoyed the experience. One staff member highlighted that they appreciated seeing the changes made from their involvement:

| Researcher reflections
Though the experiences were generally positive, we identified some potentially negative or challenging aspects of PPI within our studies. For Study A, using two approaches (panel member/faceto-face groups) worked well as staff were too busy to be involved in both aspects, and the feedback required fitted well with the experience and knowledge of the respective contributors. However, whilst there were benefits to involving staff as researchers and as contributors, such as enhanced rapport and overall engagement with the research, there was a lack of clarity on these two distinct roles. This became apparent in the feedback, which for one staff member focused using the booklet in their groups, rather than their contribution towards its design. The roles were explained at various points throughout the PhD and highlighted when asking for feedback on their PPI experience. The researcher relied on established relationships with staff from the earlier stages of research in recruiting staff as contributors. However, as previously mentioned, the researcher did not advertise or provide a person specification for the PPI role, which in hindsight could have clarified the two roles for staff. Furthermore, had the PPI meeting been arranged and clearly labelled as one larger PPI event for service users and staff, the roles and purpose may have been clearer. However, there were TA B L E 2 Impact on research processes and outcomes within both studies

Study/ Contributor(s) Impact on research process and outcomes
Study A PPI panel member • Patient information sheets/consent forms -improved readability, ensuring sheets were appropriately informative by identifying and removing unnecessary information and jargon, where possible (aim 2) • Wording of sensitive topic guide questions and study title -'low socioeconomic status area' was simply change to the '[target city] name' (aim 2) • The PPI panel member also read the PhD overview and qualitative study protocol, but had no comments to make (aim 1)

Service user group
• Layout and content of the booklet in terms of usability & appeal -making it more obvious that only one goal should be selected, choice of food images, making goals colour coordinated for ease of navigation. This was particularly important given the issues around engagement in this population, and the language and literacy barriers identified in the initial qualitative study 24 (aim 3)

Staff
• Content of the booklet -adding a lifestyle goal relating to increasing water consumption, ensuring included guidelines around fat and salt intake were correct and in line with the service's recommendations (aim 3) • Design and content of the booklet, including usability and appeal -changing images of food that might be more familiar to the population, adding easy to follow recipes (aim 3)

Study B
Research buddy • Offering potential solutions when faced with recruitment problems, such as gaining access to, and recruiting additional sites, when a site withdrew from the study (aim 1) • Developing emerging themes, by drawing on their experience, for example the categorization of family carers by community nurses and the interactional work which may have an impact on whether or not the booklet is delivered (aim 1) • Offering alternative explanations to themes identified in the data, by reading anonymized transcripts and participating in data workshops (aims 1 and 2) benefits to having separate groups, including ease of arranging, reduced resources and avoiding potential difficult situation of paying  Students should therefore be supported financially and encouraged by their institutions and supervisory teams. In this instance, the doctoral researchers had funding available through the NIHR to support PPI costs and thus were supported to include PPI in their research. Ensuring sufficient resources therefore is a particularly important consideration at the institutional level, to allow the calculation of PPI funding into the doctoral research budget. However, not all doctoral students will have the resources for co-production or user-led type PPI. This paper highlights that doctoral researchers can still conduct meaningful PPI, even with lower level involvement or minimal resources ( Table 2).

| D ISCUSS I ON
As discussed, the 'Research Buddies' in Study B reported their involvement to be valuable, contributing to both the progress and quality of the research. This was also reported by Mann and col- Whilst there are many benefits of including PPI in the research process, there are some potential issues for both the contributors and the researchers, which must be acknowledged. to include staff members within the research aspect, but to include them as stakeholders within more of a co-production approach (see Table 1). However, given that staff had limited time, particularly following cuts to their service, this may not have been feasible. count restrictions, as previously noted. Reporting PPI impact separately may also make identifying such evidence easier and making for more detail regarding the PPI activity available to others, given the sparsity of evidence identified. 18

| IMPLI C ATI ON S AND RECOMMENDATIONS FOR CONDUC TING PPI AT P OS TG R ADUATE LE VEL
We have demonstrated that conducting impactful PPI with limited resources is possible at this level, highlighting that PPI should not be dismissed on grounds of cost or time. University policies should ensure PGR students are provided with training and resources, as well as supervisory (or other) support to include PPI, both for the quality of the research and the development of researcher skills. An evidence gap exists in relation to how best to facilitate involvement with those with time commitments (eg staff, current carers) and hard to reach groups, for example through utilizing technology such as Skype.
Given the effect PPI had on the research presented in this paper, we have combined our experience of PPI at this level with the literature and recommendations currently available. 30 Recommendations summarized in Table 3 build on suggestions presented in our blog, 33 and in Figure 1 are presented in relation to both the stage of doctoral research and resources available to the researcher.

| CON CLUS ION
This paper has outlined why PPI at postgraduate level is important. Patient and public involvement does not have to result in huge changes to be meaningful; through small changes, our PPI resulted in improved quality, and importantly was meaningful to our contributors. Learning to conduct meaningful PPI at postgraduate level is also important in shaping future generations of researchers, thus impacting on the quality of future research. For doctoral researchers looking to conduct meaningful PPI, we have summarized our recommendations and suggestions on how this can be achieved at different stages, dependent on available resources. To ensure PPI at the doctoral level is non-tokenistic, it must be supported and funded sufficiently, and therefore costed into doctoral programmes, and encouraged at both the supervisory and institutional level.

ACK N OWLED G EM ENTS
We thank all staff and members of the public who gave their time to contribute to our research. We also thank Dr Sarah Cotterill and Dr Sarah Peters for their helpful feedback on our first draft.

CO N FLI C T O F I NTE R E S T
The authors declare no conflict of interest.

Recommendations
References and links Tools provided

(1) Consider your aims
What is the purpose of the PPI and what you would like to achieve from it, for example do you need to clarify and further refine your research question, or would the data collection benefit from some expert input?
Having clear aims will help avoid tokenism, and will show clear purpose for your work You can access INVOLVE's library of past and current work for ideas and inspiration, 34 and the definitions outlined by Hughes and Duffy 3 may be useful to consider the type of PPI (2) Consider your resources Is funding available to you to support PPI costs? How much time you have (within your current study/PhD)? Even if your resources are low, activities can still be high impact and/or not tokenistic Recommendations in Figure 1 should be used in accordance with public involvements standards, such as those laid out by INVOLVE 35 Figure 1 (

DATA AVA I L A B I L I T Y S TAT E M E N T
The data that support the findings of this study are available from the corresponding author upon reasonable request.  Table 2 5. Reflections/critical perspective Comment critically on the study, reflecting on the things that went well and those that did not, so others can learn from this experience 10-11