Stakeholder involvement in the development of trial material for a clinical trial

Abstract Background Stakeholder involvement includes not just patients and public, but also those delivering treatment for example clinicians and students. Each stakeholder brings unique experiences to the process. The aim of this stakeholder exercise was to explore readability and understanding of the trial material for the future trial to be conducted by the authors: Biomechanical Effects of Manual Therapy—A Feasibility Study. Design Volunteers from identified stakeholder groups were provided with trial material which included the information sheet, consent form, questionnaires and home management booklet. They provided feedback on content (readability, understanding) and style (font, layout). An additional document was provided with genres of pictures to choose the most appropriate style to be used in the booklet. Readability formulas were used to calculate reading age before and after feedback to objectively measure ease of reading. Results The public group provided a layperson's perspective to clarify the information sheet for patients, whereas practitioner and intern groups indicated where information could be clarified. The reading age of all documentation decreased following feedback; however, templated sections of the documentation did not. The majority (87%) of volunteers chose coloured classic cartoons for the booklet. Conclusion This process highlighted the importance of involving different stakeholder groups in the development of research materials as each group made a unique contribution. Readability and understanding of the trial material were improved, feeding back into the consent process contributing towards fully informed consent. Patient or Public Contribution Public helped develop materials for a future trial but not with manuscript preparation.


| BACKG ROU N D
It is suggested that a large quantity of health research in the United Kingdom is avoidably wasted. 1 Some of this research waste has been attributed to failure to publish findings; unclear reporting of findings; and failure of new research studies to systematically review previous work in similar fields, resulting in unnecessary replication. 2,3 Patients and members of the public have an interest in and role to play in research waste reduction. 2 Involving and collaborating with patients and members of the public can improve study design, methods and relevance of research. 4,5 Together with researchers, they can be involved in decisions regarding how studies are prioritized, designed and conducted. 2 This improves research by making it more relevant to the patient. 2,6 Patients and members of the public can also bring different perspectives from those who are conducting the research, such as a lay person's perspective, as well as the lived experience of the condition or caring for someone with the condition. 5 Patient and public involvement (PPI) may be used to aid development of trial material for patients which may have an impact in the recruitment and retention of trial participants, 4,6,7 as well as the consent process. 8 Stakeholder involvement is inclusive participation of all stakeholders in health-care research, from the grass roots student population to clinicians and from the public to patient. Each volunteer group brings unique experiences, knowledge and skill sets to the development of health-care research creating an authentic partnership of self-identified volunteers working towards a common goal. 9 While much of the research carried out and published in the area of stakeholder involvement is centred around PPI, there is only a small amount of literature extending the sphere of collaboration to professional members of the health-care community who may have an impact on, or be impacted by, the subject under investigation. 9 It is suggested that health-care professionals may provide personal insight into clinical trial development, as well as a breadth of knowledge relating to clinical trial design and interventions. 10 Equally, health-care professionals may provide insight into whether the trial would be practical, useful or usable in a particular setting. 11 By combining the public and patient experience and perspective, together with the knowledge and personal insight of a health-care professional, a well-rounded unique view of the study can be obtained and can enhance the development of a clinical trial. 10 Equally, involving patients and members of the public, together with healthcare professionals in research is vital as they are the end-users. 10 While the primary goal of health-care research is to generate new knowledge, according to the Declaration of Helsinki, this cannot take precedence over the interests and rights of human research participants. 12 Informed consent is the cornerstone of ethical research. 13 In health-care research, when informed consent is given, it indicates an individual has made a fully informed and voluntary decision to take part in the research trial. Therefore, the onus lies with the team conducting the trial to support the consent process by providing the participant with adequate details of the trial reasoning and procedures. This information should provide potential participants with all the materials they require to make an informed decision about their participation in the trial. 14 For example, the purpose of the research; potential benefits and risks; the right to refuse or withdraw; and treatment alternatives. 12 However, the quantity of information given can overwhelm participants 15,16 and may lead to a participant's lack of understanding of key aspects of the trial which could be considered crucial information to those who are considering their participation. [17][18][19] This may be due to the information being supplied in a complex way, not designed to support a participant's informed decision process, or using language which is better suited to a medical professional rather than a trial participant or lay person. 19,20 An additional complexity in the consent process occurs when potential participants lack adequate literacy to be able to read and understand the information sheet and consent form. The National Literacy Trust 21 indicates that the 16.4% of adults in England have 'very poor literacy skills' and are defined as functionally illiterate (a reading age at or below the average 11-year-old). Up to 74% of studies relating to informed consent and participant comprehension of research information do not assess participant comprehension, which may contribute to a lack of participant understanding. 18 One way to assess comprehension of text is to use readability formulas. These formulas generate automated numerical estimates of readability of a text. The readability formulas focus on the average number of syllables in a word (word length) and the number of words in a sentence (sentence length). Three readability formulas considered objective measures of text comprehension are: • The Flesch Reading Ease Formula 22 will output a number ranging from 0-100, the higher score indicates easier reading. A score of 90-100 can be understood by an average fifth-grade student (10)(11) year old); 60-70 can be understood by an average eighth or ninth grade student (13)(14)(15) year old); 0-30 can be understood by an average university student (18)(19)(20)(21) year old).
• The Flesch-Kincaid Grade Level 23 indicates a school grade level (USA) which the average student in that grade would be able to read.
• The Gunning Fog Formula 24 is a scale that indicates syllable and sentence length. A Fog score of 5 is readable, 10 is hard, 15 is difficult and 20 is very difficult.
However, the formulas may not be an appropriate predictor of comprehension for short question surveys or questionnaires. 25 In addition, many studies do not consider additional impacts on readability such as layout, appearance, font size, and use of diagrams or pictures. 26 According to the British Dyslexia Association, 27 up to 10% of the British population has some degree of dyslexia. As such, Dyslexia Guidelines should be considered when designing an information sheet or consent form for ease of readability.
The use of pictures in health-care booklets or information sheets enhances engagement of patients 28 and can facilitate comprehension of the written word. 29 Patients have indicated that booklets with pictures are easier to read than text alone, even when the written text is identical. 28,30 Additionally, photo or picture elicitation has been used in social science research in a variety of ways. 31 Images can elicit moods or feelings distinct from written text. 31,32 As such, consideration of the photograph or picture genre can aid understanding or create misunderstanding depending on the genre choice. 32 In the case of health-care research, this could mean the difference between the participant feeling informed and reassured, versus feeling fearful or that their condition is not being taken

| Ethics
A Stakeholder Involvement Process is a collaborative process, and not considered research by the NHS. 33 Following completion of the HRA NHS Review decision tool 34 and under the advice of local ethical guidelines, ethical approval was not required.

| Volunteer Recruitment
Five volunteers were sought from each the following groups, each group was recruited via separate advertising strategies. All advertising and recruitment material specified that taking part was voluntary, unfortunately the project did not have a budget to reimburse volunteers.
• Members of the public who have experienced low back pain were recruited via the university public and patient partnership. It was hoped that the public group would be able to give feedback from a lay person's perspective, which would be helpful in ensuring the potential trial participants would be able to read and understand what is involved with taking part in the trial.
• Registered chiropractic clinicians who have been in practice for at least two years were recruited via an advertising email. Emails were sent to Chiropractic Institution tutors who met the relevant criteria. It was expected that the practitioner group would be able to identify areas of missing information or pertinent information not highlighted sufficiently.
• Chiropractic intern students (final year chiropractic student clinicians) were recruited via an advertising email. Emails were sent via the Chiropractic Institution tutors. It was expected that the intern group would be able to give feedback from their strength in theoretical knowledge. Equally, the intern students work in the same building where the trial will take place, as such it was expected their feedback may highlight practical considerations related to carrying out the trial.
The collaboration process was carried out from the beginning of April 2019 until the end of June 2019.

| Collaboration process
All volunteers were sent the trial documentation, either as an electronic version (by email) or as a paper copy, depending on their preference or level of IT skills. These included the information sheet, consent form, questionnaires, and home management booklet. An additional document which included different genres of pictures for potential use within the home management booklet was also provided. All documents sent to volunteers were single spaced, using the font Calibri in a size 11 (Microsoft® Word for Microsoft Office 365, USA). Volunteers were asked to provide feedback on the documentation, this could be completed either as 'tracked changes' in a Microsoft Word Document or as changes made on a paper copy. The individual documents and requested feedback are outlined below.

| Participant information sheet
The information sheet underpins the participant understanding of the trial, what the purpose of the trial is, what their role in the trial will be, withdrawal process and data management. The Health Research Authority template and the university template was used to complete the information sheet. 35 Feedback was requested on: • Content: a. How easy was the wording of the information sheet to read? b. Was the information sheet clear and easy to understand?
• Style: a. Was the information sheet easy to read, specifically looking at style layout, font, font size and line spacing)?
• Were there any additional changes to the document that volunteers would like to add? These did not necessarily have to be related to readability and understanding. Practitioners and intern students were asked if they felt there was any missing information or additional information which may be helpful to a participant.

| Participant Consent Form
The consent form is a signed agreement between the researcher and the participant indicating that the participant has read and understood the information sheet and is happy to be a part of the trial. The Health Research Authority template and university template was used to complete the consent form.
Feedback was requested on.
• Content • How easy was the wording of the consent form to read?
• Was the consent form clear and easy to understand?
• Style • Was the consent form easy to read, specifically looking at style (layout, font, font size and line spacing)?

| The questionnaires
Two questionnaires will be used in the trial, the Bournemouth Questionnaire 36 and the Roland Morris Disability Questionnaire. 37 These questionnaires are used as patient-reported outcomes measures. 38 As these are validated, the questions cannot be changed without influencing the validity of the questionnaires. Equally, the readability formulas are not very accurate for short questions. 25 For these reasons, feedback was only requested on style: a. Were the questionnaires easy to read, specifically looking at style (layout, font, font size and line spacing)?

| The Home Management Booklet
The evidence informed home management booklet was developed and compiled by the research team from recent published guidelines relating to non-invasive treatment of acute low back pain. Volunteers were sent the wording of the booklet, without illustrations.
Feedback was requested on: • Content: • How easy was the wording of the home management booklet to read?
• Was the home management booklet clear and easy to understand?
• Style: • Was the home management booklet easy to read, specifically looking at style (layout, font, font size and line spacing)?
A document which included different genres of pictures was also sent. Volunteers were asked to select the picture which gave the feeling of reassurance, without giving the feeling that the condition of the patient was not being taken seriously. Pictures were Most detailed Least detailed TA B L E 1 Picture genres to be used in the Home Management Booklet (listed from most detailed picture genre to least detailed picture genre for each category). The picture genres were provided to volunteers for each category F I G U R E 1 In the category of anatomy of the back, an example of a coloured anatomically correct anime (detailed) divided into two categories, anatomy of the back, and postural and ergonomic pictures. Pictures were organized from most detailed, to least detailed (see Table 1). Volunteers were supplied with an example of each genre; however, due to copyright, not all pictures could be included in this article. In the category of anatomical pictures, examples of a coloured anatomically correct anime can be seen in Figure 1, and a classic coloured cartoon can be seen in

| Feedback
A thematic framework analysis of the feedback was carried out. 39 This is a systematic approach to analysing qualitative data, where commonalities and differences are identified. 40 Its defining feature is the development of a matrix whereby rows (in this case, each volunteers feedback) and columns (themes) provide a structure in which to systematically manage data and analyse it by theme and individual. 39,40 The matrix allowed the researcher to organize the data into the three groups to analyse common themes or common aspects of feedback highlighted by each group. 39 Feedback was combined with the original documents into one Microsoft Word Document (Microsoft® Word for Microsoft Office 365, USA), using 'tracked changes'. The primary researcher made changes to the original documents accordingly. Changes which reduced the sentence length and number of syllables in the wording were made as this may reduce reading age and increase understanding of the documents. For layout, font, font size and spacing, all feedback and comments were considered together with the dyslexia guidelines. 27 Regarding the pictures, all feedback was collated and the genre which was chosen most frequently was used for the home management booklet. All modified documents were sent to the remaining researchers for feedback and discussion.

| Readability formulas
All documents were tested using the readability formulas of Flesch Reading Ease Formula, Flesch-Kincaid Grade Level, Gunning Fog Formula, 41 before and after consultation.

| Demographic data
A total of fifteen volunteers took part in the process: • Members of the public: 4 females, 1 male; age range 38-73 years.

| Feedback
In general, the public group provided more feedback than the remaining two groups. The thematic framework analysis identified themes within the feedback. Once the thematic matrix was complete, common within group themes were identified (see Table 2). The public group and practitioner group provided feedback on structure (font and layout), the student intern group did not.
Feedback included making the font larger, as well as increasing the spacing between lines. The public group felt the booklet was very black and white and would prefer the addition of colour.

| Images of Back Anatomy
Thirteen of the 15 volunteers chose coloured classic cartoons (detailed cartoon): • Three out of five volunteers in the public group chose coloured classic cartoon (detailed cartoon) (see Figure 2). One participant did not choose a picture, and one participant chose coloured anatomically correct anime (see Figure 1).
• Five out of five chiropractic practitioners, as well as five out of five student interns chose coloured classic cartoon (detailed cartoon) (see Figure 2). Although volunteers were not asked for further feedback, one volunteer commented that the coloured anatomically correct illustration was quite scary to a lay person and could create more anxiety about their low back pain.

| Posture and Ergonomics
Again, thirteen volunteers chose coloured classic cartoon (detailed cartoon): • Three out of five volunteers in the public group chose coloured classic cartoon (detailed cartoon) (see Figure 4). One participant did not choose a picture, and one participant chose stick figures (see Figure 5).
• Five out of five chiropractic practitioners, as well as five out of five student interns chose coloured classic cartoon (detailed cartoon) (see Figure 4).

| Readability Scores
Readability scores for the trial documents, information sheet, consent form, and home management booklet were calculated. The

Home Management Booklet
Some feedback was given relating to the hot and cold pack section, can more options be listed or signpost participants to their pharmacy for other options. One participant suggested that I inform participants to speak to their pharmacist before taking any medication for their back pain. One participant recommended when pictures are added to the booklet, that they reflect a diverse population.
No comment Two of the five volunteers suggested that an exercise and rehabilitation section be added to the booklet.

Font, font size and layout
One comment relating to colour used within documents, 'it's all so black and white, it makes my eyes sore'. It was suggested that the font was quite small and could be made bigger. One person commented that they liked the font in the Information Sheet, but not in the Home Management Booklet. However, the fonts used were all the same across all documents.
It was suggested that the font spacing be 1.5 spaced to allow the reader to read the document more easily.
No comment TA B L E 2 (Continued) scores before and after the Stakeholder Involvement Process can be seen in Table 3.

| D ISCUSS I ON
The primary aim of this process was to improve the comprehension (through improvements in readability and understanding) of the trial materials for the future participants, as well as to contribute towards the development of a comprehensive information sheet. Both contribute towards informed consent for the future trial participants.
The age range of the public group was 38-73 years of age, which is slightly older than the age range for the future trial of 18-65 years of age. The location of the clinic where the trial will be taking place is ranked 113th in the Index of Multiple Deprivation, meaning it is in the top 1% of the most deprived areas in England. 42 However, the clinic where the trial will be taking place is a private clinic requiring payment for treatment. Equally, this process did not collect socioeconomic status or level of education from the volun-

teers. A chiropractic student Intern is completing a Masters (UK)
and as such is completing a level 7 qualification, equally qualified chiropractic practitioners would have completed a level 7 qualification at the very least. However, the student intern and practitioner groups were included for their clinical expertise, and not their experience of low back pain. The public group contained current clinic patients, which indicates that the public group is representative of the future trial population who will be recruited from the clinic.
Ethnic data was not collected from volunteers, which is a weakness of this stakeholder process. There is a growing need for a wider range of voices to be heard in research and trial development, as such stakeholder processes should consider recruitment of under-represented groups such as Black, Asian and minority ethnic (BAME) populations.
Few studies related to readability and understanding of trial documentation calculate readability scores. 18  words are difficult to understand, for example 'cucumber' is not considered difficult to read or difficult to understand. 45 The readability scores of the information sheet and consent form were higher than the level at which the average United Kingdom person would be able to read comfortably. Even when changes were made to the documentation in response to the Stakeholder Involvement Process, such as sentence length and paragraph length, the readability scores did not decrease. Interestingly, when the data management section was removed from the information sheet, the average readability age of the information sheet before feedback changes were made was 17-year-old, once changes were made in response to the feedback the readability improved greatly with a readability age of 13-14 year. The data management section is predominantly a templated section, which cannot be altered. This was mirrored in the comments from the public group related to the Data Management Section of the information sheet whereby one volunteer stated they did not understand any of it and essentially would like to be reassured that their data are safe. One possible reason for the high reading age in this section may be the use of legal language and jargon related to data management and data protection. Neither the practitioner group, nor the intern group raised comments related to the level of the language in the Data Management section. As the template cannot be altered, this is a limitation of this process. The consent form also largely contains templated wording, which may result in the readability age being so high. The results of this stakeholder process may provide the template authors with incentive to go through a similar process to reduce the reading age of the templates and increase understanding and comprehension.
Readability formulae can only give a reading level, it does not provide feedback on layout (font style, font size, spacing, colour), 45 style of writing (context and appropriateness), 45 difficulty of concept, 46 prior knowledge 46 or coherence of text. 46 Nor does it provide feedback on whether the medical language being used is understandable to the reader. 45 As such, the stakeholder involvement process examined these potential issues.
It was felt by the public group that the information sheet was very long and questioned whether all information was necessary.
This is supported in the literature whereby the volume of information provided to participants may exceed their preference. 15,16 Neither the practitioner group nor the intern group raised comments relating to the length of the information sheet. Conversely, the practitioner group and the intern group suggested additional information could be added to the information sheet.
Use of language (and medical terms) in the information sheet and consent form was a theme of feedback which was common from all five volunteers in the public group. A lack of understanding of medical words may be a barrier to participants understanding the information sheet. 19 The use of medical terms was not identified in the comments from the practitioner group or intern group, which indicates that they do not feel the language is an issue. This highlights the need for public involvement in the development of trial documentation. The additional information that the practitioner group and intern group felt was required would essentially increase the medical language used. A high level of medical language can increase participant fear 47 and potentially decrease recruitment. However, taking all comments on board related to additional information and the use of medical language, changes were made to the section relating to 'incidental examination findings' in a way that is hoped to decrease participant fear and increase understanding.
The comments related to readability and understanding of the in- changes were made to the sequence and flow of the procedure participants will go through during the trial, this will make the future trial process smoother and more logical for the participants. It will also reduce the participant time burden during the trial.
For the home management booklet, the public group were quick to think of other options for home management, particularly related to how to make a hot or cold pack at home. These were incredibly creative and generally were not added to the booklet as it would increase the length of the booklet considerably. Equally, some of the more creative ideas may increase risk of injury. However, the booklet does now suggest that other heat and cold packs can be used. There was a concern that participants may take over the counter pain medication without speaking to their pharmacist. It should be noted that the section relating to medication already urged all participants to consult their pharmacist before starting to take pain medication, as such no further changes were made. The suggestion from the intern group to include rehabilitation exercises was not considered as this may add an additional confounding variable to the trial; however, a link (web address) to the NHS website was provided which does have information on basic stretches for low backpain.
It has been established in the literature that pictures can complement the written word to increase understanding. 29 Volunteers were asked to choose the genre of pictures which best elicits the feeling of reassurance, and that the condition is being taken seri- Regarding fonts, font size and layout feedback was in line with the Dyslexia Guidelines. 27 As a result of the feedback, the font remained as Calibri, a standard Microsoft Word font which is sans serif which makes the font more readable. Italics and underlining were removed, with bold being used for emphasis. Font size was increased from 11 to 12, letter spacing was increased by 20% and line spacing was adjusted to 1.5. The paper used was thicker to ensure that wording on the back of the page cannot be seen through the page.
The altered trial material was not sent back to the volunteers for further feedback. This was largely due to time constraints to this doctoral project, which is not an uncommon issue related to doctoral research. 46 The altered material was however viewed by the research team for further feedback. The altered material was submitted, together with an ethical application for the future trial to a NHS Research Ethics Committee, which included lay members.
Volunteers were not paid for their time due to budget limitations. This is considered poor practice; however, budgetary limitations are not uncommon in doctoral research and can be a limitation in carrying out stakeholder involvement processes. 52 Future studies should consider building in a stakeholder process into the proposal and budget calculations of a study.

| CON CLUS ION
The Stakeholder Involvement Process was an invaluable exercise that aided the development of the trial documentation. Each group of volunteers made a unique contribution to the study design, the readability and understanding of trial documentation, and the development of the home management booklet. This in turn feeds back into the informed consent process contributing towards fully informed consent by participants in the future trial.

CO N FLI C T O F I NTE R E S T S TATE M E NT
The authors declare that they have no conflict of interest.

ACK N OWLED G EM ENTS
Volunteers who agreed in writing to be acknowledged in this article: Deborah Curtis, Daniel Heritage, Barbara Lawson, Victoria Wheeldon and Suzy Wignall. Illustrator: Jessica Fitzpartick.

AUTH O R S' CO NTR I B UTI O N
All authors have made a substantial contribution to conception and design. JR carried out the collaborative process, collation of feedback and alteration of trial material. All authors were involved in drafting and revising the manuscript. All authors have given final approval of the version to be published. All authors agree to be accountable for all aspects of this work.

FU N D I N G I N FO R M ATI O N
This forms part of a PhD match funded by Bournemouth University and AECC University College.

DATA AVA I L A B I L I T Y S TAT E M E N T
Data sharing is not applicable to this article as no new data were created or analysed in this study.