Allison Tong PhD Centre for Kidney Research The Children’s Hospital at Westmead Westmead NSW 2145 Sydney Australia E-mail: AllisonT@health.usyd.edu.au
Background Consumer involvement in guideline development is advocated, but minimal participation, such as a nominated consumer representative on a guideline working group, can inhibit their decision-making power and contribution. Little is known about how to involve consumers more effectively in guideline development.
Objective To describe a targeted approach for involving consumers actively in guideline development, by focusing on topic and outcome selection, and to discuss the impact on content and structure of the final guideline.
Design Descriptive study.
Setting and participants Patients and carers (n = 24) from a tertiary hospital in Sydney attended three structured peer-facilitated workshops to complete group-based exercises on topic and outcome selection for guidelines for early stage chronic kidney disease. These workshops were run in parallel with the guideline-writing group. For each exercise, participants formed small groups and facilitated their own discussion, recorded their responses and presented them to the wider group. The topics and outcomes identified were fed back to the guideline writers.
Results The participants actively engaged in the workshop discussions and articulated topics and outcomes they perceived should be included in clinical guidelines. Four main changes to guideline-related outputs were observed. A new guideline subtopic was introduced, guidelines were consumer-endorsed, guideline recommendations and suggestions for clinical care were augmented with consumer-focused issues, and plain English guidelines were developed.
Conclusions Consumer workshops in parallel and feeding into guideline development can be a feasible and effective approach for active consumer contribution. This process can inform the development of both consumer-focused guidelines for clinicians and specific versions for consumers.
Consumer involvement in the development of clinical practice guidelines is universally advocated.1–3However, there is little evidence on which methods are effective for actively involving consumers in this process.4 Current approaches for involving consumers appear tokenistic and leave consumers with a fringe or passive role. They have limited decision-making power and capacity to make meaningful contribution to guideline development.5 For instance, a nominated consumer representative on a guideline development group, the most common form of consumer contribution, can experience difficulty in understanding the technical language and is able to make only a modest contribution.6 Further, there is little or no evidence to suggest their contribution has any impact on the guideline process or end product.
There are a few studies that describe effective processes for involving consumers2. One study evaluated four methods of consumer involvement in developing asthma and angina guidelines and found that each had significant limitations.6 They described four different methods of consumer involvement in the north of England evidence-based guideline programme and found that patient workshops, albeit resource intensive, allowed explanation of the technical aspects of guideline development to the patients, and patients could engage with the process and make relevant suggestions.
The Caring for Australians with Renal Impairment (CARI) guidelines are written by health-care providers to improve the quality of care and outcomes for patients with chronic kidney disease (CKD). Traditionally, consumer involvement in CARI was limited to the inclusion of a patient representative in the guideline working group and one patient who would be asked to comment on the draft subtopics comprising a guideline prior to publication. From our observations and consumer feedback, this approach was found to be minimally effective at best. A process that facilitated more active and effective involvement of consumers was needed.
In 2009, a multidisciplinary working group of health-care providers was established by CARI to develop a guideline on early stage CKD. This study aims to describe a targeted approach for involving consumers more actively in clinical practice guideline development, by focusing on topic and outcome selection, and to discuss the impact their input had on content and structure of the final guideline.
To elicit topics and outcomes important to the participants, we developed an approach that incorporated group interaction and brainstorming strategies to stimulate ideas, memories and experiences among the participants. The group discussion was facilitated by a researcher (AT) to ensure that the ideas generated during the discussion were relevant to the aims of the project. Also, we chose peer-level interaction to allow participants to express themselves freely in their own vernacular language. AT drafted an initial protocol that incorporated these elements. The protocol was sent to the research team and guideline writers for feedback.
The draft protocol was also sent to a consumer for feedback. The consumer was a carer of a dialysis patient and a volunteer in the research department of the primary investigator. AT discussed the research protocol with the consumer who gave verbal feedback on the flow and clarity of the questions, which was incorporated in the question schedule. However, they did not participate in the study.
To avoid feelings of disempowerment and to encourage open and uninhibited conversation among patients and carers, clinicians did not attend the workshops. However, two points of feedback were provided to the guideline working group. The overall process is depicted in Fig. 1.
Participants and recruitment
Two separate consumer advisory panels were convened: One included late stage CKD patients and their carers (pre-dialysis CKD Stage 3–5, dialysis and transplant recipients), and the other included early stage CKD patients (CKD Stage 1–3) and their carers. The stages of CKD are defined as follows: Stage 1, kidney damage with normal or relatively high glomerular filtration rate (GFR) (≥90 ml/min/1.73 m2); Stage 2, mild reduction in GFR (60–89 ml/min/1.73 m2); Stage 3, moderate reduction in GFR (30–59 ml/min/1.73 m2); Stage 4, severe reduction in GFR (15–29 ml/min/1.73 m2); and Stage 5, established kidney failure (GFR <15 ml/ min/1.73 m2) requiring renal replacement therapy.
The patients were given the option of bringing a family caregiver, as often patients depend on family caregivers for support and management of their health-related needs. For each consumer panel, up to 20 patients and their carers were invited to attend two structured peer-facilitated workshops scheduled 1 week apart in July–August 2009. The participants were purposively sampled from local populations to reflect a range of demographic (age, gender and cultural background) and clinical (diagnosis, co-morbidities and mode of renal replacement therapy) characteristics. We ceased recruitment once 15 participants confirmed their attendance for each session. All participants were recruited from Westmead Hospital in Sydney, Australia, and were offered $50.00 (AUD) reimbursement to cover parking and travel costs. Ethics approval was obtained from the Sydney West Area Health Service.
Each 3-h session was catered and convened in a comfortable, accessible venue external to the hospital.
In both sessions, the participants were asked to complete a series of 2–3 exercises. The detailed questions and exercises for each session are provided in Appendix A. To build rapport among the participants and to encourage open discussion, the participants were asked to introduce the person they were seated next to, to the rest of the group (for example, their name, a word to describe how they were feeling today, something nice that had happened to them in the past month, and the first word that comes to mind when ‘chronic kidney disease’ is mentioned).
In Session 1, AT introduced the CARI guidelines and explained the role and development of clinical practice guidelines. Two exercises were completed; the first focused on patient and carer perspectives on living with early stage CKD, and the second on the benefits and harms of tests, treatments and interventions.
In Session 2, the participants completed three exercises on the following: (i) topics or questions they thought should be included in guidelines on early stage CKD; (ii) outcomes they perceived as important when evaluating how good or bad a test or treatment is; and (iii) what they think health-care providers need to understand about patient and carer perspectives in early stage CKD. After Session 2, the participants completed an evaluation survey (Appendix B).
Structured, peer-facilitated workshops
For each exercise, the participants formed smaller groups of 3–5. The questions, provided in Appendix A, were projected on a screen to guide small group discussion. The participants facilitated their own discussion, and one person from each group recorded their responses on a flip chart. After 10–15 min, the participants reconvened and fed back their responses to the rest of the participants. During their feedback, AT probed with further questions as necessary to seek clarification and to elicit reasons underpinning their responses on the flipchart. The other groups were encouraged to question or comment on the statements from other groups. The discussion was audio-recorded and transcribed verbatim. Photographs and video clips were taken to record the flip chart and to capture group interactions. Two observers (MH, PL) recorded field notes on the contextual details and participant interaction.
The computer software ‘HyperRESEARCH’ (ResearchWare Inc., Randolph, MA, USA 1988–2009 Version 2.8.3) was used to assist with the storage, coding and searching of the transcripts, flip charts, photographs and field notes. We identified topics and outcomes that patients thought were important to include in guidelines for early stage CKD and the reasons for their choices.
The topics and outcomes were presented by the consumer workshop facilitator (AT) to the guideline working group during a face to face meeting. The guideline writers were given a copy of the topics, outcomes and the issues raised by the participants, then asked to incorporate the participants’ perspectives in their guideline. After the guideline writers completed the draft for their subtopic, these were re-written in plain language by AT, MH, PLV and sent to the participants for review and discussion at a third workshop. Participants were asked to provide formal ‘sign-off’ if they felt the draft guidelines encompassed the topics and outcomes they thought should be included. The research team discussed the advantages and limitations of the structured peer-facilitated workshops approach and evaluated the impact of the process on guideline development.
Twenty-three of the 36 invited participants (63.9%) attended the workshops. Reasons for not participating included reluctance to attend the workshop, work commitments, conflicting clinic appointments, and feeling too unwell to participate. The participant characteristics are provided in Table 1. Participants ranged in age from 24 to 66 years. Fifteen were patients diagnosed with CKD and eight were family carers.
Table 1. Workshop participant characteristics
*Two patients brought their carers to Session 2.
Mean age in years (range)
Early chronic kidney disease (Stage 1–3)
Guideline topics and outcomes
The participants actively engaged in discussion during the workshops. Seven main topics were identified, which included patient education, monitoring of CKD, nutrition and exercise, managing fatigue, medication side-effects and interactions, emotional and financial support for patients and carers, and health-care services. A description and illustrative quotations for each of the following topics are provided in Table 2.
Table 2. Topics for guidelines on early stage chronic kidney disease (CKD) guidelines as suggested by patients and carers
Education and support resources for patients, carers and the family
‘It’s like when you’re diagnosed with diabetes you get sent to a diabetes educator and dietician. When I was diagnosed with kidney disease, I wasn’t sent to a kidney educator or dietician’. (Man, 60s)
Information should be communicated in plain language through various channels
‘A lot of these things are written in jargon, in medical speak that we don’t understand, so the information can be more user friendly so that people actually understand what they’re reading’. (Man, 40s)
Description of symptoms, causes and spectrum of CKD, medication side-effects, and the impact of CKD on lifestyle and relationships
‘Appointments and managing your symptoms, it can be difficult to manage, how is it going to impact our lives?’ (Woman, 20s)
Specific educational resources for employers and colleagues to promote flexibility, understanding and support in the workplace
‘I’m starting to experience flack from workmates about having time off’. (Woman, 20s)
Monitoring of CKD
Frequency and duration of tests for monitoring CKD, follow up tests were time-consuming and inconvenient as they were required to attend clinic appointments
‘Every time I have to go to my appointment, I have to take time off work, all the waiting, walking to the hospital from the train station, all for 5 min’. (Woman, 20s)
Description of blood tests and urine tests and what each test involves
Nutrition and exercise
Detailed guidance on nutrition, for example addressing access to recipes that fulfilled the dietary considerations of patients with early stage CKD and other co-morbidities such as diabetes mellitus
‘They give you a pamphlet that says, ‘don’t eat this, don’t eat this, and don’t eat this, and you can have a little bit of this…we need recipes’. (Man, 50s)
Vitamins and supplements such as fish oil
‘Should we take food supplements, vitamins, protein powders, sport supplement if we’ve got CKD?’ (Man, 20s)
‘We hear contradictory stuff about protein, whether a high protein diet will actually affect the function of your kidney or they say sometimes we eat too much protein’. (Man, 20s)
‘The question is, how much do you drink?’ (Man, 60s)
‘We had a query about alcohol, how much you can drink?’ (Man, 20s)
Types and duration of physical exercise
‘How much exercise and what types, what sort of exercise should you do, whether it would be cardiovascular exercise, weight training’. (Man, 20s)
Fatigue and ways to increase energy
‘‘Tiredness is a big issue with everybody. What actually is the reason we have low energy levels and is there anything you can do about it?’ (Man, 60s)
Medication side-effects and interaction
‘For the high cholesterol [medicine] I’m taking, I didn’t know I wasn’t allowed to take grapefruit juice. I didn’t realize how serious it can be’. (Woman, 40s)
Interactions with other medications and food
‘You need to know the effects of other medications that you’re taking at the time as well, if you took Sudafed, it did affect other medications that you’re taking at the time like the blood pressure tablets’. (Man, 50s)
Emotional and financial support for patients and carers
Depression, overwhelming anxiety, stress and financial difficulties
‘Depression and fatigue, how can you deal with it?’ (Man, 20s)
Emotional and financial support
‘A qualified counsellor as support to talk to someone about your issues, and that person knows what you are going through, knows about kidney problems’. (Man, 60s)
Continuity of care
‘It’s continuity of care. A lot of us are changing doctors and you have to explain yourself a few different times. It can get frustrating. So just try to stick to one doctor’. (Woman, 20s)
Consistent communication of results between health-care providers
‘We thought a bad thing was the lack of coordination, there’s not somebody that goes between the disciplines’. (Woman, 20s)
Mechanism that enables personal monitoring of results so patients can take medications and make lifestyle changes to achieve recommended clinical targets
‘A good thing we thought was that you can get to track your results, you know what needs to be adjusted’. (Woman, 20s)
Prevention of disease progression, survival, side-effects and physical impact, clinical values and costs were outcomes nominated by participants for inclusion in guidelines that address treatment or management of early stage CKD. The participants raised four major outcomes of importance for evaluating tests recommended for screening and monitoring of patients with early stage CKD, which included social impact, long-term effects, accuracy and discomfort and pain. A more detailed description and illustrative quotations for each outcome is provided in Table 3.
Table 3. Treatment and test outcomes for early stage chronic kidney disease guidelines as suggested by patients and carers
Prevent or slow progression of CKD
‘The good [outcome] is controlling your kidney function’. (Woman, 20s)
‘I guess the outcome is whether this [treatment] is going to help my life, for me to live’. (Woman, 40s)
Minimize physical symptoms and side-effects
Dizziness and light-headedness
‘I pretty much fainted, [the doctors] said it’s because of the blood pressure tablets’. (Woman, 20s)
‘It’s the feeling of invasive tiredness, it just takes over your mind’ (Woman, 40s)
‘You get paranoid as well, waking up you think, how much weight have I put on. It sounds really self-conscious and it is for me, it is important’. (Woman, 20s)
‘Sometimes people have high cholesterol, it can cause heart problems’. (Man, 50s)
‘You want to know what your blood pressure is. Then you ask what adjustments will I need to make to my diet’. (Man 40s)
Cost to the consumer
‘Medications can become very expensive and that affects your quality of life’. (Woman, 20s)
‘It affects also your mood’. (Woman, 20s)
‘I had to take it [urine sample] to work with me and it’s quite embarrassing having to do that’. (Woman, 20s)
Time (life intrusion)
‘And 24 h urine tests. Is it beneficial? It’s inconvenient and uncomfortable’. (Man, 50s)
‘Life is one continuous appointment’. (Man, 40s)
Effects of dyes and radiation
‘They say not to have too many X-rays. What effect does the radiation have on your body?’ (Woman, 20s)
Accuracy in detecting protein and glomerular filtration rate
‘I’ve gone to have a blood test and felt great. When I’ve gone to the clinic, they say no the blood tests are bad. It depends on the medication you’re on, what time you have the blood tests, what you’ve been doing the day before, your daily exercise and sweat level, it depends on how you’ve been feeling the last 24, 48 h, so the blood test you’re having on that day, is it accurate enough to tell you what your kidney function is?’ (Man, 60s)
Discomfort and pain
Discomfort from undergoing tests such as biopsies and CT scans
‘And the pain involved during the testing procedures, for example, biopsies are uncomfortable and CT scans, where he especially felt uncomfortable with it’. (Woman, 60s, carer)
Structured peer-facilitated workshops: advantages and disadvantages
The structured peer-facilitated workshops enabled active discussion about topics and outcomes that were important to the participants. The workshop format helped to facilitate rapport between the participants; and cohesiveness in diversity was observed in which the participants shared a range of experiences and views but worked cohesively as a group. During the peer interaction, we observed that participants did not feel a sense of being forced to be in the ‘public eye’. This provided participants with a sense of empowerment to contribute and encouraged active involvement from the participants. Also, the peer interaction promoted idea generation as the participants were observed to be building ideas through sharing their experiences, opinions and perspectives, and asking questions of each other. The use of flipcharts promoted a high level of interactive discussion, and the visualization and generation of ideas. On the other hand, structured peer-facilitated workshops were resource intensive. Funding was required to support logistical set-up costs, participant reimbursement, data transcription and participant materials. We also found it was difficult to achieve an adequate attendance rate as some participants were unable to attend at the last minute because of sickness or other commitments. Also, as participants were permitted to interact with each other ‘unfacilitated’ during the small group discussion, some time was lost when participants talked on issues that were not relevant in terms of guideline topics and outcomes. The results of the workshop evaluation are provided in Appendix B.
Impact on guideline development
As a result of implementing this process of consumer involvement in selecting topics and outcomes for a guideline on early stage CKD, four main changes to guideline-related outputs were observed. An additional guideline subtopic was included, consumers were able to endorse guideline topic and outcome selection, draft guideline recommendations and suggestions for clinical care were augmented with consumer-focused issues, and the process prompted the development of a plain English version of the guideline.
Based on the consumer input, the guideline working group decided to include an additional guideline subtopic titled, ‘Symptoms, natural history, and outcomes of chronic kidney disease’ to address the spectrum and progression of CKD and the symptoms patients might experience in the earlier stages of CKD. This was added to the initial list of topics developed by the guideline writers (Appendix C).
The process enabled consumer endorsement of guideline topic selection. Of the 24 participants, 20 (83%) provided formal sign-off. One participant died during the study, and three provided no reasons for not completing the survey and sign off forms. In the survey, the participants indicated that they were satisfied that the guidelines included topics and outcomes they felt were important to include in a clinical guideline for early stage CKD (Appendix D).
It was also apparent that the content of the guidelines were augmented by the views of the participants. Most of the topics and issues raised by the participants were covered by the initial topic selection by guideline writers. (Appendix C) However, the participants elaborated on the content and scope that each guideline topic should cover. For example, the participants felt that the time required to attend clinic appointments impeded on their life commitments and relationships and in consideration of this, the guideline on ‘referral to a kidney specialist’ included a suggestion to ‘consider discussing the management of chronic kidney disease with a kidney specialist by letter, email or telephone in cases where it may not be necessary for the patient to be seen by the specialist’. A comprehensive range of issues were raised particularly relating to the lifestyle and psychological impact of undergoing various tests, self-management such as lifestyle modification to prevent the progression of disease, education and social support for patients and families.
Finally, consumer input on topics and outcomes led to the development of plain English guidelines [not yet published] that addressed, in plain language, recommendations and information on topics and outcomes relevant to consumers living with early stage CKD and their carers.
The CARI guidelines tend to focus on aspects of health-care and medical management that are imminent or pertinent to current clinical practice or policy. Consequently, while the participants made several suggestions on how to improve practice and policy in early stage CKD management, it was noted that some suggestions were beyond the scope of ‘current’ clinical issues and were therefore not addressed in the draft guidelines. For example, the participants suggested that case management systems may be needed to coordinate the multiple health-care services patients and their families are required to navigate. However, given the lack of existing mechanisms and resources to support this in early CKD management, this seemed to be a ‘distant’ concern, relegated to a lower priority and not addressed in the early CKD guidelines.
Consumer workshops conducted in parallel and feeding into guideline development were a feasible and effective approach that facilitated active consumer contribution and enabled consumers to identify topics and outcomes pertinent to a guideline. This process resulted in the addition of a new guideline topic, consumer endorsement of guideline topic and outcome selection, a more comprehensive guideline in terms of content and scope that included issues identified by consumers, and the development of plain English guidelines. The topics and outcomes suggested by consumers can also inform the development of both consumer-focused guidelines, that is guidelines that address topics and outcomes important to consumers, and plain English clinical guidelines.
Our findings demonstrate that consumer involvement in the early stages, such as proposing or selecting guideline topics, can help to ensure guidelines address topics that are relevant and important to consumers. Consumer input on topic and outcome selection is important; a mismatch of views between patients and health-care professionals on research priorities7 suggests that consumers may prioritize different guideline topics to health-care professionals. While guidelines are often made available for ‘public’ review, consumers may either not be actively and directly sought to review the guideline, or they may be asked to review guidelines that potentially fail to address topics that are important or meaningful to them. Involving consumers only at the review stage can restrict the extent of their contribution.
Similar to the findings by van Wersch and Eccles,6 the workshop-style approach in our study allowed the technical process of guideline development to be explained, facilitated interaction and discussion between participants, and patients could make relevant suggestions and identify aspects in the guideline where it could be revised to make it more accessible to a ‘lay audience’. Our study has added new insight by demonstrating how the contributions of consumers on guideline topic and outcome selection made an impact on the content and scope of the guidelines, and prompted the development of plain English guidelines.
However, our study had a number of limitations. All participants were recruited from a single centre, and non-English speaking people were excluded to avoid cultural and linguistic misinterpretations. Also, the response rate was dependent on the potential participant’s interest, willingness to be involved, abilities, circumstances and other life commitments. For these reasons, the transferability of our findings may be limited. Finally, carers were also invited to participate in the study and we were unable to ascertain whether this would have inhibited or influenced responses from the patient and vice versa.
The Appraisal of Guidelines for Research and Evaluation (AGREE) Collaboration developed an instrument for guideline development, reporting and assessment and one of the criteria focused on consumer involvement, specifically asking whether ‘the patients’ views and preferences have been sought’.8 The underlying principle was that ‘information about patients’ experiences and expectations of health care should inform the development of guidelines’. This included involving patient representatives in guideline groups, and using patient interviews and literature reviews of patient experiences.8 This study expands on these suggested methods to involve consumers in guideline development by providing a detailed description and analysis of a novel method of involving consumers in guideline topic and outcome selection and its impact on the content and scope of the final guidelines.
Organizations responsible for the development and dissemination of guidelines that promote best practice in clinical and public health generally recommend involving consumers in the guideline development process.2,9,10 However, the extent to which consumers are involved in guideline development organizations remains unclear and little guidance exists on how to plan, develop and evaluate consumer involvement in guideline development.11,12 Current methods of consumer involvement in guideline development usually involve minimal or passive participation, limiting consumers’ decision-making power and contribution in guideline development.5 Generally, the extent of consumer involvement in guideline development programmes is having consumer representatives on guideline development working groups. Limiting consumer involvement to representation on committees has been criticized as inadequate and tokenistic.13 In 2007, the Guidelines International Network Patient and Public Involvement Working Group was established to improve patient and public involvement in guideline development.4 This initiative aims to facilitate collaboration and coordination of consumer involvement projects undertaken by various guideline programmes across the world.
To date, there is little evidence on effective ways to involve consumers across the spectrum of guideline development and how to assess its impact on the final guideline. We suggest the development, implementation and evaluation of multiple and novel methods of involving consumers, particularly during the early stages of guideline development. In addition, consideration should be given to the local, social and political environment, availability of resources, and the organizational structures and barriers within guideline programmes, consumer groups and other relevant organizations.
We thank all the participants and carers for their time, efforts and enthusiasm in participating in this study and are very grateful for their valuable contribution.
Conflict of interest
The authors do not have any conflict of interest to declare.
Sources of funding
The Priscilla Kincaid-Smith Foundation provided funding for this study.
1 Guideline introduction and training workshop
• What are clinical practice guidelines?
• What is the role of clinical practice guidelines?
• What do current international clinical practice guidelines on early stage CKD include?
2 Introductory exercise
• What is it like to have early stage CKD, how does it impact your daily living?
• What sorts of things concern you?
• What treatments are you currently having and are there any aspects of treatment that you regard as a benefit/harm?
1 Developing and selecting guideline subtopics (question formulation)
• What topics do you think should be included in guidelines for early stage CKD? E.g. detection, diagnosis, management of blood pressure, lifestyle modification, psychosocial support and patient education
2 Identifying outcomes measures they think are important for each question
• For each question, what outcomes do you think are important? What sorts of things should be measured? E.g. symptoms, quality of life
3 Identifying patient experiences and perspectives potentially relevant to the guideline
• What do you think health-care providers need to know about patients’ experiences and perspectives?