Development and implementation barriers of a new patient‐reported measure: The Radiation therapy‐related Inconvenience Questionnaire (RIQ)

Radiation therapy (RT) can benefit approximately 50% of cancer patients and contribute to 40% of all cancer cures, yet its utilisation in cancer is low globally. Several factors contribute to this including perceived inconvenience related to accessing and utilising RT. To quantitatively assess the latter, a new tool – the Radiation therapy‐related Inconvenience Questionnaire (RIQ) – was developed. This study aimed to pre‐test the RIQ and explore barriers and facilitators to implementing it in routine clinical practice and clinical trials.


Introduction
Cancer is a leading cause of death globally. 1Radiation therapy (RT) is an essential and cost-effective component of cancer treatment, both curatively and palliatively. 2Over 14 million new cases of cancer are diagnosed globally every year; RT could contribute to 40% of all cancer cures by eradicating tumour activity and reducing mortality, 3 and provide palliative relief via symptom control and improvements in health-related quality of life (HRQL) for an additional 25%. 2,4It is therefore recommended that patients receive at least one course of RT as part of their treatment regime 5 ; this benchmark has been the basis for planning Australian RT services.Survival outcomes correlate with the quality and availability of cancer services, 6 so government has expanded RT services for regional and rural cancer patients. 7Despite recognised benefits, proven costeffectiveness and investment in RT service availability, RT remains underutilised. 5Actual RT utilisation rates reported across the world were lower than the optimal utilisation rates. 8,9everal barriers to RT access and utilisation have been identified. 1,7,10,11In regional oncology services, access to cancer care is known to decrease with increasing geographical isolation. 6Less recognised factors include the trade-offs patients make when considering costs and benefits of treatment options. 1,10,12iven that a clinical course of RT often consists of a series of daily treatments given over a period of several weeks, attendance for RT can be inconvenient in various ways.It may disrupt family life, usual work, and other daily activities. 1 Travel to/from the RT facility takes time and may incur direct and indirect costs.For patients who live in regional or rural areas, RT treatment facilities are often situated at considerable distances from patients' homes; a course of RT imposes the additional burden of commuting large distances or even temporarily relocating away from home, with further cost and inconvenience.Patients and their families are likely to weigh up these potential costs and inconveniences against the potential benefits of RT when deciding whether to have RT.As well as influencing patients' choices, the inconvenience of RT may affect their feelings and HRQL, and consequently lead to underutilisation of RT. 1,10,13 Patient-reported measures (PRMs) are the best method to assess patients' experiences and assess outcomes that matter most to them.The ability to quantify and characterise, via a PRM, the inconvenience perceived by a given patient would indicate whether and how that patient could be better supported to access RT.Currently, there is no PRM that can accurately quantify perceived RT-related inconvenience.Our previous work with >1500 patients and health professionals generated a list of RT inconvenience issues. 12These issues formed the basis for a new RTrelated inconvenience PRMthe Radiation therapyrelated Inconvenience Questionnaire (RIQ).This study aimed to pre-test the preliminary versions of the RIQ for acceptability and content validity, and explore barriers and facilitators to implementing RIQ in routine clinical practice and research comparing different fractionation regimes +/À modalities of RT.

Methods
This study was conducted in two phases: (i) pre-testing RIQ in patients recommended RT and (ii) exploring barriers and facilitators to routine use with healthcare professionals.We followed the European Organisation for Research and Treatment of Cancer (EORTC) Quality of Life Group guidelines for developing and testing PRMs. 14or both phases, two preliminary English-language versions of the RIQ were tested The RIQ clinical is intended for use during patients' initial consultation with a radiation oncologist or referring healthcare professional (e.g.medical oncologist) to discuss RT and identify any perceived inconvenience.It has four response options: 'yes', 'no', 'yes and I want help with this', and 'not applicable'.
The RIQ research is intended for use in research to assess patient-reported RT-related inconvenience.It has four response options: 'a lot', 'a little', 'not at all', and 'not applicable'.

Participants and setting
Patients included adults (≥18 years of age), diagnosed with any stage cancer, either considering a course of RT or had completed RT as part of curative management or for symptom management within the preceding six weeks.Patients with cognitive impairment or inability to speak, read, and write in English were excluded.Patients were recruited from two metropolitan and one regional cancer service across NSW, Australia: Crown Princess Mary Cancer Centre, Westmead Hospital; Blacktown Cancer & Haematology Centre, Blacktown Hospital; and Radiation Oncology Department, Orange Base Hospital; and ethics approval was obtained prior to study commencement.

Participant recruitment and consent
Eligible patients identified from March 2021 through December 2021 by staff in participating sites were informed about the study and given a participant information sheet.Staff provided contact details of interested patients to the researcher, who contacted them by email and/or phone, provided further study details, obtained electronic consent, arranged an interview time, and emailed a REDCap 15 link to both versions of the RIQ to be completed before the interview.
We aimed to recruit 10 patients per study site, but due to COVID-19-related service disruptions and restrictions to clinic consultations, only 20 eligible patients were approached across sites.We sampled broadly, recruiting a gender and age mix.Interviews were continued until saturation with no new issues arising after three consecutive interviews.

Data collection
Semi-structured cognitive interviews were conducted by the first author (CS), who held a PhD, was experienced in patient-reported outcome research, and employed as a post-doctoral researcher on the project.They are also a qualified psychologist and conducted interviews via telephone.Participants and the interviewer had no prior relationship.Participants provided demographic information, tumour characteristics, and treatment received, and were probed about RIQ item relevance, difficulty, clarity, and appropriateness (to their condition).Participants were encouraged to vocalise their thoughts as they completed both versions of the RIQ (clinical and research) and provided their responses.The interview focused on four aspects of cognition 16 : (1) comprehension; (2) response options; (3) retrieval from memory; and (4) judgement (e.g.logic decisionswhether patients can retrieve the types of cognitive information required to answer the questions). 17See Appendix S1, for interview guide.Patient interviews were conducted successively, with tentative analysis and expert appraisal between interviews.This allowed revisions to the RIQ versions to be made and tested in subsequent interviews.

Participants and setting
Participants included healthcare professionals with experience in managing patients having RT or referring patients for a course of RT (e.g.radiation oncologist, radiation therapist, oncology nurse, nurse care coordinator, social worker, palliative care specialist, and haematologist).

Participant recruitment and consent
Eligible healthcare professionals were identified by our site investigators at the participating sites as for the patient component between April 2021 and November 2021 and emailed invitations with study information.Figure 1 shows flow diagram of participant recruitment.Contact details of interested healthcare professionals were shared with the researcher, who then contacted them by email and explained the study details.Those who agreed to participate were emailed a REDCap 15 link to the consent form and both versions of the RIQ, to be completed before the interview.Participants were recruited using snowballing method, 18 recruiting a mix of different professions.Interviews were continued until data saturation was reached.

Data collection
Following an interview guide (Appendix S2), interviewers sought feedback on RIQ item relevance, appropriateness, and clinical utility, and barriers/facilitators to implementing it routinely in clinical practice and research.The funnel approach was used to conduct these interviews: Participants were first asked about their demographics, primary clinical discipline, healthcare setting, and clinical specialty to build rapport, with the focus narrowing progressively to the study subject matter.

Expert appraisal
Findings from both phases were discussed by the research team (including two PRM methodologists, a radiation oncologist, and an oncology nurse) and decisions made by consensus whether to revise, add, or remove any RIQ items.This was done on an item-byitem basis, with weight given to the same comments by several participants and consideration of clinical utility.The amended versions were pilot tested with two patients to ensure modifications made did not compromise item relevance, difficulty, clarity, and appropriateness.

Data analysis
Recordings of patient interviews were reviewed by the researcher to identify dominant trends, ensure accuracy of wordings as used by patients and to reduce any bias.Dominant trends across interviews (problems that occurred repeatedly) and key findings (problems identified in a single interview but considered problematic) were considered by the research team.Audiotapes were referred to if any additional information or clarification was required.Healthcare professional interviews were transcribed verbatim and analysed using thematic analysis. 19Identified concepts were grouped into broader themes and sub-themes and discussed with the research team.

Pre-testing
The demographic and clinical characteristics of the 15 patients who completed interviews are given in Table 1.Ten patients were from metropolitan and five from regional areas.
All patients self-completed both versions of RIQ and did not require any assistance from the researcher.No items were omitted during completion.Average completion time was 7.85 min for the clinical version and 8.01 min for the research version.
Cognitive interviews identified problems in content, instructions, layout, length, and response options.Results are presented for the clinical and research versions of the RIQ separately below.

RIQ clinical
The instruction section of the RIQ provides a general introduction to the measure, explaining its purpose and instructions on how to complete it.Patients reported that the instructions were easy to understand and  'straightforward'.However, during team discussion, it was agreed that the first two sentences had to be reworded to provide a context for the measure (Table 2).Content: Twenty-five items considered unclear by patients were revised, eight considered irrelevant by >50% patients were removed, and four considered redundant, merged.One item 'To get assistance with logistics of treatment?' was considered complex and consequently split into three items.As patients reported that all important aspects of RT inconvenience were covered, no new items were added.Table 2 presents preliminary version items and summary of item changes.
Layout and length: All patients liked the layout/design and found it easy to complete.One patient suggested that questions about side effects and follow-up could be administered separately as 38 questions were too many to answer in one go.Hence, the clinical version was reduced by removing the last two sections and merging similar or redundant items as described above.
Response options: Some patients (15%) had difficulty selecting a response option in the clinical version.Consequently, response options were revised to 'no', 'yes', 'yes, I want help with this', 'yes, but I don't want help with this now', and 'not applicable'.Patients also reported that the numerical values beside response options were confusing and a visual distraction.These were removed and checkboxes added instead.No changes were made to response options in the research version as all patients reported they were easy to choose from and appropriate for research purposes.
Pilot testing: The modifications made to the RIQ clinical version were piloted with two additional patients.Patients reported that all items in this version were understandable, clear, and relevant and that response options were appropriate and easy to select an option.Patients reported that the response options with added checkboxes were easy to answer and instructions were clear.Hence, the final 29 items, response options and instructions were retained with no further revisions or changes made.

RIQ research
Instructions: The first two sentences in the research version were revised (Table 3).
Content: It was retained as 92% of respondents considered items to be relevant.However, items were reworded to match the clinical version wording.Four items considered redundant were removed (Table 3).
Layout and length: Patients reported layout and length were easy to use.No changes were made.
Response options: Patients reported response options were easy to choose from.No changes were made.
Pilot testing: Patients reported that all revised items were understandable, clear, and relevant and that instructions clearly explained the purpose of the measure.Therefore, all 36 items, instructions, and response options were retained (Table 3).

Overall impressions about the RIQ
Most health professionals described the RIQ as appropriate, comprehensive, relevant, and easy to use (Quotes in Table 5).
Few described the RIQ as complex and ambiguous, possibly difficult for people less competent in English to understand, and heavily focused on waiting and logistics rather than side effects (Quote in Table 5).
The majority of participants (n = 14/16, 87%) reported the term 'inconvenience' was appropriate and easy to understand for patients undergoing RT.However, few felt 'inconvenience' was negative and inappropriate and suggested 'was it a problem?' and 'how easy was it?'instead.
Barriers for implementing the RIQ.Participants noted barriers specific to staff, patients, and the RIQ versions.These are presented as separate themes below, acknowledging that barriers overlap and impact upon each other.
a Staff-specific barriers Attitude towards use of RIQ in practice: Respondents felt that staff involved in implementing RIQ may not understand the benefits of it, such as informing RTrelated decision-making at the RT referral stage, and consequently may not use it during patient consultations (Quote in Table 5).
Time constraints: Participants believed that staff working in current health systems were overburdened and lacked time for administering questionnaires (Quote in Table 5).
b Patient-specific barriers Patient characteristics: Participants reported that: (1) patients who did not want to be approached to discuss their psychosocial concerns, (2) those who refused to complete questionnaires or any patient-reported outcome measures, and/or (3) patients with lower literacy levels were perceived barriers to implementing the RIQ.They also felt some might find the RIQ difficult to complete without assistance.
A few respondents raised language and cultural barriers as inhibiting factors, given that the content of the RIQ was health related.Some felt that Australian Indigenous people and culturally and linguistically diverse populations might find the RIQ challenging to complete (Quote in Table 5).
Participants believed that patients' health and physical condition on the day of their consultation such as feeling unwell, distressed, or experiencing side effects such as pain or fatigue would affect completion of the RIQ.Time constraints: A barrier for using the RIQ research version was limited time for completing questionnaires during RT (Quote in Table 5).Workplace factors such as support from management and mechanisms for addressing identified patient concerns might enhance health professionals' adoption of the RIQ in their clinical practice.Medical institutions that encourage long-term planning and prioritise quality patient-centred care serve as facilitators for implementing PRMs such as RIQ in practice (Quote in Table 5).

Facilitators for implementing the RIQ.
One respondent thought using posters, flyers, or brochures in the radiation oncology department (such as waiting rooms) and educating staff during staff meetings might raise awareness and use of the RIQ.They also indicated that patients would likely benefit from better information about hospital wait times, parking, and local accommodation.
b Mode of administration Participants suggested sending hard-copy PRMs home with patients to complete and follow-up by telephone might increase completion rates.Some participants felt implementation would be easier if RIQ was available in both paper and electronic forms, and patients were given a choice to complete either based on their preference.Having someone available to help clarify questions could facilitate completion (Quote in Table 5).
Several respondents felt the addition of an open-ended question seeking information about any additional concerns patients had (at referral or initial consult stage) would facilitate use of the clinical RIQ.
c Imparting knowledge Some health professionals suggested staff training that included explaining the purpose and benefits of the RIQ might encourage use (Quote in Table 5).
Finally, respondents felt that patients might be enthusiastic about completing the RIQ research version if they understood how it would help future patients (e.g.improve our understanding of patient inconveniences and treatment-related toxicities).

Discussion
This study pre-tested two versions of a newly developed RIQ (clinical and research versions), to evaluate content validity and acceptability with individuals undertaking RT.We also interviewed healthcare professionals who cared for individuals making decisions about having RT to explore barriers and facilitators in adopting RIQ in clinical practice.We found that both versions of the RIQ were acceptable to patients and clinicians and suitable for use in a variety of clinical settings.The RIQ clinical can be used to identify patient-perceived inconveniences of RT, either during their initial consultation with a radiation oncologist or with their referring medical professional.Identifying inconvenience-related issues that lead patients to decline RT may allow for interventions to   Removed -address these issues, thereby improving RT utilisation.Use of the RIQ clinical may also enable monitoring of patient needs, facilitate optimal patient/healthcare provider communication, and assist patients with being actively involved in decisions about their treatment. 20][22] The research version of RIQ captures patient-reported RT-related inconvenience during or after a course of RT.It is intended for use in comparative research, administered post-RT treatment to assess how convenient or inconvenient the course of RT was for patients compared to other treatments evaluated in the study.Inclusion of the RIQ research in future research will enable assessment of differences between groups on RT treatment-related inconvenience and assess the outcome of any intervention aimed at reducing inconvenience.The RIQ research now needs to be psychometrically evaluated in a large field study to complete its development as a fit-for-purpose measure for clinical studies. 23onsistent with other studies, 21,24 we found patients' value being asked about their treatment-related concerns and having their healthcare needs met.Minimising patients' concerns about RT treatment during their initial consultation, including addressing non-physical aspects such as referral, wait times, and logistics with receiving RT, may prevent patients from declining RT due to perceived inconvenience.Our clinician interviews identified barriers and facilitators to adoption of the RIQ in clinical practice.These included the following: staff-specific barriers (attitude towards use of RIQ in clinical practice and time constraints), patient-specific barriers (patient characteristics and time constraints), workplace facilitators, mode of PRM administration, and imparting knowledge.Clinicians also discussed how low literacy levels and cultural and language barriers may affect PRM usage in clinical practice.This is likely a reflection of the increasing number of culturally and linguistically diverse patient populations and Indigenous Australians served by the healthcare providers who participated in this study.5][26][27][28] To address this barrier, we simplified items in the RIQ, ensuring understanding and suitability for a wide range of patient groups.Future research may develop RIQ versions in other languages, following established linguistic validation and cross-cultural adaptation processes. 29,30espite medical institutions and clinicians reporting positive attitudes towards the inclusion of PRMs in routine clinical care, 20,22 staff involved in implementing these measures may not realise the potential benefits of PRM data to their patients and practice.25,28 These should help staff appreciate the importance and benefits of the RIQ and convey this to patients, feel confident with its administration, and appreciate that its use will ensure consultation time is used optimally.
A strength of this study was inclusion of patients and clinicians with a broad range of experiences with RT.Information gathered will help address barriers and employ facilitators to implementing RIQ in clinical practice.Another strength was conducting patient cognitive interviews successively, with tentative analysis and expert appraisal between interviews.This allowed revisions to the RIQ to be made and tested in subsequent interviews.We adopted a qualitative approach to evaluate face and content validity and acceptability of the RIQ to ensure participants' interpretations of items and suggested improvements were taken into consideration during revisions.A possible limitation of this study is its relatively small sample size.Although it is still debated what constitutes a 'sufficient' sample size for cognitive interviews, the recommended sample size varies from 5 up to 40 participants. 30,31Our initial intention was to recruit 30 participants, but the impact of COVID-19 on the health system slowed recruitment.Similarly, due to the impact of COVID-19 and additional time constraints, we could not seek participants from regional sites to pilot test the modified RIQ version.This should be addressed in subsequent testing.Nevertheless, we reached data saturation and interviews yielded rich data enabling minor modifications and achieving acceptability of the final RIQ versions.Additionally, we did not assess the readability score of the reviewed items or instructions, and this is a potential limitation.Although participants had literacy levels sufficient to provide consent and read participant information sheet, assessing readability is important to enhance understanding of study materials for diverse population.
In conclusions, this study developed and pre-tested the face and content validity and acceptability of clinical and research versions of the RIQ, a questionnaire designed to assess radiation therapy-related inconvenience.The RIQ clinical is now ready for use in clinical settings, mindful of the range of barriers and facilitators to implementing RIQ in routine clinical practice identified in this study.Future work is planned to evaluate the measurement properties of the RIQ research in larger samples.
Facilitators for implementing the RIQ included the following: (a) workplace factors and organisational support; (b) mode of administration; and (c) imparting knowledge.a Workplace facilitators

12 .
Gaining information on support services to help you through your radiation therapy?No change 10.Difficulty getting assistance with logistics of treatment?Split into three items -Getting assistance for travel to and from radiation therapy?Getting assistance finding accommodation close to your radiation therapy treatment centre?Getting appointments that fit in with your daily routine?No change 11.Difficulty choosing a radiation therapy service provider?Removed -Difficulty obtaining additional advice or recommendation (e.g.GP)? Being able to talk about your radiation therapy with other doctor(s)?No change 13.Fear and anxiety about radiation therapy?Experiencing fear and anxiety about radiation therapy?No change 14.Fear and anxiety about relocating away from home for radiation therapy?Experiencing fear and anxiety about relocating away from home?No change Inconvenience of treatment 15.Availability of local radiation therapy services?Finding locally available radiation therapy services?No change 16.Need for pre-radiation therapy appointments and interventions?Attending appointments to prepare for radiation therapy (e.g.radiation therapy planning, putting in a feeding tube, seeing a dentist)?No change 17.Requirement for tattoos?Having permanent marks to enable radiation therapy (e.g.radiation therapy tattoos if needed)?No change 18. Requirement for immobilisation during radiation therapy?Being required to remain still during each radiation therapy session?No change 19.Long wait times for radiation therapy to start?Waiting a long time for a course of radiation therapy to start?No change 20.Long treatment duration (fractionation schedule)?Undergoing a planned treatment course that extended over many weeks?No change 21.Daily travel to attend radiation therapy appointments?Travelling daily to attend radiation therapy appointments?No change 22. Difficulty finding parking close?Finding parking close to the radiation therapy treatment centre?No change 23.Needing to be away from home or work for periods of time?No change -24.Difficulty finding accommodation when relocation was needed?Finding accommodation close to the radiation therapy treatment centre (if needed)?No change 25.Difficulty organising family commitments around radiation therapy times?Organising family commitments around radiation therapy session times (e.g.caring for children or other family members)?No change 26.Disruption to family life?Having family relationships disrupted?No change 27.Impact on family relationships?
: RIQ clinical and RIQ research.Both versions included 38 items grouped into five domains (see Box 1).Box 1. RIQ domains and domain descriptors.

Table 1 .
Demographic and clinical characteristics of interviewed patient participants (N= 15) %, percentage; N, number of participants; SD, standard deviation.†Patients could have had more than one treatment.

Table 2 .
Summary of changes in the clinical version of RIQ at each cognitive interview round based on patient reports and expert appraisal

Table 2 .
(continued) © 2023 The Authors.Journal of Medical Imaging and Radiation Oncology published by John Wiley & Sons Australia, Ltd on behalf of Royal Australian and New Zealand College of Radiologists.

Table 3 .
Summary of changes in the research version of RIQ at each cognitive interview round based on patient reports and expert appraisal

Table 4 .
Health professional characteristics (N = 16) © 2023 The Authors.Journal of Medical Imaging and Radiation Oncology published by John Wiley & Sons Australia, Ltd on behalf of Royal Australian and New Zealand College of Radiologists.

Table 5 .
Illustrative quotes from healthcare professionals' interviews for themes and sub-themes it was really good.There were a lot of issues that were dealt with in the questions that I thought were really pressing. ..' [CA001, Senior Radiation Therapist] 'I thought it was very comprehensive. ..really well thought out' [CB002, Social Worker] 'I actually thought. ..it was very heavily weighted to the waiting for radiotherapy and travelling and very little on the side effects of radiotherapy and the follow-up' [CB012, you'd need to have a lot of buy-in by the staff.So they would need to feel that it's important, and it is going to be productive. ..health staff just have so many things on their plate. ..people are not going to prioritise that' [CB003, Social Worker] Time constraints 'RTs would not be able to do this because they work on a tight schedule' [CB002, Social Worker] 'Staff having to do another survey.They are already doing one too many surveys, checklists with patients' [CA002, Registered Nurse] Patient-specific barriers Patient characteristics 'I think the culturally and linguistically diverse population groups may find it difficult.It may not be particularly helpful even in their country-of-origin or native language.For example, their medical literacy and general literacy may not be so good that they can actually complete this sort of survey; there are lots of difficult words used in this, like stigma, managing, coping sorts of things, which may be quite difficult if you have come from linguistically diverse cultural backgrounds'.[CB012, Radiation Oncologist] Time constraints 'Time can be a big influenceif you are running late or you know it's in a really busy period in the department or they are working and have kids, a lot of patients want to get in [for treatment] and get out'.[CB006, Senior Radiation Therapist] Facilitators Workplace facilitators 'Support from the management or the department, that the results of this questionnaire would result in actions that we could implement for our patients to make things more convenient, or less inconvenient (for them) is important.So, buy-in from the high level of the department management is a facilitator' [CB014, Radiation Oncologist] Mode of administration 'Have someone deliver this and explain it in detail.Also, having it as a part of standard practice would help in compliance' [CB012, Radiation Oncologist] Imparting knowledge 'If the staff that have to use the questionnaire or implement it, have a good idea about the goal of the research and what the outcome would look like, and what's going to be happening with all the answers; if people buy into that, then they'd be motivated to do it' [CB002, Social Worker]