Complementing clinical cancer registry data with patient reported outcomes: A feasibility study on routine electronic patient‐reported outcome assessment for the Austrian Myelome Registry

Abstract Objectives Routinely assessed patient‐reported outcomes (PROs), such as quality of life (QOL), are important to supplement clinical cancer data but requires rigorous implementation. This study aims at depicting the implementation procedure and evaluating the feasibility of routine electronic PRO monitoring (ePRO) for collecting data supplementing the Austrian Myeloma Registry (AMR). Methods Integration of ePRO monitoring into clinical routine was planned according to the Replicating Effective Programs framework. QOL data were assessed regularly during treatment and aftercare at the hematooncological outpatient unit at the Medical University of Innsbruck with the EORTC QLQ‐C30/ +MY20 and the EQ‐5D‐5L. Feasibility and usability testing were performed via a multimethod approach. Results Within the first year, 94.4% of the MM patients (N = 142, mean age 65.4, SD 11.8, 60% male) provided 748 PRO assessment time points overall. Patients and clinicians were satisfied with ePRO monitoring and indicated no to little disruption in clinical routine. Patient preference on assessment time points and completion frequency became evident. Conclusions Complementing the AMR with ePRO data proved to be feasible. Our findings provide useful insights for healthcare providers considering introducing ePRO monitoring to their units for informing clinical registries as well as individualised feedback to patients alike.


| INTRODUC TI ON
Due to increased availability of once so-called novel agents in firstline treatment (bortezomib and lenalidomide), the prognosis for patients suffering from multiple myeloma (MM) has been considerably improved within the last decade (Kumar et al., 2014;Mey et al., 2016). For patients with advanced disease, likewise, the number of available treatment options, similar in safety and clinical effectiveness, has substantially increased (Anderson, 2016;Orlowski & Lonial, 2016). With regard to the growing complexity of treatment decisions and prolonged survivorship in a still incurable disease following a highly protracted course characterised by a high symptom burden, the value of patient-reported outcomes (PROs), such as quality of life (QOL), supplementing clinical data for the comprehensive haemato-oncologic care and research is increasingly recognised (Efficace, Gaidano, & Lo-Coco, 2017). With the introduction of all oral regimes (e.g., Ixazomib based) and immunotherapy into MM treatment regimes, questions like the respective subjective preference towards alternative treatment approaches and adherence attitudes are of paramount importance.
PROs are defined as 'any report of a patient's health condition coming directly from the patient, without interpretation by a clinician or anyone else' (FDA, 2006). To realise their potential in informing oncological treatment and research, PROs need to be linked to patients' clinical data such as those collected systematically by cancer registries. PROs can contribute information across the spectrum of cancer registry purposes by augmenting the evaluation of adverse events, determining comparative effectiveness of oncological treatment strategies or predicting meaningful clinical outcomes such as survival. PRO data collected prospectively can be used in clinical routine to support decision-making and patient-centred care (Dubois et al., 2006;Efficace et al., 2012;Quinten et al., 2011;Wood et al., 2016). For regulators, PRO data provide crucial information for quality assurance, the allocation of health resources and the development as well as targeted provision of support services (Parkin, 2006).
The traditional role of cancer registries has already expanded beyond the monitoring of disease parameters such as cancer prevalence, mortality and survival to monitoring factors that influence clinical outcomes as well as the treatment process (Parkin, 2006). A number of studies have called for extension of the use of cancer registries to include collection of PROs (Ashley et al., 2013;Santanello, Largent, Myers, & Smalley, 2018;van de Poll-Franse et al., 2011). Up to now, few promising initiatives supplementing cancer registries with PRO data have been reported (Ashley et al., 2013;Kent et al., 2016). While the registration data reporting is an established practice, there are no standards for how to implement PRO assessment procedures in routine cancer registration. Findings, however, offer strong support for the premise that systematic and rigorous implementation is associated with better outcomes regarding the success and sustainability of any intervention (Durlak & DuPre, 2008;Efficace et al., 2017). Therefore, the aim of this study was to depict the implementation procedure and to evaluate the feasibility and usability of routine ePRO monitoring for integrating patient-reported QOL data with data of the Austrian Myeloma Registry (AMR) at a single participating centre. Additionally, differences in attitudes towards routine ePRO assessment between new and more experienced users are evaluated.

| Implementation procedure for integrating regular ePRO monitoring
Implementation of routine ePRO monitoring used a pragmatic strategy on the basis of the Replicating Effective Programs (REP) framework (Kilbourne, Neumann, Pincus, Bauer, & Stall, 2007) leveraging local healthcare providers' initiative and outside ePRO expertise (Wintner et al., 2016). REP focuses on standardisation of the implementation of healthcare interventions into routine care settings and consists of four phases. Its participatory approach involves all relevant stakeholders (detailed in Figure 1).
The first phase, pre-conditions, encompassed the initiative of the AMR executive on supplementing the registry with PRO data. Decisions on the choice of EORTC PRO measures (as suggested e.g., in Huebner et al., 2014) and software were made in consultation with the PRO advisory group in the initial phase. A common understanding, taking into account the requirements of the local leadership as well as retentions of the on-site staff, was established concerning the rationale of ePRO monitoring and possible clinical applicability of the data. Logistics with the least possible disruption of the existing clinical workflow were planned and set-up in discussion with the relevant stakeholders.
In the second phase, pre-implementation, the software Computerbased Health Evaluation System (CHES, © ESD 2018) was customised to fit local requirements (Holzner et al., 2012;e.g., interface to the clinical information system, graphical user interface, adding reference values calculated based on local reference cohorts and thresholds for clinical relevance , Wintner et al., 2016. To minimise logistic burden for the local healthcare professionals (HCPs), a designated staff member (herein, the ePRO facilitator) was introduced to the unit to support PRO data collection with designated time and technical expertise.
During the third phase, implementation, ePRO monitoring was initiated at the outpatient unit. Regular multidisciplinary team meetings were held along the whole implementation process to obtain suggestions for improvement and feedback on technical and logistic issues as well as clinical application of ePRO data of individual patients. The staff received supportive material on the use of CHES and theoretical background on ePRO monitoring.
In the fourth phase, maintenance and evolution, patient and staff surveys were conducted and during team meetings further uses of ePRO data and possible extensions of the monitoring were discussed. Given the scope of this paper, the results of the process evaluation in the final phase will be described in more detail herein.

| Sample
Patients were recruited at the haemato-oncological outpatient unit at the Department of Internal Medicine V (Haematology and Oncology) at the University Hospital of Innsbruck. The outpatient unit is staffed with five haemato-oncologists, one to two changing residents, eleven nurses and two further staff members. Patients were eligible for inclusion to the AMR as well as the ePRO assessment if they were (a) over 18 years of age, (b) diagnosed with multiple myeloma, (c) had no overt cognitive impairment, (d) were German literate and (e) provided written informed consent.

| PRO measures
The modular EORTC Quality of Life system was chosen to assess QOL in MM patients. The EORTC QLQ-C30 (Aaronson et al., 1993) is an internationally validated and widely used cancer-specific QOL measure, comprising five functional scales (physical, role, emotional, social and cognitive), three symptom scales (fatigue, nausea & vomiting and pain), six single items (dyspnoea, insomnia, appetite loss, constipation, diarrhoea and financial difficulties) and a scale for the patient's global health status/QOL. The core questionnaire was supplemented by the EORTC QLQ-MY20 (Stead et al., 1999) addressing issues relevant to myeloma patients. It consists of two symptom scales (disease symptoms, side effects of treatment), one functioning scale (future perspective) and one single item to evaluate body image. The EORTC QOL system was supplemented with the EuroQol-5D-5L (EQ-5D-5L; Brooks, 1996), which is a standardised generic preference-based measure for estimating health state utility values in five dimensions of health including mobility, self-care, usual activities, pain/discomfort, and depression/anxiety.

| Feasibility and usability testing
The 'Fit between Individuals, Task and Technology' (FITT) framework by Ammenwerth and colleagues (Ammenwerth, Iller, & Mahler, 2006) was adopted to guide the feasibility and usability assessment from the patient and provider perspective (Steele Gray et al., 2016). The FITT framework suggests that adopting a new information technology (IT) system in the clinical environment depends on the fit between the attributes of the users (e.g., comfort with technology, motivation), attributes of the technology (e.g., usability), and attributes of the clinical tasks and processes (e.g., task complexity). Herein, feasibility and usability testing comprise the assessment of the following parameters: resources required to complete tasks, for example delay in clinical practice, administrative burden (efficiency), the ability to complete tasks completely and accurately, for example reported errors, required assistance, difficulties while using the tool (effectiveness), learnability (e.g., increase in user skills after multiple assessments) and user satisfaction.
A multimethod approach was applied including the following measures: F I G U R E 1 Replicating Effective Programs phases of implementation of ePRO monitoring into clinical workflow 1. Feasibility survey. To assess the patients' and the HCPs' perspectives, a survey was designed including questions asking for the responders' perspective on the feasibility and usability of the ePRO monitoring tool, aspects of integration into clinical workflow, administrative burden of assessment, need for assistance, reported learnability and attitudes towards use of PRO data in routine care. Twenty patients were consecutively enrolled to complete the survey after their baseline ePRO assessment (hereafter, 'ePRO-naïve group') and 20 patients after multiple (at least 5) ePRO assessment time points (hereafter, the 'ePRO-experienced group'). HCP feedback was collected from all haemato-oncologists and nurses who wished to participate.

On-site performance. Observational notes on CHES performance
and assessment procedure, such as technical or logistic issues, were assessed on-site by the ePRO facilitator.
3. Additional feasibility outcomes included patient participation, response rate, dropouts and adherence with questionnaire completion (incl. reasons for non-completion, percentage of missing items).

| Infrastructure and software for ePRO assessment
The ePRO infrastructure was provided by the software CHES. CHES is a software tool for the electronic administration, collection, calculation and graphical presentation of PROs and medical data (Holzner et al., 2012;© ESD 2017). Healthcare professionals can access the questionnaire results immediately after questionnaire completion on any workstation within the hospital network via the web browser (cf. Figure 2). In addition, CHES provides data exchange interfaces to connect with clinical information systems (e.g., HL7) and cancer registries (e.g., AMR). Data are transferred via a secure interface, linked by an identifier.

| Assessment procedure of ePRO monitoring
The ePRO facilitator consecutively recruited patients for ePRO monitoring during their appointments at the outpatient unit. After informed consent for inclusion to the AMR, they were asked for their consent to participate in clinical ePRO monitoring at their clinical appointment. ePROs were assessed at any given post-diagnostic time point (according to treatment schedule) using tablet PCs running CHES. The ePRO facilitator introduced participants to routine QOL monitoring, the use of the device and provided assistance if necessary. Participants were invited to contribute comments and recommendations concerning the procedure.

| Statistical analyses
Sample characteristics are presented as frequencies, percentages, ranges, means and standard deviations. Mann-Whitney U test and chi-squared test were applied for comparing the ePRO-naïve and the ePRO-experienced feasibility survey groups. Due to one expected cell count less than five, a Fisher's exact test was performed in this case. Differences were considered as statistically significant at a p < .05. All statistical analyses were performed using SPSS 21.0. Internet non-users were significantly older than participants who indicated to be Internet users (mean 72.5, SD 9.2, 95% CI 70.1-75.1 vs. mean 59.1, SD 10.3, 95% CI 56.9-62.1; p < .001).

| Sample characteristics
Patients participating in ePRO monitoring had a mean age of 65.4 years (SD 11.8) with about 60% being male. Mean age of patients deciding against participation in routine monitoring was higher (mean 74.5 years, SD 9.2, range 61-91) compared to patients participating (mean 64.9 years, SD 11.7, range 38-93). The majority of the patients providing consent (68.4%) indicated more than compulsory education. Sixty-two per cent of the patients reported regular Internet usage for private and/ or professional purposes.
Mean time since diagnosis was 4.2 years, up to 26 years. The majority of the patients (68.4%) were undergoing active or maintenance therapy, being treated with chemo-or myeloma specific therapy such as proteasome inhibitors (e.g., bortezomib or carfilzomib), immunemodulatory drugs (lenalidomomid, or pomalidomide), melphalan or bendamustin, or a combination with antibodies like Daratumumab.

| Patients' perspective on feasibility and usability of routine ePRO monitoring
About 87.5% (35/40) of the feasibility subsample reported not having problems with the usage of the tablet for ePRO assessment (85% in the ePRO-naïve group; cf. complete the ePRO questionnaire significantly less often (75% monthly or less during treatment and 50% once to twice a year in aftercare) than patients after multiple assessments (90% weekly to monthly during treatment and 60% monthly in aftercare). When asked which location for questionnaire completed they would prefer if given the option, the majority of patients (24/40, 60%) indicated the clinic, while 32.5% did not report any preference at all.

| Healthcare professionals' perspective on feasibility and usability of ePRO monitoring
Fifteen HCPs (four physicians, 11 nurses) participated in the survey on ePRO feasibility. Eighty per cent of these HCPs indicated being quite to very satisfied with ePRO monitoring, about 67% (10/15) did not experience any delay in clinical routine (cf.

| Observational data on on-site performance
Since initial operation (13th June 2016), logistic and technical issues observed on-site refer to lack of privacy (no designated room) and to matters of surface disinfection when using a touch screen device in the healthcare setting. These were resolved in due course and supplemented with additional interventions fostering routine monitoring. Hence, measures to increase adherence with ePRO monitoring included adapting assessment frequency to patient preference and further customisation of the software to patient demands (i.e., the size of the icons especially for older patients with peripheral neuropathy).

| D ISCUSS I ON
This study aimed at depicting the implementation procedure and evaluating the feasibility of routine monitoring of ePROs for integration with data in the AMR at a single participating clinical centre.
Adherence to routine monitoring was high, while the percentage of missing data proved to be low across all assessment time points.
ePRO monitoring demonstrated feasibility in clinical practice, while undergoing further adaption and improvement based on stakeholder feedback.
A key challenge to successful ePRO monitoring is achieving high levels of sustained patient participation. Participation rates have been far beyond those reported in the literature (Ashley et al., 2013;Hutchings, Neuburger, Grosse Frie, Black, & van der Meulen, 2012;Smith et al., 2016;Thiel et al., 2015). Since patient participation is higher when patients are approached face-to-face (Ashley et al., 2013), the presence of the ePRO facilitator might have been a supporting factor. Even more so for 13% of the patient sample requiring assistance with ePRO assessment on a regular basis. Significantly more Internet non-users seemed in need for support in handling the system than Internet users which might be due to the fact that the former were significantly older than participants who indicated to use the Internet regularly. This difference might resolve in the future with greater penetration of new technologies and generational shift.
Experiencing the treating HCP addressing issues from the ePRO data in real-time strengthens patients' motivation to participate further, resulting in a positive HCP-patient feedback loop. Likewise, it is also important to take into account the patient preference on assessment time points and completion frequency, which differed significantly between the ePRO-naïve and the ePRO-experienced group. The majority of the ePRO-experienced group (80%) was willing to spend significantly less time (15 min

| Lessons learned and action taken based on stakeholder evaluation
Empirical observations from clinical practice were used to both, expand theoretical knowledge about implementation and to take actual steps for adjustment. Based on patient feedback, the rhythm of ePRO monitoring was adjusted not only to treatment schedule but also to patient preference concerning completion frequency.
The importance of receiving patient data and also making it a subject of discussion with the patient was evident for 67% of the HCPs completing the feedback questionnaire. Given the importance of HCPs' feedback for patient adherence and the benefits of routine monitoring for patient care (Basch et al., 2016), it is necessary to raise awareness for the availability of QOL data among healthcare providers. Hence, exchange was fostered at regular meetings and logistics adapted to support sustainability with routine procedures.
On a more technical note, when assessing with touch screen devices, providers need to consider the locally existing guidelines on surface disinfection. Although regular disinfection with alcohol-based wipes is recommended (Albrecht et al., 2013), we found this procedure to be detrimental for some devices. Alternative options would be using hydrogen peroxide wipes, UV-light based approaches and asking patients to apply hand sanitisers before using the tablet.

| Limitations
There are limitations to this study. The ePRO assessment has been limited to one outpatient unit providing data for the AMR.
Considering the high patient turnover in an outpatient facility, the low dropout rates and low percentage of missing questionnaire data give way to the assumption that ePRO monitoring might be feasible in lower frequented wards (e.g., inpatient units) as well. However, there needs to be a person responsible for ePRO assessment, since approx. ten per cent will require assistance on a regular basis according TA B L E 6 (Continued) to literature (Ala-Aldeen et al., 2018). Composition of patient groups for feasibility survey was not balanced regarding their current treatment and consequently, regarding the frequency of their outpatient appointments. In the patient group solely completing baseline assessment, 55% have been undergoing active treatment, while 85% of the ePRO-experienced patients in treatment were seen at the unit each week to month. This distribution is due to the study design based on consecutive recruitment: patients were enrolled during a period of 12 months; therefore, only the patients in active treatment with regular clinical appointments could become experienced in the ePRO monitoring.

| Implications
The results presented herein have encouraged the extension of ePRO monitoring within the institution and on a multi-institutional level. Driven by a local HCP initiative, further patient groups (according to cancer entity) and units have been included. Routine ePRO monitoring has been extended beyond the clinical setting by means of an online patient portal, where patients can access disease-specific information, complete their QOL questionnaire and look at their QOL values that are linked to self-management interventions. Next steps include optimising the logistics by integrating electronic reminders to enhance shared responsibility and buy-in from the local HCPs to foster sustainable implementation into routine care. From a scientific standpoint, ePRO data are an important tool to validate differential treatment approaches from a patients perspective, especially in the chosen model of myeloma were multiple regimens and drugs without a clear oncological ranking or sequencing strategy compete for application.

| Conclusion
Linking PROs with cancer registry data are essential for oncological treatment planning as well as for targeting quality assurance and research issues. ePRO monitoring can overcome the challenges of routinely collecting PROs for supplementing cancer registry data and has proven feasible. Our findings may provide useful insights for HCPs considering introducing ePRO monitoring to their units for informing clinical registries as well as individualised feedback to patients alike.

ACK N OWLED G EM ENTS
We want to acknowledge Tirol Kliniken GmbH for supporting data collection and Jens Lehmann for assistance.