Implementing successful systematic Patient Reported Outcome and Experience Measures (PROMs and PREMs) in robotic oncological surgery — The role of physicians

Patient Reported Outcome and Experience Measures (PROMs and PREMs) play an increasingly important role in monitoring the quality of the oncological pathway. The aim of this study is to describe the case of five hospitals a year after the adoption of PROMs and PREMs for robotic oncological colorectal surgery in Tuscany and to investigate how the clinicians can impact the process of implementation and the efficacy of such measures. We used 14 months of data from the five robotic centers in Tuscany. Above all, the phy-sician's personal motivation to improve the treatment of patients, the teamwork, and the possibility to use data for research purposes proved to be the essential factors for their engagement and the successful implementation of patient reported measures. Physicians play a key role in the adoption of systematic PROMs and PREMs. The higher their level of engagement, the higher the collection success, both in terms of number of patients enrolled and response rates. Moreover, the collection of patient reported measures may become part of physicians' daily practice and may lead to a change in their relationship and communication with patients, as clinicians accept to have their job reviewed and are not afraid to be evaluated by their patients. that leader physicians are engaged along the entire process, from the survey design to the use of data, and that managerial mechanisms are used to activate reputational effects and to improve performance of physicians. Further research is needed to support our findings and to explore other factors related to the role of physicians in routine collection of patient reported measures, also looking at other care pathways than robotic oncological surgery.


Summary
At the same time, there was a rise in the oncological conditions (OCs) that pose a great challenge to health care systems due to their increasing prevalence and expenditure. Indeed, cancer is the second leading cause of death worldwide behind cardiovascular diseases. In 2015, there were 17.5 M cancer cases worldwide, an increase by 33% with respect to 2005, and 8.7 M deaths 4,5 . However, the recent developments in personalised medicine, the continuous progress of cancer research, and the advance of mini-invasive surgical techniques have raised hope for patients and considerably improved cancer survival [6][7][8][9] . At the same time, the increasing cost of cancer treatments leads to an ongoing debate regarding both the equity and the affordability of care from individual and societal perspectives 10,11 .
The continuously growing number of oncological procedures delivered by means of robotic surgical systems and the controversial debate regarding their cost-effectiveness with respect to traditional surgery led to the need of performance measurement systems to evaluate the effects of treatments, including surgery, that are able to go beyond the traditional clinical view of mortality and complications rates [12][13][14][15] . Beside the objective outcome measurements evaluated by clinicians, there are also subjective measures such as patient satisfaction, quality of life, and patient experience.
PROMs have received increased attention as a useful tool to evaluate the effects of treatments and the outcomes of patients who underwent robotic oncological surgery, as the importance of the patient's perspective on disease impact has been increasingly recognised 16 . PROMs data have been initially introduced in clinical trials in order to integrate the clinical data with the people's subjective perception of their health status and health-related quality of life 17 . Most studies present the results of clinical trials and focus on radical prostatectomy and use PROMs as a comparative tool between the outcomes achieved by cohort of patients operated by the open, laparoscopic or robotic technique 18-21 , while in literature the evidence of systematic collection of patient reported measures is still limited. However, the intrinsic characteristic of oncological diseases where treatment is not a one-shot event but a pathway that moves across the different care settings requires the design of performance measurement tools that are able to overcome the "silo-vision" and evaluate performance results of the entire care pathway 22,23 . In this direction, PROMs for oncological patients should cease to be limited to clinical trials, and their collection should be systematic and should investigate outcomes and quality of life throughout the entire pathway from diagnosis, to surgery, therapy, and follow-up phase. One of the few examples of initiatives to foster a systematic approach towards patient reported measures is the International Consortium for Health Outcomes Measurement (ICHOM) that has defined a standard set of patient reported indicators for 28 conditions, including five OCs: colorectal, breast, lung, localised, and advanced prostate cancer.
Beside their use in clinical trials, PROMs have been promoted as a means of enhancing patient involvement in the decision-making process, improving communication between health professionals and patients, and improving both detection and monitoring of symptoms 17,24,25 .
A different use for PROMs is the provision of evidence of the performance and quality of health care services 26 , without a feedback system at the level of the individual patient. The best known example of routine collection of PROMs data at a system level is the case of the English National Health System (NHS), which in 2009 has started the collection of data before and after surgery for four elective procedures: hip replacement, knee replacement, varicose vein, and groin hernia [27][28][29] . Despite the fact that since September 2017 the data collection has been ceased for varicose vein and groin hernia surgery, the experience of the English NHS led health care systems worldwide to express growing interest in implementing systematic collection of PROMs.
The English NHS experience has proven that the routine collection of PROMs is in certain circumstances possible. Although the main aim has been to evaluate health care performance and punishments have been put in place for low performers, it is important to mention that surgeons were not involved in the collection process but were provided with individual level outcome data in order to foster comparisons and reduce variation. Still, literature reports no evidence that the availability of data had any impact on clinicians' performance. In this context, understanding which are the main barriers or facilitators that health professionals face in the implementation and use of patient reported measures becomes extremely relevant for the success of such programmes.
International literature lists several problems and barriers for the systematic collection of PROMs, ranging from the lack of time, lack of training, doubts about the tool's validity and reliability, the complexity and the interpretation of measures, language translation, cultural adaptation, cognitive impairment of patients and difficulties of analysing data and dealing with missing values, lack of structure and staff to assist data collection, time consuming, and irrelevance for patients [29][30][31][32][33] .
Focusing on OCs, both health professionals 34 and patients [35][36][37] tend to respond positively to the idea of routine collection of PROMs. Previous studies have found positive relationships between the collection of PROMs and patient satisfaction 38 , perceived quality of care 39,40 , patient outcomes 41 , symptom management 42 , and acceptability 35,42 . On the physicians' side, the collection of PROMs proved to have a positive impact on patient-clinician communication 38,43,44 , contribute to a better detection of symptoms and improve their monitoring 25,37,45 , may support clinical decision making 40,46,47 , while studies did not find a significant impact of PROMs implementation on the length of the clinical encounter [39][40][41]48 .
The mentioned above positive benefits could depend, at least partly, on the physicians' engagement in the collection process. Given the trust that patients place in them, we aim to investigate what is the role they play in the success of the routine adoption of patient reported measures, as they can influence patient participation and response rates. Therefore, shedding light on barriers and facilitators could lead to improved benefits from patient reported measures collection.
Over time, multiple PROMs tools have been developed for people who suffer from OCs. They can be divided into two groups, generic and disease-specific PROMs. Generic ones are broader and more suitable to compare results across different OCs, but they would not be able to capture the specific needs of patients; in case the aim is to take into account the context and needs of a specific population, a disease-specific PROMs should be used 49 .  54 . Also, reporting and consequent use of patient data suffered from the weaknesses of the administration method 55 . This pushed the regional administration to seek an alternative and sustainable collection method for patient reported measures. The MeS Lab proposed a digital and continuous collection of such measures. All eligible patients are informed and enrolled directly by health care professionals on a web platform and receive the questionnaires by email and/or SMS. The digital approach allows real-time reporting back to clinicians, managers, and policy-makers the patient data, in order to inform decision making at the various levels.
In order to obtain a comprehensive evaluation of the performance of health care services, to be integrated along the traditional performance measure indicators, the regional administration has decided to start the collection of patient reported measures, jointly outcomes and experience. Given all the reasons mentioned above, the choice was to implement a collection that is routinely and done by digital means. The MES lab of Scuola Superiore Sant'Anna was in charge of the design phase, that started in September 2016 for hip and knee replacement, breast cancer, chronic heart failure, and three robotic oncological surgery pathways: prostate, lung, and colorectal cancer. The robotic pathways were chosen due to the growing prevalence of these diseases, the high number of robotic surgical systems available in Tuscany, and the ongoing controversial debate regarding the cost-effectiveness of robotic surgery with respect to laparoscopic or traditional surgery.
The data collection methodology, the PROMs questionnaires, and the experience measures have been chosen through a shared discussion with clinicians, managers, and policy-makers. PROMs questionnaires have been chosen based on whether they had been internationally validated, translated in Italian and, when available, shorter forms were preferred to reduce the burden for patients. Once the design phase was concluded, the project was officially kicked-off on 1 January 2018, and 2181 patients were enrolled during the first year while 900 have replied to the first questionnaire (41.3%). The regional administration has set targets for LHAs, both for the percentage of patients enrolled (80% of the total eligible) and for the response rates (40% for the pre-operative questionnaire).

| METHODS
We decided to draw our attention on robotic surgery in order to avoid the heterogeneity among the other pathways and due to the regional coordination that exists for this activity. Moreover, the regional coordinator has been involved from the very beginning in the process and acted as a discussant for the project. This kind of surgery is done in all the five robotic centres in Tuscany: the three THs of Pisa (AOU Pisana), Florence (AOU Careggi) and Siena (AOU Senese), the Misericordia hospital in Grosseto, and the San Donato hospital in Arezzo.
In order to test our hypotheses on comparable data, we chose to focus our attention on robotic surgery for patients who suffer from colorectal cancer. Patients are eligible if they are ≥18 years old and undergo one of the following ICD9-CM procedures: • 4573-right hemicolectomy • workshops were organized every 6 months with physicians, in order to discuss data; • in December 2018, preliminary results were presented to the regional governor, to the regional health councillor, and to the regional health general director; • reports aimed at monitoring performance were sent weekly to LHAs and THs management; • overall results for the first year were reported to the regional administration and to the management in a joint meeting on 2 April 2019.
The first step of our study consisted into analysing, for each hospital, the overall figures of: number and percentage of enrolled patients, response rate to the pre-operative questionnaire (T0), response rate for the 1-month questionnaire (T1), and response rate for the 3-months questionnaire (T2). The 6 and 12-month questionnaires were not included due to the low number of responses at this stage.
Then, after 8 months from the project kick-off, during a workshop organised to discuss the preliminary results of their activity, a survey has been administered to the 40 physicians who are in charge of enrolling PROMs patients, including the six who are responsible of the colorectal cancer pathway in four robotic centres. Due to organisational difficulties and managerial turnover, one of the robotic centres did not implement the collection of patient reported measures.
Physicians were asked to fill in the questionnaire at the beginning of the workshop, in order to avoid that preliminary results may influence their responses. The questionnaire aimed to investigate multiple dimensions of the main barriers and incentives to the implementation of the project according to their direct experience in introducing and managing it inside their hospitals. More in detail, drawing on previous works aimed at analysing factors impeding or facilitating the introduction of PROMs and PREMs collection, the following aspects were investigated: • perceived importance of collecting patient-reported measures; • potential uses of these data; • beneficiaries of PROMs and PREMs data collection; • facilitators and barriers in implementing the data collection process; • motivating factors to join the project; • impact of PROMs and PREMs introduction on the administrative and logistic structures; • open section for comments.
Focusing on the motivating factors that convinced them to adhere to the project, clinicians were asked to score, on a 1 to 10 scale, the importance of the following elements: • their personal interest in patient reported outcomes; • the support and enthusiasm of the team; • project novelty in the Italian landscape; • the existence of successful international experiences; • the possibility to benchmark results against their colleagues.
A complete copy (in Italian) of the questionnaire is available in the additional material section.
The third step consisted of in-depth semistructured interviews conducted with clinicians from the robotic colorectal cancer pathway 14 months after the project kick-off, when the overall results of the first year were available.
Besides elaborating on the insights emerged from their previous answers to the survey administered during the first workshop-such as their personal motivation as a key incentive, the interviews aimed at assessing whether the implementation of routine collection of PROMs and PREMs had an impact on their performance, both at an individual and organisational level. The interviews have been conducted and analysed by two researchers who participated in the design and implementation of the system itself (Table 1). where rates range from 0% to 63.6%. Variation has been observed also on the response rates for the pre-operative, 1-month and 3-month questionnaires, as depicted in Table 2 below.

| RESULTS
From the figures shown in Table 2, two groups of clinicians can be distinguished: the high performers (hospital D and the two departments of hospital A) or group 1, and the low performers (the two departments of hospital B and hospital C) or group 2. No conclusions can be drawn for hospital E that did not take part to the project.
The survey administered to clinicians during the workshop provided interesting insights about their main reasons to join the project. Findings show that the most motivating factors for the adhesion to the project, in descending order, are: their personal interest in patient reported outcomes (8.5 out of 10), the possibility to benchmark results against their colleagues (8.3 out of 10), the project novelty in the Italian landscape (7.3 out of 10), the support and enthusiasm of the team (6.3 out of 10), and the existence of successful international experiences (5.7 out of 10).

T A B L E 1 Characteristics of clinicians who joined PROMs data collection system
Results are aligned to what has been found from the questionnaire administered to all physicians, not only robotic colorectal surgeons, involved in the project. All differences are statistically significative. A comparison between the two groups is shown in Figure 1 below.
Moreover, some insights emerge when comparing these scores between the two groups of physicians, the high and the low performers. Differences are displayed in Figure 2 below.
Small differences can be seen in the first two factors, personal interest and benchmark opportunity, where low performers value them more than high performers, while a relevant difference disparity was observed between the T A B L E 2 Enrolment and response rates for the five Tuscan robotic centres in 2018 two groups regarding the support and enthusiasm of the team. Indeed, clinicians who are more active and engaged in the enrolment of patients seem to give less importance to the support they receive from their team as motivating factor, with respect to the group of physicians who are less engaged.
As the limited sample size does not allow any significative statistical analysis, the third and final part of our analysis consisted into in-depth interviews carried out with the six robotic colorectal pathway physicians in order to investigate both the results displayed in Figure 1 and the differences emerged between the two groups.
When asked how much their personal motivation counts for a successful collection of patient reported measures, the two groups provided similar answers but radically different explanations; indeed, group 2 considered their motivation as "fundamental" without any further explanation, while group 1 considered it "very important" as their main interest is to analyse the results one year after surgery and use the data to improve the quality of life of their patients. Moreover, group 1 stated that the project has not introduced any additional burden as the whole team consider it as a collective goal for the whole ward, while physicians from group 2 complained about the additional effort it takes, since robotic surgery wards are generally under-staffed.
Another aspect investigated during the interviews was the impact of PROMs and PREMs collection, both at an individual and at an organisational level. Clinicians from group 1 stated that they are more careful with patients, especially during the pre-surgical visit and, from an organisational viewpoint, they encountered enthusiasm from their ward colleagues who now pay more attention to make sure all patients are enrolled into the project. On the other hand, group 2 stated that nowadays the introduction of patient reported measures collection has not had any organisational impact, not transforming their current routine, and represents an additional burden for them.
Differences were found also in the implementation mechanism and in the barriers perceived; physicians from group 1 stated that patients are explained and enrolled during the Pre-Admission Testing (PAT), generally 14 days before surgery. In addition, they did not perceive any significant barrier to the implementation of the project, as they consider it a matter of good will. Instead, physicians from group 2 mentioned organisational problems regarding the in-hospital pathway as often they get to see the patient the day before surgery or sometimes directly in the operating room, along with under-staffing. They also recognised that this leads to enrolling the vast majority of patients at the moment of hospital admission; therefore, the same day or the day before receiving the surgical intervention. The implications of this latter behaviour will be further discussed in the next section.
Finally, respondents were asked to express their overall opinion about the project, trying to take into account its benefits and impact. Clinicians from group 2 consider the project mainly as an evaluation tool with two possible purposes: to assess robotic surgery from an economic viewpoint and to determine ward surgical performance in order to benchmark results with other colleagues. Furthermore, their opinion is that the main beneficiary of the project is the hospital management as they can use the data to drive improvement. On the other hand, physicians from group 1 considered the project as a quality improvement tool useful for three stakeholders: • the patients, who are both more involved in their care pathway and can benefit from potential clinical practice changes as a result of the findings obtained from their data; • the physicians, who can use data to improve their performance and for research purposes; • the regional health care system, that can reduce fragmentation of care settings and is able to obtain a real pathway evaluation directly from patients.

| DISCUSSION
The rising number of oncological surgical procedures delivered by robotic systems and the ongoing debate over their cost-effectiveness led to an increased interest in the collection of patient reported measures from those who undergo this kind of surgical intervention. Most of the published studies present experiences of PROMs collection in clinical trials especially for patients who receive robotic radical prostatectomy [18][19][20][21] , while no evidence has been found of systematic collection of patient reported measures for robotic oncological surgery.
Given that Tuscany is the Italian region with the highest number of robotic surgical systems per capita (1 unit every 287 000 inhabitants against a national average of 1 unit every 600 000 inhabitants), the ongoing debate around the cost of robotic surgery and the persisting unwarranted variation for surgical services 56 , the regional administration has decided to put in place a routine collection of patient reported measures for patients who suffer from prostate, lung, or colorectal cancer and receive a robotic procedure.
Unlike other key experiences of systematic and continuous collection of PROMs (ie, NHS) 57-59 , the model chosen by Tuscany for the collection of patient reported measures foresees a key role of physicians in informing and enrolling patients, analysing data, and discussing benchmarked results with their colleagues to improve performance.
Consequently, it is important to explore the role of physicians and the factors they consider relevant for the success of routine collection of patient reported measures and to investigate the impact it had both on the clinicians and on the health care organisations.
Results from 14 months of patient reported data for robotic colorectal surgery display a polarised situation in Tuscany: on the one hand, a group of physicians-high performers or group 1-actively take part to the project by enrolling patients and informing them, while the other group-low performers or group 2-did not enrol any or a few, and probably without informing them properly, as low response rates suggest.
Differences between the two groups have been observed throughout the in-depth interviews aimed at exploring the impact that the introduction of systematic patient reported measures collection had on physicians and health care organisations through four dimensions: motivation, overall project evaluation, implementation, and impact. A summary of results is available hereafter in Table 3.
One of the most relevant elements that emerged from the interviews is the role of ward colleagues: indeed, physicians from group 1 clearly stated that their personal motivation is important, but they can rely on their colleagues who are enthusiastic about the project and actively contribute to it. This is aligned with previous studies that have documented the effect of collaboration among physicians on the effectiveness in delivering health services and on improving quality [60][61][62][63][64] . The leadership of involved professionals is a key aspect for the successful collection and consideration of patient data 65 . Physicians from group 2 instead probably had a more individualistic approach and did not want, did not succeed, or did not have sufficient ward staff to involve in informing and enrolling patients. As they attribute a higher importance to their personal motivation and consistently complained about under-staffing, it feels like poor performance is due to the shortage of human resources, although they put a great individual effort that consists into an additional burden. It is worth pointing out that the availability of resources for staff to collect patient data is often stated as one of the main barriers by the health care professionals 65 . The different approaches to collaborative efforts of the staff can explain also the discrepancies of how the project has been implemented: indeed, physicians from group 1 stated that the whole ward is committed into making sure that patients are enrolled during PAT, receive all the necessary information, and are handed an informative flyer to take home.
Another interesting element to discuss that might explain performance differences is the perceived benefit and beneficiaries of the project. On the one hand, group 1 thinks that the collection of patient reported measures can benefit the patients, but also themselves and the health care system, while on the other hand physicians belonging to group 2 mentioned the hospital management as the sole beneficiary. We argue that, to a certain extent, the perceived lack of individual benefit and benefit for the patients might have decreased their motivation to enrol and inform patients, as the international literature reported that physicians' occupational motivators are mainly intrinsic, such as helping others and self-expression [66][67][68] .
A third potential determinant of the success of the collection of routine patient reported measures stands in the overall view, that the two groups have about the project: clinicians from group 2 have a more classical view and see it as an evaluation tool, of both economic and ward performance, while those from group 1 consider it a quality improvement instrument that can benefit multiple stakeholders without any additional burden, and therefore they feel motivated to give their contribution. This aspect fits with the above mentioned perception of the real beneficiaries of the PROMs project itself. Patient data need to be relevant to the specific ward or clinician 65 . In this sense, an early involvement of physicians in designing the project can reduce scepticism about it and can increase participation and perceived relevance of data.
Finally, in regard of the impact of the PROMs project, we observed that for physicians belonging to group 2, it consisted into an additional effort at an individual level and did not have any organisational impact. Further research is needed for investigating the burden and costs of organisational changes needed to make the PROMs collection a routine activity within wards. In this respect, also the management's support for change should be deeper studied more extensively in the future.
In order to overcome the barrier of perceived effort and to foster the routine collection of patient reported measures, health professionals should be properly informed that a little effort from their side could produce deep advantages in terms of care quality, patient satisfaction, and outcomes 69,70 . As anticipated, the early involvement of professionals enhances their participation in the initiative, by sharing goals, identifying what is relevant also in the professionals', point of view and co-designing the process. The choice of whom should be involved earliest is also important: involving recognised leaders, champions, and influencers can help in engaging other colleagues and collaborators. Other managerial mechanisms can be used to improve this viral effect, such as reputation 71 or the inclusion of PROMs data into the performance evaluation systems [72][73][74] .The comparison of performance within a community of professional practice as well as through public disclosure of performance benchmarking can result in improved performance.
Group 1 reported that the introduction of the project led to an increased attention to patients, especially during the presurgical visit. Moreover, it impacted also the organisational level as the ward staff increased their attention  70 . Starting from this last promising finding, future research should investigate if a deeper participation of professionals in the collection of data can improve the use of patient data or enhance their impact on the professionals' day-by-day behaviours towards patients.

| CONCLUSIONS
In the PROMs initiative conducted in Tuscany, physicians are given a central role in the process, as they were initially engaged in the collection design phase, they are in charge of informing and enrolling patients, and they have realtime access to data and periodically meet their colleagues to discuss results.
In routine collections of patient reported measures, physicians play a central role in the success of such projects.
Our research intended to understand what are the determinant factors that can contribute to the successful implementation of systematic collection of patient reported measures, in terms of high percentage of patients enrolled and response rates.
Physicians' personal interest in patient reported measures is a key motivator, but one of the success factors seems to be their ability to engage their colleagues and to make the enrolment of patients a collective effort. Indeed, higher performance can be achieved by early engaging leaders and champions among professionals, as well as using managerial mechanisms for spreading the involvement of other professionals, such as the reputational effect through the community of professional practice 75 that discusses PROMs data and shares good practices, and through public reporting of benchmarking of performance as well 23,71 .
At the same time, the physicians' perception of the final goal and the beneficiaries of the PROMs initiative is fundamental. High performers consider patient reported data as a tool able to produce value for themselves as well as for a wider range of stakeholders: the patients, other physicians, and the health care system as a whole 55,76 . In fact, high performers argue that the PROMs initiative leads to an increased attention towards patients, both from themselves and from the ward staff. In this sense, the professionals' engagement can positively impact not only their performance in PROMs collection, but also the PROMs final outcomes, such as a better communication and decisionmaking with patients.
Therefore, health care organisations that plan to introduce successful systematic collection of patient reported measures should make sure that leader physicians are engaged along the entire process, from the survey design to the use of data, and that managerial mechanisms are used to activate reputational effects and to improve performance of physicians. Further research is needed to support our findings and to explore other factors related to the role of physicians in routine collection of patient reported measures, also looking at other care pathways than robotic oncological surgery.