Using patient-reported outcomes to assess and improve prostate cancer brachytherapy


  • James A. Talcott,

    Corresponding author
    1. Massachusetts General Hospital Cancer Center, Boston, MA, USA
    2. Continuum Cancer Centers of New York, New York, NY, USA
    3. Albert Einstein School of Medicine, New York, NY, USA
    4. Harvard Medical School, Boston, MA, USA
    • Correspondence: James A. Talcott, Center for Health Care Quality and Outcomes Research, Continuum Cancer Centers of New York, 325 West 15th Street, New York, NY 10011, USA.


    Search for more papers by this author
  • Judith Manola,

    1. Dana-Farber Cancer Institute, Boston, MA, USA
    Search for more papers by this author
  • Ronald C. Chen,

    1. Department of Radiation Oncology, University of North Carolina at Chapel Hill, Chapel Hill, NC, USA
    Search for more papers by this author
  • Jack A. Clark,

    1. Center for Health Quality, Outcomes, and Economic Research, Edith Nourse Rogers Memorial Veterans Hospital, Bedford, MA, USA
    2. Boston University School of Public Health, Boston, MA, USA
    Search for more papers by this author
  • Irving Kaplan,

    1. Beth Israel-Deaconess Medical Center, Boston, MA, USA
    2. Brigham and Women's Hospital, Boston, MA, USA
    Search for more papers by this author
  • Anthony V. D'Amico,

    1. Brigham and Women's Hospital, Boston, MA, USA
    2. Harvard Medical School, Boston, MA, USA
    Search for more papers by this author
  • Anthony L. Zietman

    1. Department of Radiation Oncology, Massachusetts General Hospital, Boston, MA, USA
    2. Harvard Medical School, Boston, MA, USA
    Search for more papers by this author

  • Research support: USA Army Medical Research and Material Command Grants DAMD17-02-1-0090 and W81XWH-05-1-0093
  • Earlier results presented in abstract form at the Annual Meeting of the American Society of Clinical Oncology, 2007



  • To describe a successful quality improvement process that arose from unexpected differences in control groups' short-term patient-reported outcomes (PROs) within a comparative effectiveness study of a prostate brachytherapy technique intended to reduce urinary morbidity.

Patients and Methods

  • Patients planning prostate brachytherapy at one of three institutions were enrolled in a prospective cohort study.
  • Patients were surveyed using a validated instrument to assess treatment-related toxicity before treatment and at pre-specified intervals.
  • Unexpectedly, urinary PROs were worse in one of two standard brachytherapy technique control populations (US-BT1 and US-BT2). Therefore, we collaboratively reviewed treatment procedures, identified a discrepancy in technique, made a corrective modification, and evaluated the change.


  • The patient groups were demographically and clinically similar.
  • In the first preliminary analysis, US-BT2 patients reported significantly more short-term post-treatment urinary symptoms than US-BT1 patients.
  • The study's treating physicians reviewed the US-BT1 and US-BT2 treatment protocols and found that they differed in whether they used an indwelling urinary catheter.
  • After adopting the US-BT1 approach, short-term urinary morbidity in US-BT2 patients decreased significantly. Brachytherapy procedures were otherwise unchanged.


  • Many procedures in cancer treatments are not evaluated, resulting in practice variation and suboptimal outcomes. Patients, the primary medical consumers, provide little direct input in evaluations of their care.
  • We used PROs, a sensitive and valid measure of treatment-related toxicity, for quality assessment and quality improvement (QA/QI) of prostate brachytherapy. This serendipitous patient-centred QA/QI process may be a useful model for empirically evaluating complex cancer treatment procedures and for screening for substandard care.


Cancer treatments are complex processes comprised of individual procedures that may vary. Trials typically compare overall treatment strategies but not the specific procedures used to execute them. Rather, specific procedures are chosen because of other factors, e.g. institutional tradition, clinical reasoning and provider experience. Although patients currently have a peripheral role in evaluating medical care, patient-reported outcomes (PROs) are responsive, valid outcome measures underused in quality assessment.

Percutaneous ultrasound (US)-guided prostate brachytherapy [1] is a standard treatment option that may cause urinary, bowel and sexual dysfunction [2, 3]. Of special interest are short-term urinary obstruction and irritation and long-term incontinence, which emerges ≈5 years after treatment [3] and is associated with high-dose urethral irradiation [4]. To mitigate urinary toxicity, D'Amico et al. [5] modified the standard technique to sharply reduce radiation dose to the periurethral prostate using MRI to ensure more accurate radioactive seed placement and possibly reduce radiation ‘cold spots’ that may allow cancer recurrence.

To determine whether the experimental technique (MRI-guided brachytherapy technique, MR-BT) successfully mitigated urinary toxicity, we initiated a prospective comparative effectiveness study comparing it with standard US-guided brachytherapy technique (US-BT), which includes the entire prostate in the target volume, using PROs as the primary outcome. We established three treatment cohorts who received either the experimental MR-BT technique or US-BT at one of two other sites (US-BT1 and US-BT2). Unexpectedly, one US-BT control population reported worse urinary outcomes in an unplanned preliminary analysis. Alerted by these results, treating physicians reviewed procedures, collaboratively identified a difference in procedures, made a corrective modification, and used the PRO measures to evaluate the change in technique. We present our serendipitous experience as a potentially useful model for patient-centred, outcomes-driven quality improvement in cancer care.

Patients and Methods

Patients with previously untreated clinically localised prostate cancer who selected brachytherapy at one of three prostate brachytherapy treatment programmes were eligible for the study. Decreased urinary toxicity was the objective of the MR-BT protocol. The study goal was to ascertain whether short-term and long-term urinary toxicity was successfully reduced in the MR-BT group. To provide a more generalizable comparison than a single control group would allow, we compared MR-BT patients with patients at two other sites who were using standard US-BT. The study was approved by each participating institution's Institutional Review Board.

Brachytherapy Techniques

All brachytherapy techniques used 125I and a target dose of 145 Gy. The MR-BT treatment plan was designed to deliver 150% of the prescription dose to the peripheral zone, with no urethral tissue dose to exceed 200 Gy. Both US-BT treatment plans were designed to deliver at least 100% of the prescription dose to the entire prostate gland with no central point (the anatomical surrogate for the urethra) receiving >130% of the prescription dose. The initial US-BT2 treatment plan did not formally identify the urethra during the planning or implant but assumed it was in the middle of the central/transition zone. Catheter use was rejected to reduce pelvic trauma. Treatment procedures did not significantly change during the study period other than the US-BT2 programme's use of a bladder catheter.

Data Collection and Outcome Measures

Study staff registered patients who completed the baseline questionnaire before receiving treatment at the data management centre. At each specified follow-up time (1, 3, 12, 24, 36, 48 and 60 months) study staff mailed patients follow-up questionnaires and performed follow-up telephone calls and mailings. An experienced genitourinary oncology research nurse abstracted information documenting prostate cancer prognostic factors, comorbid diseases, other clinical results and treatments from medical records using procedures from earlier studies [6-9].

Patient demographics

Patients reported demographic information, including age, race, marital status and education, on the baseline (pretreatment) questionnaire.

Urinary, bowel and sexual dysfunction

Patient-completed questionnaires included the four domains of the validated Prostate Cancer Symptom Indexes (PCSI): urinary incontinence, urinary obstruction/irritation, bowel dysfunction, and sexual dysfunction, as well as related bother or distress [10]. These domains assess the most common side-effects of local therapy of prostate cancer. The Urinary Incontinence Index contains three questions gauging the degree of urinary control. The Urinary Obstruction and Irritation Index contains five questions assessing hesitancy, frequency, nocturia, dysuria, and urgency. While some PROs instruments do not distinguish symptoms of obstruction and irritation from urinary incontinence [11], others, including the PCSI, do [10, 12]. Clinical [13, 14] and psychometric evidence [15] indicates they overlap, in part because of urethral irritation resulting in urge incontinence, although the evidence is mixed [16]. The Bowel Problems scale items include diarrhoea, urgency of bowel movements, rectal pain, bleeding, passing mucus, abdominal cramping, and tenesmus. The Sexual Dysfunction scale items assess patients' perceptions of their erectile function and the quality of orgasm and ejaculation. Each index is scored by summing the component items and standardising values to vary from 0 (no dysfunction) to 100 (maximum dysfunction).

These measures have been classified into levels of function using patient-reported distress: ‘normal’, ‘intermediate’ and ‘poor’. Normal function describes a patient who reported no distressful symptom within a symptom scale. For example, normal bowel function represents no reported symptom level associated with distress for any item in the Bowel Problems scale. Reporting at least one distressful but no very distressful symptom in a scale classifies function as intermediate, while at least one very distressful symptom classifies function as poor [17].


To control for significant comorbid disease, we used the Greenfield Index of Co-Existent Disease (ICED), which assesses comorbid disease severity and the level of functional impairment [18].

Clinical risk groups were defined by the method of D'Amico et al. [19]. Patients were considered low risk if they were clinical stage T1c or T2a, with a PSA level of <10 ng/mL, and Gleason score 4–6, intermediate risk if clinical stage T2b, Gleason score 7 or PSA level of 10–20 ng/mL, and high risk if clinical stage T3, Gleason score 8–10 or PSA level of >20 ng/mL.

Statistical Methods

All reported P values are two-sided. For categorical variables, we used Fisher's exact test, and for continuous variables, the Wilcoxon rank-sum test (Mann–Whitney). We defined clinically significant group changes in scale scores as mean differences of 0.5–1.0 sd of the baseline scores [20, 21].


We enrolled 286 eligible patients, including 77 MR-BT patients and 209 US-BT patients (110 US-BT1 patients and 99 US-BT2 patients). We excluded from analysis the 23 patients (8%) who discontinued before completing the 3-month follow-up (four MR-BT patients, 11 US-BT1 patients and eight US-BT2 patients. The preliminary analysis that prompted the quality assessment and quality improvement (QA/QI) process was performed in September 2005 to obtain an early impression of the impact of the MR-BT technique on early urinary morbidity. It included 74 US-BT1 and 55 US-BT2 patients. US-BT2 patients received the modified technique as early as October 2005, but adherence to the modified technique was incomplete until December 2005. In all, 60 US-BT2 patients were treated with the original technique and 31 patients with the modified technique.

Pretreatment Characteristics

Overall, US-BT1 and US-BT2 patients were demographically and clinically similar before and after modification of the technique (Table 1). Enrolled patients were almost exclusively Caucasian and highly educated. Clinical prognostic factors for both prostate cancer and other comorbid medical diseases were also similar and favourable. More than 80% of each treatment group had Gleason score ≤6, >90% had stage T1 tumours, and three quarters were low risk by D'Amico's criteria. Comorbid disease burden was modest. Nearly one-third had no other medical diagnosis, and nearly all remaining patients had only asymptomatic diagnoses (ICED 1).

Table 1. Comparison of sociodemographic and clinical characteristics at treatment for 190 early prostate cancer brachytherapy patients who completed at least the 3-month follow-up who underwent one of the two standard brachytherapy treatment programmes, US-BT1 and US-BT2, at the time of the first preliminary analysis and all patients who underwent the US-BT2 treatment programme before and after modifying the technique
CharacteristicLevelUS-BT1 patients at preliminary analysisUS-BT2 patients at preliminary analysisUS-BT2 patients original techniqueUS-BT2 patients altered techniqueUS-BT1 patients after preliminary analysisP
  1. Wilcoxon ranksum test for continuous or ordered variables and Fisher's exact test for categorical variables. US-BT1 patients vs US-BT2 patients. *US-BT2 patients treated with original technique vs US-BT2 patients treated with altered technique. #US-BT2 patients treated with altered technique vs US-BT1 patients treated after preliminary analysis.
Number of patients 7455603125 
Age, yearsmedian69.867.766.965.970.50.10
Ethnic group, %Caucasian97989894921.0
Marital status, %married73787881800.81
divorced, separated or widowed20161713161.00#
Highest education, %≤ high school23181733240.82
college, no graduate degree46474847560.16*
graduate degree31363520200.73#
Baseline PSA level, ng/mLmedian4.
PSA group, %≤10 ng/mL939595971001.0
10–20 ng/mL755301.0*
>20 ng/mL000001.0#
Gleason score, %4–681828381891.0
Clinical stage, %T1918788971000.60
Risk category, %low77767881891.0
ICED level, %035312819220.67
2 or 3143600.83#

Early PROs

At the preliminary analysis, US-BT2 patients reported increased urinary dysfunction compared with US-BT1 patients during the first 3 months after treatment that were no longer evident by 12 months. The differences in urinary incontinence at 3 months and obstruction/irritation at 1 and 3 months were clinically significant (mean 6.8 to 11.2 points), and were statistically significant for 3-month incontinence and 1-month obstruction/irritation and borderline statistically significant for 3-month obstruction/irritation (Table 2). US-BT2 patients were less likely to maintain their pretreatment level of function, but the differences in these grouped data did not reach statistical significance (data not shown). For example, 74% of US-BT2 patients with normal pretreatment function for obstruction/irritation had poor function at 1 month, compared with 47% of such US-BT1 patients. As expected, post-treatment bowel problems and sexual dysfunction did not differ by technique (data not shown).

Table 2. First preliminary analysis of early (pretreatment to 12 months after treatment) urinary dysfunction for the initial 129 early prostate cancer brachytherapy patients who had completed at least 3-month follow-up and underwent the standard brachytherapy technique, including 74 patients who had undergone the US-BT1 technique and 55 patients who had undergone the US-BT2
ScalePatients, nBaseline score (sd)Mean change from baseline score (sd) at
1 months3 months12 months
  1. *Wilcoxon rank sum.
Urinary incontinence     
US-BT1743.5 (9.8)7.9 (15.3)7.0 (15.6)3.9 (17.6)
US-BT2552.0 (6.6)10.3 (17.4)14.4 (20.6)5.3 (14.0)
P* 0.430.620.040.36
Urinary obstruction/irritation     
US-BT17417.1 (10.1)24.2 (18.5)21.1 (18.8)8.5 (17.0)
US-BT25518.0 (12.3)35.4 (19.8)27.9 (19.2)9.8 (16.5)
P* 0.840.0020.070.93

Results of Change in Brachytherapy Technique for US-BT2 Patients

Because of the unexpected discrepancy in short-term urinary toxicity, we convened the treating radiation oncologists (I.K., A.D. and A.Z.) to collaboratively review the US-BT1 and US-BT2 treatment protocols. The US-BT1 protocol used an indwelling bladder catheter during treatment to allow better visualization of the urethra that helped the brachytherapist manoeuver seeds away from its immediate vicinity. In contrast, the US-BT2 treatment protocol did not use the catheter to minimise acute urethral trauma, improve short-term tolerance of radioactive seeds and increase the procedure's acceptability to patients. The radiation oncologists concluded that this procedural difference most probably accounted for the difference in urinary toxicity. Therefore, the US-BT2 treatment protocol was amended to use the bladder catheter.

Table 3 indicates the reduced urinary dysfunction in subsequent US-BT2 patients who received the altered treatment using the catheter. Because the study was designed to compare the three full cohorts, we had modest power to compare the subgroups treated before and after the change. Despite this, improved short-term urinary outcomes after the change were evident, particularly in urinary incontinence. The mean decreases in 1- and 3-month urinary incontinence and 1-month obstruction/irritation were clinically significant (5–10 points), although the 1-month improvement in obstruction/irritation did not achieve statistical significance, and the benefit was no longer evident by 3 months. Urinary outcomes of US-BT2 patients who received the altered technique were also similar to subsequently treated US-BT1 patients.

Table 3. Early (pretreatment to 12 months after treatment) urinary dysfunction for 91 patients in the US-BT2 brachytherapy treatment group before and after change in technique
ScalePatients, nBaseline score (sd)Mean change from baseline score (sd) at
1 months3 months12 months
  1. *Wilcoxon rank sum.
Urinary incontinence     
US-BT2 before change601.8 (6.4)10.2 (16.9)15.5 (20.2)4.9 (13.5)
US-BT2 after change314.8 (9.1)5.2 (12.1)5.6 (17.1)0.8 (9.7)
US-BT1 after preliminary analysis251.0 (8.9)2.0 (10.8)5.0 (15.5)8.6 (17.3)
P* US-BT2: before vs after change
P* US-BT2 after change vs US-BT1 after preliminary analysis 0.080.440.870.15
Urinary obstruction/irritation     
US- BT2 before change6018.2 (12.5)34.6 (19.6)29.1 (21.7)9.4 (15.9)
US-BT2 after change3117.8 (10.4)24.6 (11.0)28.3 (20.1)11.6 (14.9)
US-BT1 after preliminary analysis2518.3 (10.0)22.3 (21.5)16.3 (17.4)9.7 (13.2)
P* US-BT2 before vs after change 0.880.060.800.48
P* US-BT2 after change vs US-BT1 after preliminary analysis 0.700.430.070.81


Unexpectedly discrepant preliminary results in the control arms of our comparative effectiveness study of an experimental brachytherapy technique designed to reduce urinary toxicity provided an opportunity to use PROs as the measurement tool for a successful QA/QI process. Because there was no evidence basis to choose, the US-BT treatment protocols at the control institutions differed on whether to use an indwelling bladder catheter during radioactive seed placement. The unexpected difference we found in short-term urinary symptoms between treatment groups prompted a collaborative re-evaluation of the US-BT brachytherapy techniques that identified catheter use as the most likely cause. After catheter use was incorporated into the US-BT2 treatment protocol, short-term urinary morbidity was improved by statistically and clinically meaningful amounts in several patient-reported measures and became similar to those of US-BT1 patients, whose protocol used the catheter. No other changes in brachytherapy technique occurred during the study period.

These results may have significant implications for QA/QI processes in cancer care, which has become an urgent focus of researchers and policy makers [22]. Despite the recognised potential value of PROs in quality of care studies [23], current QA/QI processes largely confine input from patients, the primary consumers of medical care, to satisfaction surveys [24], whose relationship to quality of care is modest and variable [25, 26]. This approach uses patient reports about treatment-related symptoms, on which they have authority, in contrast to their judgments of the technical quality of their care, on which they do not. PROs that indicate increased treatment-related toxicity are a potentially useful signal of substandard care, permitting earlier corrective actions to limit the impact of suboptimal or poorly executed procedures, such as unfortunate recent experiences with brachytherapy [27] and intensity-modulated radiation therapy [28]. In the present case, short-term (3-month) follow-up sufficed to identify the treatment group with poorer outcomes and initiate corrective action. Longer follow-up would be required to assess quality of care using treatment-related morbidity that develops later after brachytherapy, e.g. sexual dysfunction or urinary incontinence. However, much of the treatment-related toxicity for prostate [9, 29, 30] and other cancers [31] occurs in the first year after treatment.

This QA/QI process uses clinical outcomes to evaluate care, rather than indirect measures, e.g. compliance with process guidelines [32, 33]. Guideline compliance is difficult to document [34] and with rare exceptions [35] has little demonstrated relationship to improved clinical outcomes in oncology [36, 37].

This approach may prove valuable, but questions remain. Our present experience represents a single experience that requires replication. Measuring quality of life can be difficult and costly, although web-based PRO data collection methods [38] may simplify data collection and reduce cost. Further, the empirical relationship between patient-reported toxicity and other measures of treatment quality has not been demonstrated. However, increased treatment-related toxicity is a plausible signal of lower treatment quality, as minimising toxicity is both intrinsically beneficial and an implicit goal of high quality medical care. We have begun to rigorously assess and extend the use of PROs in QA/QI processes at Continuum Cancer Centers of New York for patients treated for head and neck, lung and breast cancers.


We would like to thank Anita Rodrigues, Catherine Reyes, and Nida Zaidi for their efforts to maintain ongoing contact with study participants, ensure complete data collection and meticulously carry out and document study procedures and to Pamela Fontaine-Rothe for her meticulous chart abstractions.

Conflict of Interest

None declared.


Greenfield Index of Co-Existent Disease


MRI-guided brachytherapy technique


Prostate Cancer Symptom Index


patient-reported outcome


quality assessment and quality improvement




US-guided brachytherapy technique