Validation and comparison of the two Kattan nomograms in patients with prostate cancer treated with ¹²⁵iodine brachytherapy

Abstract

Study Type – Prognostic (cohort)

Level of Evidence 2b

What's known on the subject? and What does the study add?

Nomograms are based on large patient population. Their applicability should be externally validated.

Among 747 brachytherapy patients we evaluated two kattan nonograms and conclude that they have limited value to predict PSA-free survival.

OBJECTIVE

• • 
  To validate and compare the preoperative and postoperative Kattan prediction nonograms for prostate cancer recurrence after brachytherapy.

PATIENTS AND METHODS

• • 
  Patients (n= 747) treated with ¹²⁵I-brachytherapy were evaluated.
• • 
  Both nomograms were used to calculate the prediction of 5-year biochemical-freedom from failure (BFFF) based on clinical stage, Gleason score, prostate-specific antigen (PSA) level, receipt of androgen deprivation therapy and the post-implant dosimetry variable D90 (values of the minimal dose received by 90% of the prostate volume).
• • 
  The predicted values using the Kattan nomograms and the observed values were compared.
• • 
  Predictive accuracy was determined using the concordance index.

RESULTS

• • 
  The 5-year BFFF probability was 94% (95% confidence interval [CI], 92–96%) for the modified American Society for Radiation Oncology (ASTRO) definition and 97% (95% CI, 95–98%) for the Phoenix definition using Kaplan–Meier analysis.
• • 
  The predicted values of BFFF using both Kattan nomograms were lower than the observed rates in our cohort.
• • 
  The concordance index values were 0.51 and 0.52 for preoperative and postoperative nomograms, respectively.
• • 
  Concordance correlation coefficient between the two nomograms was 0.15.

CONCLUSIONS

• • 
  In our population, the 5-year BFFF outcomes rates were superior to nomogram predictions.
• • 
  Neither nomogram predicted outcomes after ¹²⁵I-brachytherapy in this non-US cohort.
• • 
  The postoperative nomogram was also a poor predictor, although it included D90 dosimetry values, as a variable of treatment quality.
• • 
  Strict inclusion criteria, perhaps more favourable than the ones on which the Kattan nomograms were based, could be the explanation for these discrepancies.

Abbreviations

ADT

androgen deprivation therapy

ASTRO

American Society for Radiation Oncology

BFFF

biochemical freedom from failure

EBRT

beam radiation therapy

INTRODUCTION

Accounting for nearly 25% of incident cancer in American males, prostate cancer is the most common non-skin malignancy among men in the developed world [1]. While several potentially curative treatment modalities exist, to date there have been no randomized controlled trials to guide clinicians on the optimal management of patients with localized prostate cancer [2]. Traditionally, treatment decisions were based on the judgment of experienced physicians alone and the patient's personal preference, taking into account PSA relapse-free data and quality-of-life outcomes [3,4]. Over the last decade, predictive nomograms have emerged as a popular and effective method of assessing PSA-free survival probabilities after treatment for localized prostate cancer [2]. These nomograms – specific to external beam radiation therapy (EBRT), or radical prostatectomy – are currently available online [5].

Brachytherapy has re-emerged as an effective treatment option for localized prostate cancer [6,7] and nomograms specific to this treatment were subsequently published [8,9]. The Kattan nomograms are the original and most widely used prediction models for prostate cancer recurrence after brachytherapy [8]. They are valuable for both patient counselling and in considering adjuvant treatment modalities such as EBRT or androgen deprivation therapy (ADT) [9]. The original brachytherapy nomogram, published in 2001, was based on a US three-centre cohort of 3512 patients. It calculated the 5-year biochemical freedom from failure (BFFF) probability and included EBRT as an independent variable [8,9]. The most important prognostic marker for survival in that nomogram is the preoperative PSA value [8]. Other prognostic indicators included patient age, Gleason score, clinical stage and whether or not the patient received adjuvant EBRT. Recent studies have called into question the accuracy of the 2001 nomogram [9–11]. Potters et al. published recently a new, postoperative Kattan nomogram that took into account additional variables such as isotope and dosimetry outcome, and this nomogram was validated with a larger US cohort [9].

Using real-time intraoperative dosimetry ¹²⁵I brachytherapy, our brachytherapy dedicated team has achieved excellent clinical PSA-based results [12]. We decided to test the predictive accuracy of both the preoperative and postoperative nomograms in a cohort of Israeli patients. We then conducted a head-to-head comparative analysis of the two nomograms based on predictive accuracy at the 5-year follow-up.

PATIENTS AND METHODS

The institutional review board of the Tel Aviv Sourasky Medical Center approved the present study and waived informed consent requirements. We retrospectively reviewed the medical records of consecutive 851 patients with prostate cancer treated with brachytherapy at the centre from February 2000 to February 2009. All patients were treated by permanent ¹²⁵I-based brachytherapy utilizing the intraoperative plan methodology as previously described [13,14]. Patients with Gleason score ≤ 6, clinical stage T₁–T₂ and PSA level < 20 ng/mL were given ¹²⁵I brachytherapy as monotherapy targeting a minimal peripheral dose of 160 Gy. Patients with Gleason score = 7, and with the same stage or serum PSA values, were given a combination of ¹²⁵I brachytherapy (minimal peripheral dose = 107 Gy) and EBRT (45 Gy at a daily dose of 1.8 Gy). ADT was given either to reduce gland size, if gland volume was above 55–60 mL or in all patients with Gleason 7 disease treated with combined EBRT. The duration of ADT ranged from 6 to 9 months. Patients with a PSA level > 10 ng/mL were advised to obtain a bone scan and those with PSA level > 15 ng/mL were instructed to perform an abdominal CT scan as well. We do not accept Gleason score 8–10 into our brachytherapy-based program.

For nomogram prediction calculations, we used Kattan's preoperative nomogram to asses the 5-year BFFF probability for each patient based on pretreatment PSA level (ng/mL), Gleason score, clinical stage according to the 1997 American Joint Committee on Cancer staging system and whether the patient was given adjuvant EBRT [8]. Additionally, the 2009 postoperative nomogram was adjusted to calculate the 5-year BFFF probability based on pretreatment PSA level (ng/mL), Gleason score, clinical stage according to the 2002 American Joint Committee on Cancer system, specific radioactive isotope, EBRT status and D90 (Gy, values of the minimal dose received by 90% of the prostate volume) [9].

The actual BFFF outcomes of the patients in the present study were then determined. Biochemical failure was defined using the Phoenix definition of a rise by 2 ng/mL or more above the nadir PSA level for comparison with the postoperative nomogram predictions [15]. For comparison with the preoperative nomogram predictions, the modified ASTRO definition was applied after the method originally described by Kattan et al. [8,16]. As evaluated in the original preoperative nomogram, we used 5 years as an endpoint and assessed the biochemical status at that date without taking into account PSA bounce as failure [8]. Twenty-six patients were lost to follow-up, and 51 died during the follow-up, none of prostate cancer.

For the statistical analysis, descriptive statistics, such as mean, median, range and proportion, were used to summarize patient characteristics. Kattan-predicted BFFF probabilities from both the preoperative and the postoperative nomograms were determined. To analyse the performance of each prediction tool, each patient's predicted probability of 5-year BFFF in each model was compared with the observed outcome (‘actual outcome’). In comparing the performance of the models, we focused on discrimination and calibration. The time-dependent Harrell's C concordance index was calculated. Concordance index is the proportion of randomly paired patients for whom the patient with the higher probability of recurrence (lower nomogram score) also had earlier disease recurrence [17]. The concordance index ranges from 0.5 (no predictive ability at all) to 1 (perfect predictive ability) [18]. The calibration performance of a model describes the extent to which the predicted probability rate reflects the true probability of 5-year BFFF. Calibration of nomogram was assessed by comparing the predicted probability of BFFF with actual outcome computed using Kaplan-Meier analysis. To facilitate this analysis, nomogram-predicted 5-year BFFF probabilities were divided into quartiles because the number of failures was relatively low. For each quartile the mean of nomogram-predicted probabilities was compared with the actual 5-year BFFF. The calibration of models was judged by constructing calibration plots, relating the predicted and observed probabilities [19]. All tests were two-tailed and P < 0.05 was considered to indicate statistical significance. The analyses were performed using STATA 11.0.

RESULTS

There were 747 patients whose medical records contained all data points necessary for evaluation using both preoperative and postoperative nomograms. The characteristics of the entire cohort are displayed in Table 1. The median patient age was 67 years with a median follow-up of 47 months. Median pretreatment PSA level was 7.2 ng/mL. Most of the patients (90%) had Gleason score ≤ 6 at presentation. Clinical stage T1c was the most common (75%). Of all patients, 89% were treated with brachytherapy alone without EBRT and 62% received no adjuvant ADT at any time. As shown in Table 2, according to the Phoenix definition, 21 patients had failed, and the 5-year BFFF probability was 97% (95% CI, 95–98%). When the modified ASTRO was applied, 31 patients had failed, and the 5-year BFFF probability was 94% (95% CI, 92–96%). Of the failed patients, 19 fulfilled both failure definitions.

Table 1.  Patient characteristics (n= 747)

IQR, interquartile range.
Age, years
Mean (range)	67 (47–81)
Median (IQR)	67 (62–71)
Biopsy Gleason score, %
4	1.3
5	6.6
6	81.9
7	10.2
Clinical stage, %
T1c	74.7
T2a	21.3
T2b–T2c	4.0
No. (%) with EBRT	82 (11)
No. (%) with ADT	285 (38)
Pretreatment PSA level, ng/mL
Mean (range)	7.2 (0.6–22)
Median (IQR)	6.9 (5.2–9.1)
D90, Gy
Monotherapy
Mean (range)	182 (118–217)
Median (IQR)	182 (174–189)
Combined with EBRT
Mean (range)	126 (114–158)
Median (IQR)	125 (121–131)
No. of failures
Phoenix	21
Modified ASTRO	31
Follow-up, months
Mean (range)	51 (1–125)
Median (IQR)	47 (29–71)

Table 2.  Observed BFFF according to Kaplan–Meier analysis

* Modified ASTRO definition (see ‘Patients and Methods’ section).
Interval time, months	0–20	20–40	40–60	60–80	80–100	100–120	120–125
No. of patients at risk
Phoenix	747	658	458	240	137	58	1
ASTRO*	747	646	448	231	130	54	1
Cumulative no. of events
Phoenix	4	10	15	14	21	0	0
ASTRO*	9	17	28	30	31	0	0
Cumulative BFFF, %
Phoenix	99	98	97	95	93	93	93
ASTRO*	99	97	94	93	92	92	92

The concordance index values were 0.51 and 0.52 for the preoperative and postoperative nomograms, respectively. The concordance index remained almost the same even when patients who received ADT were excluded in both models. Moreover, the concordance correlation coefficient between the two nomograms was also low, at 0.15 (95% CI, 0.06−0.24).

Actual rates of BFFF were higher than the preoperative and postoperative nomogram-predicted probabilities in all cases (Fig. 1). The actual 5-year BFFF compared with the nomogram-predicted probabilities in quartiles 1 to 4, respectively, were as follows: preoperative nomogram: 95% vs 79%, 93% vs 86%, 95% vs 89% and 94% vs 92% (Fig. 1A); postoperative nomogram: 99% vs 80%, 95% vs 89%, 97% vs 91% and 98% vs 93% (Fig. 1B).

Figure 1. Comparison of predicted and actual 5-year BFFF rates using preoperative (A) or postoperative (B) Kattan nomograms. The dotted line indicates theoretical perfect agreement between prediction and observation. Bars, 95% CI.

DISCUSSION

The diagnosis of prostate cancer brings with it much anxiety and concern for patients and their families [20]. Physician judgment and patient preference were once the main factors contributing to the treatment decision, but physician estimates of success could be clouded by subjective and objective confounders that make accurate predictions difficult [2]. The introduction of prognostic nomograms to overcome these confounding variables has greatly improved the ability to predict outcomes. That being said, prognostic nomograms are only as good as their accuracy and should thus be validated against varied populations if they are to be universally applied. Having extensive experience using brachytherapy to treat localized prostate cancer, we decided to perform validation of both the preoperative and postoperative nomograms in our non-US cohort. In addition, our study provides comparative results between the predictive accuracy of these two models. To the best of our knowledge, this is the first head-to-head comparison between the two models.

Using both models, the concordance index between predicted and actual outcomes was poor. In designing the preoperative nomogram, Kattan excluded patients receiving ADT, as have subsequent external validations [8,10]. In the patient population in the present study, the concordance index between the preoperative model and the results of the present study remained poor, even after excluding patients who received ADT.

Using the postoperative nomogram, the concordance index between predicted and actual outcomes was low too. Potters et al. [9] reported a concordance of 0.71 in designing the postoperative nomogram. The results of the present study show that the postoperative nomogram is not an accurate predictor of outcomes in our total cohort and is not a better predictor than the preoperative nomogram. In a recent external validation of the preoperative nomogram, Frank et al. [10] commented that, due to the retrospective nature of the data that goes into creating a predictive model, nomograms need to be periodically updated to stay clinically relevant. Based on the results of the present study, the updated postoperative nomogram still has not shown clinical relevance for our patient population.

The postoperative nomogram, taking into account a more extensive evaluation of patient characteristics, is expected to be a better predictor of outcomes than the original model [9]. We found that the agreement between preoperative and postoperative nomogram predictions was very poor in the total cohort (0.15), although performed similarly (Fig. 1).

Given the inclusion of several post-hoc variables, the poor performance of the postoperative model in the cohort was surprising. We hypothesized that this observation was due to the excellent D90 values we achieved with a mean D90 of 176 Gy in the total cohort. However, due to the logarithmic scale of the nomogram, incremental increases in D90 values above the standard dose of 145 Gy recommended by the American Brachytherapy Society [21] have a minimal impact on the total points received [9], making this explanation unlikely.

We believe the remarkable disagreement between the nomograms and the results of the present study could be explained by the following. First, our cohort consists of relatively homogeneous patients with 90% having a Gleason score ≤ 6 and 75% having a clinical stage T1c with a narrow range of PSA levels and high values of D90 (Table 1). Secondly, the number of patients who failed during 5 years of follow-up is relatively low (up to 31 patients). Both these factors attest to the probable selection bias of patients with a very good prognosis and excellent dosimetry delivered. Thirdly, the present cohort does not contain patients with Gleason scores 8–10, which have been included in the postoperative nomogram [9]. Lastly, our non-US patient population has a genetic and nurturer background different from the patient populations included to generate these original nomograms.

Because D90 is considered one of the strongest independent predictors of outcomes, one would expect the postoperative model to perform better [9]. Based on the results of the present study, the postoperative nomogram did not offer improved predictive accuracy either.

Neither Kattan nomograms offered reasonable predictions of 5-year BFFF in patients undergoing permanent prostate ¹²⁵I brachytherapy at the Tel Aviv Sourasky Medical Center. This external validation (the first time it has been tested outside the US) suggests that neither of the nomograms confer enhanced predictive accuracy. While counselling patients, physicians should be cautious when using nomogram predictions for brachytherapy patients, as contemporary actual results are, thus far, superior to both nomogram predictions.

ACKNOWLEDGEMENTS

We thank Dr Michael W. Kattan (Cleveland Clinic) for calculating 5-year adjusted postoperative predictions for our patient group. Amira Stenger, RN, MA, and medical physicists Rubi Agai and Natan Shtrauss are thanked for their assistance and efforts in the brachytherapy programme.

CONFLICT OF INTEREST

None declared.