Prognostic value of the CAPRA clinical prediction rule: a systematic review and meta-analysis

Authors


Correspondence: Rose Galvin, HRB Centre for Primary Care Research, Department of General Practice, Royal College of Surgeons in Ireland, 123 St Stephens Green, Dublin 2, Republic of Ireland.e-mail: rosegalvin@rcsi.ie

Abstract

What's known on the subject? and What does the study add?

  • Prostate cancer is a significant cause of mortality among men. A number of prognostic instruments exist to predict the risk of recurrence among patients with localised prostate cancer. This systematic review examines the totality of evidence in relation to the predictive value of the CAPRA clinical predication rule by combining all studies that validate the rule.

Objectives

  • To perform a systematic review with meta-analysis that assesses the 3- and 5-year predictive value of the CAPRA rule, a clinical prediction rule derived to predict biochemical-recurrence-free survival in men with localized prostate cancer after radical prostatectomy.
  • To examine the predictive value of the CAPRA rule at 3 and 5 years stratified by risk group (0–2 low risk, 3–5 intermediate risk, 6–10 high risk).

Patients and Methods

  • A systematic literature search was performed to retrieve papers that validated the CAPRA score.
  • The original derivation study was used as a predictive model and applied to all validation studies with observed and predicted biochemical-recurrence-free survival at 3 and 5 years stratified by risk group (0–2 low, 3–5 intermediate, 6–10 high).
  • Pooled results are presented as risk ratios (RRs) with 95% confidence intervals, in terms of over-prediction (RR > 1) or under-prediction (RR < 1) of biochemical-recurrence-free survival at 3 and 5 years.
  • A chi-squared test for trend was computed to determine if there was a decreasing trend in survival across the three CAPRA risk categories.

Results

  • Seven validation studies (n = 12 693) predict recurrence-free survival at 5 years after radical prostatectomy. The CAPRA score significantly under-predicts recurrence-free survival across all three risk strata (low risk, RR 0.94, 95% CI 0.90–0.98; intermediate risk, RR 0.94, 95% CI 0.89–0.99; high risk, RR 0.72, 95% CI 0.60–0.85).
  • Data on six studies (n = 6082) are pooled to predict 3-year recurrence-free survival. The CAPRA score correctly predicts recurrence-free survival in all three groups (low risk, RR 0.98, 95% CI 0.95–1.00; intermediate risk, RR 1.03, 95% CI 0.99–1.08; high risk, RR 0.87, 95% CI 0.73–1.05).
  • The chi-squared trend analysis indicates that, as the trichotomized CAPRA score increases, the probability of survival decreases (P < 0.001).

Conclusions

  • The results of this pooled analysis confirm the ability of the CAPRA rule to correctly predict biochemical-recurrence-free survival at 3 years after radical prostatectomy.
  • The rule under-predicts recurrence-free survival 5 years after radical prostatectomy across all three strata of risk.

Introduction

Cancer of the prostate is a significant cause of death among men in Europe and is the most common non-skin-cancer neoplasm [1]. The prevalence in Europe was estimated to be in the region of 382 000 in 2008, with incidence still increasing rapidly due to early diagnosis with PSA testing [1]. In Ireland, it is estimated that men have a one-in-nine chance of developing the disease before the age of 75 years [2]. Selection of the most effective treatment modality for prostate cancer represents a challenge to clinicians as there are a number of therapeutic options available to patients including active surveillance, surgery, hormonal therapy and radiation therapy [3, 4]. Risk stratification at diagnosis permits the identification of patients who may benefit from active surveillance, those who might benefit from immediate local treatment and those who require aggressive multi-modal therapy [5]. In recent years, a number of algorithms, nomograms and clinical prediction rules have been developed to assist clinicians in predicting patient outcome following these different treatments. However, many of these tools are unsuitable for clinical practice due to the complex nature of their administration and their poor predictive values [6].

In 2005, Cooperberg et al. [7] derived the CAPRA rule (Cancer of the Prostate Risk Assessment) to predict the pre-operative probability of biochemical-recurrence-free survival after radical prostatectomy in patients with clinically localized prostate cancer. This score includes five independent variables: serum PSA level, Gleason score, clinical T stage, percentage positive prostate biopsy and age at diagnosis. An overview of the score is provided in Fig. 1. Several validation studies have been completed to determine the predictive value of the CAPRA rule. The aim of this systematic review and meta-analysis is to determine the predictive value by combining all studies that validate the score.

Figure 1.

Overview of the CAPRA score. The PSA value used is the highest value recorded in the 9 months prior to diagnosis. Gleason scores are recorded from the diagnostic biopsy cores with the highest total and highest primary scores. The clinical TNM stage is the highest reported from 1 month before to 3 months after the date of diagnosis. Percentage positive biopsy (PPB) is calculated from the biopsy pathological report.

Methods

Search Strategy

The PRISMA guidelines for reporting of systematic reviews and meta-analysis were followed to conduct this review [8]. We aimed to identify all studies that validate the CAPRA score irrespective of setting or study design. An online literature search was conducted in June 2011 (updated in March 2012) and included the following search engines: Pubmed, EMBASE, Cochrane Library and EBSCO. The databases were searched using a combination of the keywords and MeSH terms including risk assessment, prostate cancer, prostatic neoplasms, CAPRA and prostate risk assessment. The search was supplemented by hand searching references of retrieved papers and searching Google Scholar. The original CAPRA derivation paper was published in 2005; therefore we limited our search to 2005 onwards.

Study Selection and Data Extraction

Studies were included if they met the following inclusion criteria: study design – prospective or retrospective cohort studies; patient population – patients with localized prostate cancer undergoing a radical prostatectomy; explanatory variables – CAPRA rule validated; setting of care – population and hospital based studies; outcome – biochemical-recurrence-free survival at 3 or 5 years. We used the original definition of biochemical recurrence after radical prostatectomy: two consecutive PSA values ≥0.2 ng/mL at any time postoperatively or the administration of any adjunctive prostate cancer treatment more than 6 months after radical prostatectomy. Studies that included the same data set for more than one publication were only included once in the meta-analysis. Two reviewers (PM, RG) read the titles and/or abstracts of the identified references and eliminated irrelevant studies. Studies that were considered eligible for inclusion were read fully in duplicate and their suitability for inclusion was independently determined by both PM and RG. Disagreements were managed by consensus. Additional data were sought from authors where necessary. Data were extracted on study design and setting, patient characteristics, method of data extraction and outcome at follow-up.

Quality Assessment

Quality assessment was independently performed by two researchers (PM and RG) following the modified methodological standards of McGinn for validation studies of clinical prediction rules [9]. The McGinn criteria examine the internal and external validity of studies in terms of blinded assessment of predictor variables and outcome (survival/death), numbers followed up in the study (minimum ≥80%), methods of patient selection and spectrum of patients included with prostate cancer.

Statistical Methods

The initial CAPRA derivation study was used as a predictive model against which subsequent validation studies were compared. The number of individuals predicted to survive across the three strata of risk – low risk (0–2 points), intermediate risk (3–5 points) and high risk (6–10 points) as determined by the original derivation study – was compared with the observed number of individuals who survived in each of the subsequent validation studies. A chi-squared test for trend was computed to determine if there was a decreasing trend in survival across the three CAPRA risk categories.

Review Manager 5 software from the Cochrane collaboration was used to perform the analysis, determine heterogeneity and produce forest plots. Results are presented as risk ratios (RRs) with 95% CIs using the Mantel–Haenszel statistical method. An RR score of 1 represents accurate prediction by the CAPRA rule, <1 represents under-prediction and >1 over-prediction. A random-effects analysis was applied and heterogeneity across the studies was quantified using the I2 statistic. If the I2 statistic was >50%, it was deemed that there was significant heterogeneity between the studies.

Results

Study Identification

A flow diagram of the search strategy is presented in Fig. 2. Two researchers screened all potential papers. The search strategy yielded 4742 papers of which 4724 publications were excluded on their title or abstract. Seven of the remaining 18 studies met the inclusion criteria and were selected for analysis [5, 10-15].

Figure 2.

Flow diagram of search strategy.

Study Description

Table 1 summarizes the characteristics of the included studies. All publications are in English. All studies are hospital based retrospective validation studies of prospective cohorts. Additional data were provided from three authors. The included studies range in size from 211 to 6737 patients [5, 12]. A total of 12 693 participants are included in the analysis.

Table 1. Characteristics of studies
AuthorsStudy settingStudy typeParticipants n, mean age (SD)Definition of biochemical recurrenceOutcomes reported
Cooperberg et al., 2006 [13]Hospital basedRetrospective validation of prospective consecutive cohortn = 1309, 61.9 years (±6.6 years)1 PSA level >0.2 ng/mLBiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy
2 PSA levels of 0.2 ng/mL
Secondary treatment for elevated postoperative PSA
May et al., 2007 [11]Hospital basedRetrospective validation of prospective consecutive cohortn = 1296, 63.7 years (±5.5 years)1 PSA level >0.2 ng/mLBiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy
2 PSA levels of 0.2 ng/mL
Secondary treatment for elevated postoperative PSA
Zhao et al., 2007 [12]Hospital basedRetrospective validation of prospective consecutive cohortn = 6737, 58 years (no SD provided)1 PSA level >0.2 ng/mLBiochemical-recurrence-free survival at 5 years after radical prostatectomy
Lughezzani et al., 2009 [14]Hospital basedRetrospective validation of prospective consecutive cohortn = 1976, 62.1 year (no SD provided)1 PSA level >0.1 ng/mLBiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy
Loeb et al., 2010 [10]Hospital basedRetrospective validation of prospective consecutive cohortn = 726, 59.3 years (no SD provided)Repeated measurement of PSA ≥0.2 ng/mLBiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy
Secondary treatment for elevated postoperative PSA
Ishizaki et al., 2011 [5]Hospital basedRetrospective validation of prospective consecutive cohortn = 211, 62.2 years (5.8 years)Two consecutive measures of PSA ≥0.2 ng/mL or secondary treatment for elevated postoperative PSABiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy
Tamblyn et al., 2011 [15]Hospital basedRetrospective validation of prospective consecutive cohortn = 635, median age 62 years (range 37–75 years)Two consecutive measures of PSA ≥0.2 ng/mL or secondary treatment for elevated postoperative PSABiochemical-recurrence-free survival at 3 and 5 years after radical prostatectomy

Study Quality

The methodological quality of the studies is detailed in Fig. 3. The external validity is good overall. However, patients in one of the studies were selected from a single surgeon database [10]. In relation to the internal validity of the studies included, none of the studies reported whether assessors were blinded to patient outcome or predictor variables.

Figure 3.

Methodological quality of the studies.

Three-Year Survival

Six studies (n = 6082) report biochemical-recurrence-free survival 3 years after radical prostatectomy [5, 10, 11, 13-15]. The CAPRA rule correctly predicts biochemical-recurrence-free survival across all three risk strata: low risk, RR 0.98, 95% CI 0.95–1.00, I2 = 61%; intermediate risk, RR 1.03, 95% CI 0.99–1.08, I2 = 49%; high risk, RR 0.87, 95% CI 0.73–1.05, I2 = 43%). The results are displayed in Fig. 4. Using the original derivation study as a predictive model, 4880 men are predicted to be alive without biochemical recurrence of prostate cancer at 3 years. The pooled observations in the subsequent validation studies indicate that 4889 men are alive without biochemical recurrence at 3 years after radical prostatectomy.

Figure 4.

Recurrence-free survival at 3 years.

The chi-squared analysis indicates that, as the trichotomized CAPRA score increases, the probability of survival decreases (P < 0.001). Our pooled data indicate that 92.1% of men in the low risk group are alive without biochemical recurrence of prostate cancer at 3 years, 75.7% in the intermediate risk group and 47.6% in the high risk group.

Five-Year Survival

Seven studies (n = 12 693) report recurrence-free survival 5 years after radical prostatectomy [5, 10-15]. The CAPRA rule significantly under-predicts biochemical-recurrence-free survival across all three risk strata, particularly in the high risk group: low risk, RR 0.94, 95% CI 0.90–0.98, I2 = 80%; intermediate risk, RR 0.94, 95% CI 0.89–0.99, I2 = 55%; high risk, RR 0.72, 95% CI 0.60–0.85, I2 = 13%. The results are presented in Fig. 5. There are 9154 biochemical-recurrence-free survivals predicted and 9941 survivals observed 5 years after radical prostatectomy.

Figure 5.

Recurrence-free survival at 5 years.

The chi-squared trend analysis demonstrates that the probability of survival decreases (P < 0.001) in the higher risk strata. The pooled data indicate that 89.9% of men in the low risk group are alive without biochemical recurrence of prostate cancer at 5 years, 67% in the intermediate risk group and 33.7% in the high risk group.

Discussion

Statement of Principal Findings

This systematic review shows that the CAPRA rule correctly predicts biochemical-recurrence-free survival 3 years after radical prostatectomy in patients with prostate cancer across all three strata of risk. However, our results show that there is a significant under-prediction of biochemical-recurrence-free survival 5 years after radical prostatectomy.

Current Context and Future Research Directions

This is the first systematic review to examine the predictive value of the CAPRA rule in patients following radical prostatectomy. Recent studies have modified the CAPRA predictor variables and the outcomes and examined the applicability of the rule across a variety of prostate cancer treatment groups [6, 16, 17]. In 2009, Cooperberg et al. [6] investigated the value of the rule to predict metastases, prostate-cancer-specific mortality and all-cause mortality in patients with clinically localized prostate cancer who were managed with one of five primary modalities including radical prostatectomy, cryotherapy, brachytherapy, external beam radiation therapy, primary androgen deprivation therapy and active surveillance. The CAPRA score was reasonably accurate for predicting metastases (c-index = 0.78), cancer-specific mortality (c-index = 0.80) and all-cause mortality (c-index = 0.71). Halverson et al. [17] also tested the CAPRA rule in a population treated with external beam radiation therapy. Compared with our systematic review, patients treated with radiation therapy had higher disease risk (28% with a CAPRA score of 6–10 vs 7% respectively). However, in general, with increasing risk men are less likely to undergo prostatectomy and more likely to receive radiation or androgen-deprivation monotherapy [18].

The CAPRA rule significantly under-predicts recurrence-free survival across all three risk strata at 5 years. In an effort to improve the predictive value of the CAPRA rule at 5 years, Cooperberg and colleagues added additional predictor variables to the original CAPRA rule by incorporating pathological data in addition to the pretreatment data applied in the CAPRA score. The inclusion of pathological radical prostatectomy or lymph node specimens has been shown to improve the precision of biochemical recurrence predictions [14]. In a manner similar to the original CAPRA rule, the revised CAPRA-S rule (the Cancer of the Prostate Risk Assessment post Surgical rule) is scored from 1 to 12 points [19]. However, the CAPRA-S score is based on data available before and after the surgical resection of the prostate and variables include pre-operative PSA, pathological Gleason score of the operative specimen, surgical margin status, presence or absence of extracapsular extension, seminal vesicle invasion and/or lymph node involvement. Results indicate that the CAPRA-S rule performed better than the original CAPRA score (c-index 0.77 vs 0.66 respectively). These findings suggest that this new scoring system improves prediction of outcomes after radical prostatectomy compared with the original rule. However, this new score should undergo the further narrow and broad validation in large international data sets to determine its predictive ability compared with the original rule. Furthermore, research to date indicates that the majority of clinical prediction rules do not undergo impact analysis [20]. Therefore there is a need for future large multicentre randomized controlled trials to examine the impact of applying the original CAPRA rule and the revised CAPRA-S rule in different clinical settings and amongst ethnically diverse populations, in terms of patient outcome, clinician behaviour, cost effectiveness and resource use, or any combination of these.

Strengths and Weaknesses of the Study

Our systematic review pools data from seven separate cohort studies of individual patients with localized prostate cancer and presents the results across three different strata of risk. The method of calibration we used examines the predictive ability of the score using the ratio of biochemical-recurrence-free survival (from the original derivation study) to observed biochemical-recurrence-free survival in the subsequent validation studies. The absolute risk of recurrence-free survival is presented in risk strata so that the value of the CAPRA rule across these strata can be interpreted by clinicians in a meaningful way. Our method of analysis is based on a comparative approach that extends and employs the absolute risk from the derivation study as a model to generate predicted values in subsequent validation studies. This method of analysis has been used in previously published studies of this nature [21-23].

Despite its ability to accurately determine recurrence-free survival at 3 years, clinical use of the CAPRA rule has some limitations. There is significant heterogeneity in the calibration analysis (I2 = 43%–61% at 3 years and 13%–80% at 5 years). Heterogeneity in the studies could be due to a variety of factors. First, there are only seven papers included in the review and all studies are subject to information bias. Second, one study excluded one of the CAPRA variables (percentage positive prostate biopsy) and reported a nine-point CAPRA score instead of the original 10-point CAPRA rule [12]. Additionally the authors used a slightly different categorization of the Gleason score. However, a subsequent sensitivity analysis demonstrated that this study did not contribute significantly to heterogeneity. Overall the initial validation study by Cooperberg et al. [13] contributed the most to the heterogeneity. However, it has been noted that risk prediction tools tend to perform better among academic cohorts compared with population based cohorts, most probably due to the uniformity of practice in the academic setting [19]. For example, the CAPRA score yielded a c-index of 0.68 in the initial population based validation study whereas it performed with a c-index as high as 0.81 in academic based validation studies [11, 12].

Clinical Implications

The CAPRA rule provides a straightforward instrument to facilitate disease risk classification in clinical decision making and research. Accurate pre-operative risk assessment is an integral component of pre-operative counselling for patients undergoing radical prostatectomy for clinically localized prostate cancer. The rule is not intended to replace individualized clinician–patient decision making but rather to assist clinicians with this process. The CAPRA score is not designed to be used as a clinical decision rule and research shows that it is not a driver of treatment choice [18]. The rule was primarily derived to assist clinicians in predicting patient outcome following radical prostatectomy. Our findings show that it has good predictive ability at 3 years but that it significantly under-predicts recurrence-free survival at 5 years. Furthermore, while the CAPRA rule has been shown to predict pathological stage, PSA biochemical recurrence and distant outcomes (metastasis and mortality) following a range of treatment strategies, we have only pooled data from individuals who have undergone a radical prostatectomy. Therefore clinicians should interpret the rule with caution when applying it to other treatment groups. We attempted to pool the data from other treatment groups but it is not possible to pool data from fewer than three studies. In addition clinicians need to be mindful that three of the CAPRA variables, biopsy Gleason score, clinical T stage and percentage of positive biopsy cores, are by nature approximations and may therefore underestimate or overestimate true grade and extent of cancer [19]. Cooperberg et al. [19] have suggested that because distribution and interpretation of prostate cancer risk factors such as the Gleason grading system are subject to change over time, it is important to verify the relevance of the rule in contemporary patients. Finally, in the absence of robust evidence or clinical guidelines supporting any active treatment approach over another for localized prostate cancer, there is a need for high quality research comparing treatment effectiveness in localized prostate cancer to help guide treatment decision making [18].

In conclusion, the results of this pooled analysis confirm the ability of the CAPRA rule to correctly predict recurrence-free survival at 3 years after radical prostatectomy. However, our results show that there is a significant under-prediction of biochemical-recurrence-free survival 5 years after radical prostatectomy.

Funding Sources

This work was supported by the Health Research Board (HRB) of Ireland through the HRB Centre for Primary Care Research under grant HRC/2007/1.

Conflict of Interest

None declared.

Ancillary