The University of California, San Francisco (UCSF) Cancer of the Prostate Risk Assessment (CAPRA) is a novel preoperative index which predicts the risk of biochemical recurrence after radical prostatectomy. The performance of the index is at least as good as the best available instruments based on clinical variables, and the 0 to 10 score is simple to calculate for both clinical and research purposes. This study used a large external dataset to validate CAPRA.
Data were abstracted from the Shared Equal Access Regional Cancer Hospital (SEARCH) database, a registry of men who underwent radical prostatectomy at 4 Veterans Affairs and 1 active military medical center. Of 2096 men in the database, 1346 (64%) had full data available to calculate the CAPRA score. Performance of the CAPRA score was assessed with proportional hazards regression, survival analysis, and the concordance (c) index.
Of the studied patients, 41% were non-Caucasian, and their mean age was 62 years. Twenty-six percent suffered recurrence; median follow-up among patients who did not recur was 34 months. The hazard ratio (HR) for each 1-point increase in CAPRA was 1.39 (95% CI [confidence interval], 1.31–1.46). The 5-year recurrence-free survival rate ranged from 86% for CAPRA 0–1 patients to 21% for CAPRA 7–10 patients. Increasing CAPRA scores were significantly associated with increasing risk of adverse pathologic outcomes. The c-index for CAPRA for the validation set was 0.68, compared with 0.66 for the original development set.
With an expected incidence of 234,460 in the United States in 2006 and an estimated mortality of 27,350, prostate cancer is the most common noncutaneous human malignancy, and it supersedes all neoplasms except lung and colorectal in terms of mortality burden among men.1 However, the natural history of this disease can be quite protracted,2 and definitive therapy, although effective in reducing cancer-specific and overall mortality,3 may exert a significant effect on health-related quality of life.4 Risk classification at time of diagnosis aims to help identify who among diagnosed patients would be likely to do well on active surveillance, who should be treated immediately with local monotherapy, who may benefit from aggressive multimodal therapy, and who should be given early systemic therapy for presumed advanced disease.
Numerous nomograms and algorithms exist to classify patients according to pretreatment risk, thus intending to facilitate physician-patient decision-making with respect to prostate cancer management.5 The most widely used models predict pathologic outcomes or biochemical recurrence with a good degree of accuracy, but their use requires paper tables or handheld computers. The University of California, San Francisco (UCSF) Cancer of the Prostate Risk Assessment (CAPRA) is a novel scoring system developed to predict risk of recurrent disease after radical prostatectomy (RP). The CAPRA score is a straightforward 0 to 10 sum of weighted risk factors. In initial development tests that used the community practice-based Cancer of the Prostate Strategic Urologic Research Endeavor (CaPSURE) registry, the CAPRA score performed as well as the best available nomograms based on preoperative clinical variables, and the score can be easily determined without a calculator or paper table.6 This score had not yet been externally validated, however. We, therefore, conducted validation studies of the CAPRA score by using the Shared Equal Access Regional Cancer Hospital (SEARCH) database, a large, sociodemographically diverse, multiinstitutional cohort of RP patients.
MATERIALS AND METHODS
SEARCH is a registry of patients with localized prostate cancer treated with RP at 4 Veterans Affairs medical centers and one active military hospital. These sites offer access to healthcare to veterans or military personnel, respectively, regardless of insurance status or ability to pay for care. Variables collected include sociodemographic data (age, ethnicity, height, weight), clinical tumor characteristics (preoperative prostate-specific antigen [PSA], clinical stage, biopsy Gleason grade, and percentage of biopsy cores positive for prostate cancer), surgical pathology data (Gleason grade, pathologic stage, and margin status), and follow-up PSA values. Patients receiving neoadjuvant hormonal or radiation therapy are excluded from the registry. Data collection is governed and approved by institutional review boards at each participating medical center. Additional details on the SEARCH methodology have been published previously.7
Exclusion criteria for this study were missing data needed to calculate the CAPRA score (biopsy Gleason grade, preoperative PSA, clinical T classification, age, and percentage of positive cores from diagnostic biopsy), preoperative PSA <2 ng/mL, or fewer than 6 cores taken for the diagnostic biopsy. CAPRA scores were calculated for the men in the analytic population as previously described (Table 1).6 Recurrence was defined as a single PSA level >0.2 ng/mL, 2 PSA levels of 0.2 ng/mL, or secondary treatment for an elevated postoperative PSA.
CAPRA indicates cancer of the prostate risk assessment; PSA, prostate-specific antigen.
CAPRA score (0 to 10) is the total of points in each category.
Biopsy Gleason score
Positive biopsy cores
The performance of the CAPRA score in this validation set was assessed by a Cox proportional hazards regression model, with the hazard ratio (HR) for recurrence calculated with CAPRA score as both a continuous variable and a categorical variable (i.e., separately at each CAPRA score). The proportional hazards assumption of the Cox model was tested through examination of graphs of log-log plots of the variables used in the model. These plots formed approximate parallel straight lines as required. In addition, internal validation of the model was tested by comparing the Kaplan-Meier and Cox estimated values for several subsets that were defined by using factors not included in the CAPRA model. In these cases, the estimated points at the recurrence times appeared randomly scattered about the Kaplan-Meier curves.
Kaplan-Meier analysis was used to determine the probability of disease-free survival (DFS) at 3 and 5 years for each CAPRA score level. To focus on the ability of the CAPRA score to differentiate among patients at relatively low levels of risk, pairwise hazard ratios were also calculated to compare outcomes between adjacent scores: CAPRA 2 vs. CAPRA 0–1 and CAPRA 3 vs. CAPRA 2. We also calculated the concordance (c) index for the scoring system in this dataset. The c-index in survival analysis is the proportion of randomly paired patients for whom the patient with the higher probability of recurrence (i.e., higher CAPRA score) also had the earlier observed disease recurrence. The c-index ranges from 0 to 1, with 1 indicating perfect concordance and 0.5 indicating no concordance. We calculated the c-index separately for patients treated in or before 1998 and for those treated after 1998 to assess whether CAPRA may perform better or worse among more recently diagnosed patients.
Pathologic outcomes were assessed as secondary endpoints. Rates of positive surgical margins, extracapsular extension, seminal vesicle invasion, and lymph node involvement were calculated at each CAPRA level, and odds ratios (OR) were calculated by logistic regression for each endpoint for each unit increase in CAPRA score. All analyses were performed by using STATA (StataCorp, College Station, Texas) software, and all P-value calculations were 2-tailed.
Between 1988 and 2004, 2096 men were enrolled in the SEARCH database. For this analysis, we excluded 258 patients who had fewer than 6 cores taken for their diagnostic biopsies, and 78 patients with PSA of <2 at diagnosis, for whom the CAPRA score cannot be calculated.6 An additional 494 patients who were missing at least 1 of the data fields needed to calculate CAPRA were also excluded: 155 with missing PSA value, 96 with missing Gleason score, 91 with missing clinical T stage, and 339 with unknown percentage of positive biopsies. (The total is greater than 494 because some men were missing more than 1 variable.) Excluded patients comprised most of the patients treated before 1992, about half of those treated from 1992 to 1994, and only 20% of those treated since 1995. CAPRA scores could be calculated for 1346 patients; of these, 1309 (62% of the full SEARCH registry and 97% of otherwise eligible patients) had sufficient follow-up data available and were included in validation analyses.
Among the 1309 subjects in the dataset, the mean age was 61.9 (±6.6) years, and 41% of the patients were non-Caucasian. The mean and median PSA values were 9.4 (±8.1) and 7.0 ng/mL. Other sociodemographic and clinical characteristics of the cohort are presented in Table 2. Of patients in the study, 436 (33%) had positive surgical margins, 337 (25%) had extracapsular extension, 98 (7%) had seminal vesicle invasion, and 19 (2%) had positive lymph nodes on pathologic analysis. Three hundred thirty-six (26%) of the patients recurred. Mean and median follow-up times among men not recurring were 42 (±35.4) and 34 months, respectively.
Table 2. Demographic and Clinical Characteristics of Patient Cohort
Body mass index (kg/m2)
>35.0 (very obese)
Biopsy Gleason score
Pathology Gleason score
The full range of CAPRA scores was represented among the patients in the SEARCH dataset (Table 3); 18% had scores >4. Pathologic outcomes by CAPRA score are presented in Table 4. With increasing CAPRA score, there were steady increases, both clinically and statistically significant, in rates of each pathologic outcome: positive margins, extracapsular extension, seminal vesicle involvement, and lymph node involvement.
Table 3. CAPRA Score Distribution Among Validation Cohort
CAPRA indicates cancer of the prostate risk assessment; OR, odds ratio for CAPRA as a continuous variable; CI, 95% confidence interval; ECE, extracapsular extension; SV, seminal vesicle; LN, lymph node.
Number and percentage of patients at each CAPRA level with each pathologic finding after surgery.
P < .0001
P < .0001
P < .0001
P = .0005
When treated as a continuous variable across the full spectrum of CAPRA scores, the HR for recurrence for each 1-point increase in CAPRA score was 1.39 (95% CI, 1.31–1.46). Squaring this result yields an HR of 1.93 (95% CI, 1.72–2.13) for each 2-point increase in CAPRA score. HRs for recurrence at each categorical CAPRA score relative to CAPRA 0–1 are presented in Table 5, and range up to 9.90 (95% CI, 6.34–15.46) for CAPRA 7–10. No patient with CAPRA 0 recurred. In pairwise comparison, the HR for recurrence for CAPRA 2 patients was 1.89 (95% CI, 1.22–2.94) relative to CAPRA 0–1 patients (P = .005), and the HR for recurrence for CAPRA 3 patients was 1.56 (95% CI, 1.11–2.19) relative to CAPRA 2 patients (P = .010).
Table 5. Results of Cox Model Regression and Kaplan-Meier Analysis
SEARCH indicates Shared Equal Access Regional Cancer Hospital database; CaPSURE, Cancer of the Prostate Strategic Urologic Research Endeavor registry; CAPRA, Cancer of the Prostate Risk Assessment; HR, hazard ratio; 95% CI, 95% confidence interval; RFS, recurrence-free survival; Ref, reference.
Biochemical survival data from the original CAPRA development study from the CaPSURE database are provided for comparison.6
The 3- and 5-year actuarial recurrence-free survival estimates fell steadily from 92 and 86%, respectively, for CAPRA Score 0–1 to 35% and 20%, respectively, for CAPRA Scores 7–10. These data are similar to those obtained in the original development study (Table 5).6 Kaplan-Meier recurrence-free survival curves at each CAPRA score are presented in Figure 1. The c-index for CAPRA score within the SEARCH dataset was 0.68. For patients treated in or before 1998 (n = 675), the c-index was 0.65; for those treated after 1998 (n = 671), the c-index was 0.74. Figure 2 presents Kaplan-Meier curves for CAPRA scores categorized to scores indicative of low (CAPRA 0–2), intermediate (CAPRA 3–5), and high (CAPRA 6–10) risk of recurrence.
The UCSF-CAPRA was developed as a preoperative prognostic index which would perform as well as the best available instruments for prediction of biochemical recurrence after RP, yet would be easier to calculate. A previous analysis from CaPSURE found that the nomogram developed by Kattan et al,8 probably the instrument used most commonly in contemporary practice, performs well in the community setting, although it tends to underestimate risk of recurrence especially among low-risk patients.9 The derivation of the paper nomogram or computer programs required to calculate the Kattan score, however, may be difficult to understand for patients who are counseled with the assistance of the nomogram. Furthermore, because the formula behind the nomogram is not in the public domain, the Kattan score cannot be calculated easily for large numbers of patients for research purposes, and the scores need to be arbitrarily combined into a more limited number of groups for meaningful risk classification in the clinical research setting. It is likely for this latter reason that, while successful in the clinic, the Kattan score has not been used widely in clinical trials. In the original CAPRA development study using CaPSURE data, the CAPRA score met the goal of equaling the accuracy of the Kattan score, predicting recurrence with a c-index of 0.66, comparable to the Kattan nomogram, which had a c-index of 0.65 for the same cohort.6 The c-index for the current validation analysis was 0.68, which compares favorably to the value of 0.66 calculated for the original development study.
Pathology outcomes were not available for the development study because of logistical problems with pathology data in the CaPSURE database at the time of that study. In the current study, however, we were able to analyze pathologic endpoints as secondary outcomes. All 4 of the endpoints we studied—positive surgical margin status, extracapsular extension, seminal vesicle involvement, and lymph node involvement—were statistically significantly more likely with increasing CAPRA score. The association of CAPRA score with seminal vesicle involvement was particularly striking, with a near doubling of risk (OR 1.80, 95% CI, 1.60–2.02) with each 1-point increase in CAPRA score.
Biochemical recurrence, the primary outcome, was strongly associated with increasing CAPRA score. Likelihood of recurrence-free survival at 3 and 5 years also decreased steadily with increasing CAPRA score, and there was minimal overlap of Kaplan-Meier survival curves between adjacent scores. Whereas 3- and 5-year outcomes were similar between the CaPSURE and SEARCH cohorts, as demonstrated in Table 5, at least as important is the validation of the CAPRA index as a classification tool for assigning relative risk. A 2-point increase in CAPRA score produced an HR for recurrence of 1.93, reflecting well the original intention of the CAPRA model, which was that each 2-point increase in CAPRA score should roughly indicate a doubling of risk of recurrence.6
The impetus for development of the CAPRA score initially was better differentiation among patients who would be classified as intermediate- or high-risk under the 3-level classification system published by D'Amico et al.10 Although the CAPRA score did indeed demonstrate better discrimination of intermediate- and high-risk patients in the original study,6 the CAPRA scoring system also offers improved resolution of risk at low-risk levels. A patient classified as low-risk in the D'Amico classification (PSA < 10 ng/mL, clinical classification ≤T2a, Gleason score ≤ 6) could have a CAPRA score ranging from 0 to 3 depending on his age, percentage of positive cores, and PSA level. The current analysis confirms an age of <50 years as a favorable prognostic factor, as has been demonstrated previously both in SEARCH and in other prostatectomy cohorts.11–13
As in the development study, we combined CAPRA 0 and 1 as the reference lowest risk group for recurrence-free survival and proportional hazards analyses, because the low number of CAPRA 0 patients (who must be aged <50 years at diagnosis and have all other low-risk features). However, none of the 18 patients with CAPRA 0 in this series, and only 1 of the 18 in the original CaPSURE series6 recurred, supporting the maintenance of a separate 0 score. These findings should be useful for counseling patients at low risk of recurrence concerning treatment options at diagnosis. Also of note, this study confirms that primary Gleason pattern 4 tumors have significantly worse outcomes than those with secondary pattern 4.14, 15 We have noted that CAPRA scores may be categorized to indicate risk as low (CAPRA Scores 0–2), intermediate (CAPRA 3–5), and high (CAPRA 6–10) (Fig. 2). Although the instrument does work well as a 10-point scale, for some applications 3-level risk grouping may be more appropriate, and this approach does identify an intermediate-risk group more clearly separated from the high-risk group than does the D'Amico classification.16
External validation of any novel risk-prediction instrument is a crucial step in development of the instrument. Validation of the CAPRA score in the SEARCH database is particularly useful as patients in the SEARCH registry are notably different as a cohort than the CaPSURE patients whose data was used to develop the score, and, therefore, SEARCH offers highly valuable complementary data. Compared with the general United States population, CaPSURE patients are relatively well-educated, wealthy, and well-insured; they are also relatively homogeneous ethnically.17 SEARCH, by contrast, is a database of patients at equal-access medical centers and includes patients of greater sociodemographic diversity than most other extant prostate cancer databases. In this study, for example, 41% of patients were non-Caucasian, compared with 12% of patients in the original development series.6
SEARCH patients overall also presented with higher risk factors at diagnosis than CaPSURE patients. Although in this analysis, the distribution of CAPRA scores is concentrated among low scores, as would be expected from an RP series, the full range of 0–10 CAPRA scores are represented. As in the CaPSURE series, there are very few patients who have multiple adverse-risk features who underwent RP monotherapy; only 4.5% of patients were in the CAPRA 7–10 group. However, whereas in the CaPSURE series, <11% of patients had scores over 4,6 among SEARCH patients this group accounted for more than 18% of the patients, providing better analysis of the performance of the CAPRA score at intermediate levels of risk in particular. The likelihood of recurrence-free survival at 3 and 5 years at each CAPRA score was similar in this validation analysis to those calculated in the original development study.6
Two aspects of exclusion criteria for calculation of the CAPRA score bear reemphasis. Patients with a PSA of <2 ng/mL at diagnosis are a small proportion of RP patients (1.5% of the RP patients in the original development study, 3.7% of the RP patients in the present validation study) with unusually indolent tumors. Establishing specific score points for these patients would complicate the index significantly. The goal of assigning the “typical” low-risk prostate cancer a score of 0 or 1 was an important aspect of the desired simplicity of the index, for which reason we decided originally to exclude patients with a PSA of <2 ng/mL.6 Although it is true that some patients harbor high-grade tumors that produce little if any PSA, these tumors are not typical, and such patients likely cannot be accurately staged with standard prognostic systems. We likewise found in developing the index that including patients from whom fewer than 6 cores had been taken at diagnostic biopsy (12.3% in the SEARCH cohort) markedly reduced the accuracy of the score. Because in contemporary practice performing less than a sextant biopsy is quite uncommon, except in suspected advanced disease, we decided to require at least 6 cores for calculation of the percentage of cores variable.
This study does have limitations. First, 1 of the SEARCH sites (San Francisco Veterans Affairs Medical Center) has also contributed patients to CaPSURE. However, there were only 8 patients from this site included in the original CAPRA development study.6 This minor overlap between cohorts, otherwise comprising more than 1000 patients each, should cause minimal bias in the validation analysis. Second, although percentage of cores found to be positive during a biopsy is a well-established prognostic factor, recently incorporated into an updated version of the Kattan nomogram18 and also now associated with metastatic progression and mortality as well as biochemical progression,19 it may not be the best measure of tumor burden available from biopsy data, at least in part because the significance of a given percentage of positive cores may vary with the number of cores taken. Indeed, a previous SEARCH study found that the total percentage of biopsy tissue involved was a better predictor of pathologic and biochemical outcomes.20 Total percentage of biopsy tissue, however, is not yet collected in CaPSURE, and, therefore could not be considered for inclusion in the original design of the CAPRA score; and the score was designed to be applicable to patients who had had either a sextant or extended-template biopsy.
Third, the CAPRA score was developed by using data from patients diagnosed between 1992 and 2001, whereas this validation study included patients diagnosed between 1988 and 2004. Changes over time in disease presentation—including downward stage migration and changes in Gleason score assignment—may affect the relative importance of variables in the model, and pathologic and biochemical outcomes may be affected by changes over time in surgical techniques, such as extent of lymphadenectomy. These trends highlight the importance of additional future validation studies. We are reassured, however, that the CAPRA score performed best among more recently diagnosed patients as assessed by the c-index. Measures of pretreatment PSA kinetics, such as doubling time and velocity, are increasingly recognized as adding significant prognostic information over a static PSA value.21, 22 No consensus yet exists, however, on the best measure of favorable vs. unfavorable PSA kinetics. Moreover, men may proceed to treatment with only a single elevated PSA value followed by a positive biopsy, and to include measures of kinetics in the model would exclude its applicability to patients being counseled with only a single available PSA level.
Finally, and perhaps most importantly, biochemical recurrence may not be an accurate predictor of clinical metastases or cancer-specific and overall survival.23 These latter outcomes are difficult to assess because of the need for prolonged follow-up between biochemical recurrence and clinical progression and the heterogeneity of secondary treatment modalities and timing among patients experiencing recurrence by biochemical criteria. However, recent studies have confirmed that time to biochemical recurrence is directly associated with increased risk of development of metastases24 and with time to cause-specific mortality.25 Moreover, the strong association between CAPRA score and pathologic outcomes is reassuring; advanced pathologic stage is known to predict both local and metastatic prostate cancer progression.26
The median follow-up in this study is relatively short, and we certainly plan to assess long-term clinical outcomes in both the CaPSURE and SEARCH databases as longer follow-up accumulates in these cohorts; we will examine clinical metastases and cancer-specific mortality as more patients meet these endpoints. We will also study postoperative PSA kinetics (e.g., doubling time) as a possible surrogate endpoint,25, 27, 28 although a consensus on the best definition of adverse kinetics does not yet exist, and this surrogate endpoint is not yet standard either in clinical practice or in nomogram development.18 We also plan to conduct studies of the CAPRA score's performance among other cohorts of patients undergoing other treatment types (e.g., radiation therapy). This study demonstrates the power of successful cooperation among large national prostate cancer disease registries. We hope that similar collaborations will continue to yield novel insights into prostate cancer biology, outcomes, and management strategies.
In the future we anticipate that genetic and/or molecular markers will emerge that will be sufficiently well-validated and generally available to include in risk assessment strategies applicable to the general community. In the interim, instruments based on clinical variables play a crucial role in helping to guide treatment decisions. We emphasize that no nomogram or scoring system can replace individualized clinician-patient decision-making, which must account for a given man's life expectancy, utilities for health-related quality-of-life outcomes, and treatment preferences. Nonetheless, given successful validation of the UCSF-CAPRA in the large, sociodemographically diverse, multiinstitutional SEARCH database, we hope that this powerful and straightforward instrument will prove to be a valuable tool for facilitating risk classification, both in clinical decision-making and in future research on novel management strategies for men with prostate cancer.
Supported by the National Institutes of Health (NIH) Specialized Programs of Research Excellence Grant P50 C89520 (PRC); Department of Veterans Affairs, NIH R01CA100938 (WJA); NIH Specialized Programs of Research Excellence Grant P50 CA92131-01A1 (WJA); the Georgia Cancer Coalition (MKT); Center for Prostate Disease Research (CPDR) grant from the United States Army Medical Research and Materiel Command (CLA); the Department of Defense, Prostate Cancer Research Program, PC030666 (SJF); and the American Foundation for Urological Disease/American Urological Association Education and Research Scholarship Award and the Prostate Cancer Foundation (SJF). Views and opinions of, and endorsements by the author(s) do not reflect those of the US Army or the Department of Defense.