SEARCH

SEARCH BY CITATION

Keywords:

  • neural network;
  • logistic regression;
  • prognosis;
  • bladder cancer

Abstract

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES

OBJECTIVE

To compare the prognostic performance of an artificial neural network (ANN) with that of standard logistic regression (LR), in patients undergoing radical cystectomy for bladder cancer.

PATIENTS AND METHODS

From February 1982 to February 1994, 369 evaluable patients with non-metastatic bladder cancer had pelvic lymph node dissection and radical cystectomy for either stage Ta-T1 (any grade) tumour not responding to intravesical therapy, with or with no carcinoma in situ, or stage T2–T4 tumour. LR analysis based on 12 variables was used to identify predictors of overall 5-year survival, and the ANN model was developed to predict the same outcome. The LR analysis, based on statistically significant predictors, and the ANN model were the compared for their accuracy in predicting survival.

RESULTS

The median age of the patients was 63 years, and overall 201 of them died. The tumour stage and nodal involvement (both P < 0.001) were the only statistically independent predictors of overall 5-year survival on LR analysis. Based on these variables, LR had a sensitivity and specificity for predicting survival of 68.4% and 82.8%, respectively; corresponding values for the ANN were 62.7% and 86.1%. For LR and ANN, the positive predictive values were 78.6% and 76.2%, and the negative predictive values were 73.9% and 76.5%, respectively. The index of diagnostic accuracy was 75.9% for LR and 76.4% for ANN.

CONCLUSIONS

The ANN accurately predicted the survival of patients undergoing radical cystectomy for bladder cancer and had a prognostic performance comparable with that of LR. As ANNs are based on easy-to-use software that can identify nonlinear interactions between variables, they might become the preferred tool for predicting outcome.


Abbreviations
LR

logistic regression

ANN

artificial neural network

TP

true positive

TN

true negative

FP

false positive

FN

false negative

(P)(N)PV

(positive) (negative) predictive value.

INTRODUCTION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES

TCC of the bladder is the second most common malignancy of the genitourinary tract, the second most common cause of death among all genitourinary tumours, and the 11th most common cancer worldwide, accounting for 3–4% of all malignancies [1]. Despite the urgent need for prognostic predictors for this cancer, to better select optimum adjuvant treatments, stratify patients in controlled studies and compare classic and new prognostic factors, to date predicting the prognosis for these patients remains a challenge for the urologist. Several clinicopathological factors have been suggested as prognosticators [2–6], even though the independent relevance of each factor remains controversial [5]. Furthermore, by using these factors it is still impossible to predict the outcome for an individual patient.

The statistical interpretation of medical data represents an alternative solution to the problem of accurately predicting the behaviour of cancer. Statistical methods based on logistic regression (LR) are traditionally used for this purpose. Unfortunately, the complex interactions within medical data do not always allow the use of these methods, which are, moreover, difficult to use by clinicians, because they require statistical knowledge and formal training. To overcome the shortcomings of conventional statistical methods, a new statistical approach, the artificial neural network (ANN), was recently applied to medical analysis [7–10]. ANNs are based on software that is easy to use, logical and fast, and that imitates low-level brain function to ‘learn’ from data used as an example (‘training dataset’) and make intelligent predictions given new, limited data.

In medicine ANNs have been applied to several problems, e.g. diagnosis, prognosis and diagnostic tests, or the interpretation of imaging techniques [7]. ANNs have a high specificity and sensitivity for classifying acute disease, e.g. acute myocardial infarction and pulmonary embolism [7]. In oncological urology ANNs have been investigated specifically to resolve the diagnostic, staging and prognostic problems of prostate cancer, with very promising results [11–24]. Conversely, studies exploring the use of ANNs for kidney, testicle and bladder cancers are uncommon [25–33] (Table 1).

Table 1.  Applications of ANNs in oncological urology
TumourField of applicationReference
KidneyDiagnostic aid[9]
BladderDiagnostic aid[31]
Determination of prognosis[32,33]
TesticleStaging aid[30]
Determination of prognosisInt GC Coll Group, 1998
ProstateDiagnostic aid[12–16]
Staging aid[8,17–23]
Determination of prognosis[24]

We aimed to develop a ‘feed-forward multilayer’ ANN model capable of estimating the overall 5-year survival in patients undergoing radical cystectomy for bladder cancer; this model, named Netcyst, was compared to LR.

PATIENTS AND METHODS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES

The study was based on a comprehensive database that included clinical and pathological data, prospectively gathered and concerning all patients undergoing radical cystectomy. From February 1982 to February 1994, 535 patients with bladder cancer had pelvic lymph node dissection, radical cystectomy and urinary diversion, with intent to cure. The indications for surgery were either stage Ta-T1 (any grade) tumour not responding to intravesical therapy, with or with no concomitant carcinoma in situ (Tis), or muscle-infiltrating tumour (stage T2); no patient had distant metastases (all M0).

We excluded patients treated with chemotherapy or radiation therapy (124) and patients not having a formal bilateral node dissection (42). Consequently, the final study population comprised 369 patients. The stage and grade were assigned according to 1997 TNM system and WHO classification, respectively [34,35]. Cystectomy and lymphadenectomy specimens were analysed and reviewed according to a standard protocol by a reference pathologist, with particular attention to lymphovascular and perineural invasion, depth of invasion into the bladder wall, prostate invasion and nodal status. All patients were followed at 3 and 6 months after surgery, and thereafter every 6 months until progression or death. Chest X-ray, abdominal ultrasonography, total blood count and serum biochemical investigations were carried out at every follow-up visit, with CT at 6 and 12 months and yearly thereafter. A bone scan was taken when bone metastasis were suspected. Patients with either pathologically high-risk tumours (≥ pT3, lymph nodes metastasis) or relapse were treated with systemic chemotherapy; radiotherapy was used as a palliative treatment.

The overall survival at 5 years after radical cystectomy was the dependent variable and the endpoint of the LR analyses. The independent variables (covariates) evaluated were: gender (male/female); age at surgery (<70/ > 70 years); previous bladder cancer (yes/no); vascular invasion (yes/no); lymphatic invasion (yes/no); perineural invasion (yes/no); prostate infiltration (yes/no); concomitant prostate adenocarcinoma (yes/no); neoplasm of the upper urinary tract (yes/no); pT stage (T0/T1/Tis/T2/T3a/T3b/T4); pathological N stage (N0/N1/N2/N3); and WHO grade (Gx/G1/G2/G3). The threshold criterion for entry into the subset was statistical significance at P < 0.05; all P values were two-sided. Cox’s proportional-hazards regression curves were also used to estimate the overall survival of subgroups classified according to pT stage and N. The LR model was then used to predict survival based on statistically significant independent variables and using a threshold value of the survival probability at 5 years of 0.5 to classify each patient as dead or alive.

We developed an ANN, termed ‘Netcyst’, developed using the software ‘Neural Planner’. Netcyst has the classic architecture of feed-forward with three layers, i.e. an input layer, an output layer and one hidden layer (Fig. 1). To predict the 5-year overall survival, the patients were divided randomly into two sets, one for the learning phase (246) and one for the testing phase (123) of neural simulation. An ‘on-line back-propagation’ method was used as the learning procedure (Fig. 2), using six training sets (each of 41 patients). Initially a random weight (value) was assigned to each connection. The learning procedure for each set was interrupted when the error was minimised.

image

Figure 1. Netcyst’s structure, Netcyst has the classic architecture feed-forward with three layers: an input layer, an output layer and one hidden layer.

Download figure to PowerPoint

image

Figure 2. Diagram of the ‘on-line back propagation’ model to gradually adjust the weights of connections in the network.

Download figure to PowerPoint

When the training was completed the reliability of the network was checked using three independent test sets (each of 41 patients), randomly extracted from the 123 patients not used in the training phase. Thus, 369 patients were analysed with LR, whereas only 123 were used to test Netcyst with the actual fate of each patient (dead or alive at 5 years).

Concordance between the results of the statistical classification and the real outcome was expressed using the true positive (TP), true negative (TN), false positive (FP) and false negative (FN). We adopted standard measures to evaluate the accuracy of a diagnostic test, i.e. sensitivity = TP/(TN + FN); specificity = TN/(TN + FP); positive predictive value (PPV) = TP/(TP + FP); negative PV (NPV) = TN/(TN + FN); and the concordance index (an index of diagnostic accuracy), calculated as (TP + TN)/(TP + FP + TN + FN).

RESULTS

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES

Table 2 shows the patients’ characteristics; the median (range) age was 63 (23–82) years. Organ-confined tumours (≤ pT2, pN0) were present in 174 (47.2%) of the patients. Despite negative preoperative staging in all patients, the final pathology showed positive lymph nodes in 78 (21%) patients. Most tumours were G3 (66.4%). Overall 201 (54.5%) patients died, 177 (48%) within 5 years from radical cystectomy. The median (range) follow-up was 48.5 (12–149) months. All censored patients in the survival analysis had at least 5 years of follow-up.

Table 2.  The patients’ characteristics
Independent variablesn (%)
  • *

    UICC TNM [34];

  • WHO system [35].

Gender
 Male338 (91.6)
 Female 31 (8.4)
Previous bladder cancer125 (33.9)
Vascular invasion 21 (5.7)
Lymphatic invasion 110 (29.8)
Perineural invasion 22 (6.0)
Prostate invasion 36 (9.8)
Concomitant prostate adenoma 31 (8.4)
Neoplasm of the upper urinary tract 25 (6.8)
Pathological T stage*
 T0 17 (4.6)
 T1 48 (13.0)
 Tis 46 (12.5)
 T2 67 (18.2)
 T3a 70 (19.0)
 T3b 72 (19.5)
 T4 49 (13.3)
Pathological N stage*
 N0291 (78.9)
 N1 17 (4.6)
 N2 44 (11.9)
 N3 17 (4.6)
Grading
 Gx 17 (4.6)
 G1  8 (2.2)
 G2 99 (26.8)
 G3245 (66.4)

The covariate selection process identified pN stage and pT stage as the only two variables with independent prognostic significance (P < 0.001; Table 3). In particular, N stage had a maximum prognostic value when categorized in two categories, N0 and N+ (N1 + N2 + N3) with a relative risk of ≈14. The covariate T stage had the best prognostic significance when categorized into three groups, T0 + T1 + Tis, T2 + T3a and T3b + T4.

Table 3.  The results from LR
CovariateCovariate levelCoefficient βP (Wald test)exp(β)*
  • *

    Relative risk.

Stage NN0
N12.637<0.00113.973
Stage TT0-T1-Tis
T2-T3a1.899<0.001 6.681
T3b-T40.9080.004 2.479

The observed overall survival curves calculated by the Cox proportional-hazards method for the six possible combinations of the covariates N and T are shown in Fig. 3. On analysis the relevant prognostic significance of N stage was evident, but a higher T stage implied a poorer prognosis even in patients with positive lymph nodes (small differences between the curves with a low survival rate imply important differences in relative risk).

image

Figure 3. Overall survival curves calculated with Cox’s proportional-hazards regression model, according to six possible combinations of covariates N and T.

Download figure to PowerPoint

For the performance in predicting the survival of each patient (dead or alive at 5 years) the testing phase of Netcyst gave results (with a threshold value of 0.5) for TP, TN, FP and FN of 32 (26%), 62 (50.4%), 10 (8.1%) and 19 (15.5%), respectively. The regression model gave the following classification (threshold value of 0.5): 121 (32.8%), 159 (43.1%), 33 (8.9%) and 56 (15.2%), respectively.

The comparison of the efficacy measures of the two statistical methods are shown in Table 4. Netcyst had sensitivity, specificity and accuracy comparable to those obtained with LR.

Table 4.  A comparison of the efficacy measures
Efficacy measure, %LRNetcyst
Sensitivity68.462.7
Specificity82.886.1
Positive predictive value78.676.2
Negative predictive value73.976.5
Index of diagnostic accuracy (concordance)75.976.4

DISCUSSION

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES

We developed an ANN model that could be useful for urologists to better predict the survival of patients with bladder cancer after radical surgery. The ANN model had a satisfactory prognostic performance in terms of specificity (86.1%), sensitivity (62.7%) and index of diagnostic accuracy (76.4%), and which were equivalent to those of the LR of 82.8%, 68.4% and 75.9%, respectively.

Our results agree with those reported in other urological reports supporting the satisfactory diagnostic and prognostic performance of ANNs. Most trials have been focused on prostate cancer, but occasionally ANNs were applied to diagnostic and prognostic problems of kidney, testicle or bladder tumours [9,11–33]. In these studies ANNs were an optimal analytical statistical method to analyse complex problems, e.g. distinguishing between malignant and benign cystic lesions of the kidney, or staging prostate and testicular tumours. ANNs showed equivalent and sometimes better predictive accuracy than LR. Few authors investigated the performance of an ANN model when used to predict the prognosis of bladder cancer and, to our knowledge, the present is the first study of an ANN applied to the problem of predicting survival after radical cystectomy for bladder cancer. Qureshi et al.[31] reported that a ANN was significantly more accurate than clinicians for predicting stage progression in a retrospective series of patients with T1G3 bladder cancer (overall accuracy 82% vs 40%). However, when analysing the entire Ta/T1 cohort, there was no significant difference. In another report [32] an ANN was used to predict long-term progression-free survival (negative predictive value 100%) in patients with superficial bladder cancer followed for ≥15 years, but it could not predict tumour recurrence. Catto et al.[33] confirmed that the ANN provides a powerful tool, better than traditional statistical methods (accuracy of 90% vs 77%), for predicting relapse in patients with superficial and muscle-infiltrating bladder cancer, based on clinicopathological data.

In a recent review [36], including 28 studies and based on datasets of >200 patients, the performance of both LR and ANN for prognosis and classification were compared. The ANN outperformed LR in 10 studies but it was outperformed by LR in four, and the two methods had a similar performance in the remaining 14 (Fig. 4). However, considering the eight larger studies (with samples of >5000 patients), LR and ANN had similar efficacy in seven studies, with the LR better in the remaining study. The ANN predominated only in studies with <2000 patients. The authors of this important review indicated that a publication bias in studies on ANN might jeopardize the results. ANNs represent an innovative approach and studies showing a better performance of this technique might be preferentially published [36]. Studies with negative results might only receive a favourable review if the dataset is particularly large. Nevertheless, ANNs could outperform LR when applied to small or moderate datasets.

image

Figure 4. Comparison of LR with the ANN in a review of 28 studies (modified from Sargent [36]). The black square represents the present study.

Download figure to PowerPoint

In addition to the evidence cited above, of comparable and sometimes better results of the ANN than LR, there are several theoretical advantages to the ANN; they can recognize all possible relationships that might exist between dependent (healthy/sick, dead/alive) and independent (age, gender, stage, etc.) variables. They appear to be particularly promising for identifying the complex interactions typical of medical data, especially when the primary goal is to predict the outcome (prognosis). By contrast, due to the characteristics of medical datasets, e.g. high variability, non-Gaussian or homogenous distribution and nonlinear interaction between prognostic variables, statistical methods have important shortcomings when applied to predicting the clinical outcome. The mathematical equations of regression models are very difficult to determine and often require the explicit assumption of certain relationships within the data that are not confirmed. For data analysis ANNs are more robust than statistical methods also because they can accommodate small variations in parameters, and ‘noise’.

Furthermore, regression methods are based on often complex statistical concepts, not always understood by clinicians. Conversely, the process by which ANNs analyse problems is fully automated. Commercial ANN software automatically trains, tests and predicts in one easy step, saving time and trouble in setting up the analyses. The procedure is started by the user and the software proceeds to learn the structure of the data, thus enabling it to classify the groups; thus, unlike more traditional statistical methods, ANNs require no extensive academic training or detailed statistical knowledge to be used successfully. In addition, some features of modern ANN programs, such as implementation in MS Excel, and live prediction (which automatically updates the prediction when the input data changes) make ANNs very intuitive, fast and easy to use, particularly by clinicians with no formal statistical training.

However, ANNs are considered as ‘black boxes’ because of their hidden network, which remains an obstacle to its acceptance. Moreover, they are intrinsically not comparable, and have a limited ability to explicitly identify possible causal relationships between variables. ANNs are also unable to quantify the ‘weight’ of single variables on the final outcome. Conversely, regression models are widely recognized as easy to reproduce and able to inform clinicians about the weight of each prognostic variable on the final outcome, in terms of relative risk. LR remains the method of choice when the dependent variable is dichotomous and when the primary goal is not only to predict the outcome but also to discover possible causal relationships between independent and dependent variables [37].

Presently there is no strong evidence that an ANN could replace conventional statistical methods for clinical problems. The evidence of satisfactory efficacy, at least equivalent to LR, as shown by our analysis, should be the starting point for future research in this field. We suggest that large studies are continued to find the method with the best performance for each clinical problem, and in each particular medical context.

In conclusion, we show that an ANN can be used to accurately predict the survival of patients after radical cystectomy for bladder cancer. The ANN had a prognostic accuracy comparable with that of LR. However, the ability to identify all complex nonlinear relationships between variables in medical datasets, coupled with their intuitive use and easy handling by clinicians with no specific statistical knowledge, supports the future role of ANNs as the preferred method of predicting prognosis by the practising urologist.

REFERENCES

  1. Top of page
  2. Abstract
  3. INTRODUCTION
  4. PATIENTS AND METHODS
  5. RESULTS
  6. DISCUSSION
  7. CONFLICT OF INTEREST
  8. REFERENCES