Serum receptor activator of nuclear factor κB ligand (RANKL) levels predict biochemical recurrence in patients undergoing radical prostatectomy



What's known on the subject? and What does the study add?

  • There is increasing evidence that the receptor activator of nuclear factor κB ligand (RANKL) pathway not only contributes to the development of bone metastases, but also influences tumour biology in earlier stages of cancer.
  • The study shows that preoperative serum levels of RANKL and its inhibitor osteoprotegerin (OPG) have a prognostic impact in patients undergoing radical prostatectomy for clinically localized prostate cancer. Both high levels of RANKL and a higher RANKL/OPG ratio are independent predictors of early biochemical recurrence in these patients.


  • To assess the prognostic impact of proteins of the receptor activator of nuclear factor κB (RANKL) pathway in serum samples from patients undergoing radical prostatectomy.

Patients and Methods

  • We retrospectively determined soluble RANKL (sRANKL) and osteoprotegerin (OPG) by ELISA in serum samples of 178 patients undergoing radical prostatectomy between 2004 and 2006.
  • Clinical and patient follow-up data were analysed using the Wilcoxon–Mann–Whitney test, the Kaplan–Maier method, and single variable or multifactorial Cox proportional hazards analysis.


  • Higher serum sRANKL levels (P = 0.01), lower serum OPG levels (P = 0.01) and a higher sRANKL/OPG ratio (P = 0.004) were significant risk factors for biochemical recurrence (BCR).
  • In multifactorial analysis, adjusted for the common risk factors for BCR, sRANKL and sRANKL/OPG ratio were confirmed as independent prognostic factors.
  • Neither sRANKL nor OPG showed a clear association with histopathological factors such as pT stage, pN Gleason score or resection margin status, nor were they associated with prostate-specific antigen level.


  • Greater activity of the RANKL pathway in the serum of patients with prostate cancer undergoing radical prostatectomy is a risk factor for BCR.
  • The RANKL pathway seems to contribute to the biological behaviour of prostate cancer even at the organ-confined stage of the disease.

receptor activator of nuclear factor κB


soluble RANKL




biochemical recurrence


castration-resistant prostate cancer


log-normal distribution


hazard ratio


A large proportion of patients with castration-resistant prostate cancer (CRPC) develop bone metastases in the course of the disease, leading to an increased risk of fractures and an impaired prognosis [1]. The development of bone metastases is linked to several signalling pathways mediating the interaction between bone marrow, bone and cancer cells [2]. The receptor activator of NF-kB ligand (RANKL) has been identified as the main mediator of prostate-cancer-induced bone changes by increasing the absorption of bone matrix. RANKL, which exists in a membrane-bound and soluble form, is expressed by osteoblasts and mesenchymal stromal cells [3]. Recently, osteocytes were identified as one of the main sources of RANKL [4]. The binding of RANKL to its receptor, RANK, on osteoclast precursors promotes the differentiation of these cells to mature osteoclasts [5]. The activity of RANKL is regulated by osteoprotegerin (OPG), a so-called decoy receptor, which is expressed by osteoblasts and prevents binding of RANKL and its receptor, RANK. Although evidence exists from other malignancies that the RANKL pathway may be of major importance even in earlier stages of cancer, the role of this pathway in non-metastatic prostate cancer has not yet been identified [6]. RANK and RANKL have been shown also to be expressed by prostate cancer cells and are assumed to contribute to the homing of these cells to the bone marrow [7, 8]. A specific inhibition of RANKL has been shown to prevent the development of metastases in preclinical animal models of prostate cancer [9]. After its proven efficacy in prolonging the time to skeletal-related events in metastatic prostate cancer [10], a large phase III trial showed that the monoclonal antibody denosumab also delays the onset of bone metastases in patients with CRPC [11]. We recently showed that proteins of this pathway are altered in the serum and bone marrow samples of patients with prostate cancer compared with patients with BPH [12]. As the prognostic relevance of these changes remains unanswered, we performed a study to assess retrospectively the concentrations of sRANKL and OPG in frozen serum samples of patients undergoing radical prostatectomy and to evaluate the levels of these proteins with respect to patient outcome data.

Patients and Methods

Patients and Samples

Serum samples of 178 consecutive patients who underwent radical prostatectomy and bilateral pelvic lymphadenectomy (performed by multiple surgeons) between 2004 and 2006 in a single institution were analysed. All patients with intermediate- and high-risk prostate cancer, according to D'Amico criteria, underwent preoperative bone scans for staging [13].

Serum samples (at least 5 mL) were drawn on the day before prostatectomy into a serum tube (Monovette®, Sarstedt, Nümbrecht, Germany) and immediately (<1 h after sampling) centrifuged (10 min at 3800 revolutions/min (3060g) in a Hettich Rotanda 96 centrifuge (Hettich, Tuttlingen, Germany), frozen and stored at −80 °C. To avoid thaw–freeze cycles, all samples were kept frozen until analysis of sRANKL and OPG. The study was approved by the local ethics committee (IRB: 113/2012 B02).

Determination of Soluble RANKL and OPG

Concentrations of total sRANKL and OPG in serum samples were measured by commercially available ELISA kits according to the manufacturer's protocols (Immundiagnostik AG, Bensheim, Germany, No. K 1016 and No. KB 1011 for sRANKL and OPG, respectively). All serum samples were thawed shortly before ELISA analysis. Concentrations were determined using an Anthos 2010 reader (Anthos Mikrosysteme GmbH, Krefeld, Germany). For both proteins, wavelengths for measurement and reference were 450 and 620 nm, respectively. Reference curves were developed by the point-to-point method in lin/log diagrams.


The postoperative clinical course was assessed by questionnaires filled out by urologists in private practice who had access to all follow-up data and, in the case of increasing PSA levels, follow-up imaging. The questionnaire was used to obtain information regarding postoperative PSA levels, potential development of metastases, disease recurrence and survival (cancer-specific and overall survival). After radical prostatectomy, patient follow-up included PSA measurements and clinical monitoring according to the European Association of Urology guidelines [14]. Two consecutive values of PSA >0.2 μg/L represent the international consensus defining recurrent cancer [15]. Patients not reaching a nadir after radical prostatectomy of <0.1 μg/L were excluded from biochemical recurrence (BCR) analysis. The development of metastases was detected by standard imaging methods, including bone scan and CT.

Statistical Analysis

Concentrations of sRANKL and OPG were compared with regard to clinical data using the Wilcoxon–Mann–Whitney test. For further analysis, sRANKL, OPG and the sRANKL/OPG ratio were logarithmically transformed as both OPG and sRANKL had a positively skewed distribution within the population [16, 17]. Additionally, the pathway proteins (sRANKL, OPG) were calculated as dichotomized variables according to their median values. Time-to-event data were analysed using the Kaplan–Meier method and Cox single variable or multifactorial proportional hazards analysis. JMP 7.0 (SAS Inc., Cary, NC, USA) was used for all analyses, and a P value <0.05 was considered to indicate statistical significance.


Patient Characteristics

The median (range) age of the patients at time of radical prostatectomy was 63 (43–76) years. The median Gleason score was 7 (5–9). The median preoperative PSA level was 7.1 μg/L (0.4–55.8). Organ-confined (pT <3) and locally advanced disease (pT ≥3) were present in 134 (75.3%) and 44 (24.7%) patients, respectively. Resection margin status was positive in 34 patients (19.1%). Ten patients (5.6%) had nodal-positive disease (pN1).

Preoperative serum sRANKL and OPG levels

The median (range) levels of sRANKL and OPG in serum samples were 11 385.0 (15.6–5 499 296.0) pg/mL and 75.2 (20.2–211.2) pg/mL. Comparisons of sRANKL and OPG levels according to standard histopathological variables are shown in Table 1.

Table 1. Patient characteristics and comparison of preoperative concentrations of sRANKL, OPG and ratio of log-normal distribution sRANKL/log-normal distribution OPG.
VariablensRANKL (median; range)POPG (median; range)PRatio Ln sRANKL/Ln OPG (median; range)P
  1. Ln, log-normal distribution.
Tumour stage       
pT ≥34411 383.0 (15.6; 1 013 365.0)0.8175.8 (20.2; 127.2)0.702.14 (0.59; 3.98)0.81
pT <313411 319.3 (15.6; 5 499 296.0) 75.1 (32.2; 211.2) 2.14 (0.59; 3.83) 
Gleason score       
≥71028 279.8; (15.6; 2 362 798.0)0.7075.8 (20.2; 127.2)0.842.08 (0.57; 3.98)0.11
<77611 446.0 (15.6; 5 499 296.0) 74.0 (32.6; 211.2) 2.24 (0.60; 3.70) 
Preoperative PSA       
≥10 μg/L4515 766.9 (15.6; 1 013 365.0)0.3573.4 (29.6; 104.6)0.392.23 (0.60; 3.98)0.30
<10 μg/L13310 310.1 (15.6; 5 499 296.0) 75.8 (20.2; 211.2) 2.11 (0.57; 3.70) 
Resection margin status       
R13410 254.2 (15.6; 2 362 798.0)0.9679.0 (29.6; 127.2)0.122.12 (0.57; 3.98)0.60
R014411 392.5 (15.6; 5 499 296.0) 74.2 (20.2; 211.2) 2.15 (0.57; 3.70) 
Nodal status       
pN1109 029.3 (15.6; 159 779.0)0.8474.7 (47.6; 101.6)0.772.07 (0.59; 3.10)0.98
pN016811 383.8 (15.6; 5 499 296.0) 75.2 (20.2; 211.2) 2.16 (0.57; 3.98) 
>63 years8013 383.1; (15.6; 2 362 798.0)0.7578.2 (36; 211.2)0.012.18 (0.57; 3.40)0.84
≤63 years989 629.7; (15.6; 5 499 296.0) 73.4 (20.2; 127.2) 2.11 (0.57; 3.98) 

Clinical Course

Follow-up data were available from 164 of 178 patients (92.1%). Seven patients were excluded from follow-up analysis as they did not reach a post-prostatectomy PSA nadir of <0.1 ng/mL (of those, two were node-positive, three had a positive resection margin and five had locally advanced disease (≥pT3a)). The median follow-up was 74.5 (54–86) months. Of the remaining 157 patients, 34 (21.7%) had BCR, 10 (6.3%) developed metastasis and three (1.9%) died from prostate cancer within the observational period. Four patients died from causes other than prostate cancer. The 5-year metastasis-free survival rate was 97.0% and the 5-year cancer-specific survival rate was 98.0%.

Association of sRANKL and OPG with Standard Clinical and Pathological Variables

Neither sRANKL nor OPG showed any clear association with standard clinical and pathological variables such as pT, pN, Gleason score, preoperative PSA level or age. OPG concentrations were significantly higher in patients >63 years compared with those ≤63 years (P = 0.01, Table 1).

sRANKL and sRANKL/OPG Ratio as Risk Factors for BCR

The 5-year BCR-free survival rate in patients with sRANKL concentrations above the median was 74.1 vs 89.8% in patients in the lower 50% of sRANKL concentrations (log-rank test, P = 0.02, Fig. 1A). The 5-year BCR-free survival in patients with OPG concentrations above and below the median were 85.2% vs 78.9% (P = 0.34, Fig. 1B). In patients with an sRANKL/OPG ratio below and above the median, 5-year BCR-free survival rates were 89.5 and 74.7%, respectively (P = 0.01, Fig. 1C).

Figure 1.

A, Kaplan–Meier analysis showing shorter time to BCR in patients with sRANKL serum levels above the median. B, Kaplan–Meier analysis showing no significant differences regarding time to BCR in patients with OPG serum levels above and below the median. C, Kaplan–Meier analysis showing shorter time to BCR in patients with serum sRANKL/OPG ratios above the median.

In single variable Cox proportional hazards analysis, higher sRANKL concentrations as well as a higher sRANKL/OPG ratio were significant risk factors for BCR, both as undichotomized continuous variables after normalization and as split at the median value (Table 2). A lower continuously entered OPG was associated with a higher risk of BCR, whereas dichotomized OPG was not found to be significant.

Table 2. Single variable Cox proportional hazards analyses for BCR risk factors.
VariableHR (95% CI)P
  1. Ln, log-normal distribution.
Tumour stage (pT ≥3 vs <3)5.69 (2.91; 11.33)<0.001
Tumour grade (Gleason ≥7 vs <7)2.76 (1.22; 5.68)<0.001
Preoperative PSA (PSA ≥10 vs <10 ng/L)2.82 (1.43; 5.49)0.003
Resection margin status (R1 vs R0)2.64 (1.27; 5.22)0.01
Nodal stage (pN >0 vs pN0)2.00 (0.48; 5.63)0.29
Age (≥63 vs <63 years)1.00 (0.51; 1.96)0.97
Serum sRANKL (sRANKL ≥ median vs < median)2.35 (1.19; 4.85)0.01
Serum OPG (OPG ≥ median vs < median)0.72 (0.35; 1.41)0.23
Ratio sRANKL/OPG (ratio Ln sRANKL/Ln OPG ≥ median vs < median)2.34 (1.17; 4.97)0.02
Ln sRANKL as continuous variable1.10 (1.01;1.20)0.01
Ln OPG as continuous variable0.27 (0.12;0.74)0.01
Ratio sRANKL/OPG as continuous variable (Ln sRANKL/Ln OPG)1.70 (1.17;2.52)0.004

With respect to the overall sample size and the limited number of 34 events for BCR, six multifactorial models with inclusion of a maximum of three variables each, including known clinical and pathological risk factors for BCR and proteins of the RANKL pathway, were performed (Table 3A, 3B). Regardless of any two other factors included in the model, higher sRANKL concentrations and elevated higher sRANKL/OPG ratio were shown as independent risk factors for BCR throughout all models. Notably, this was the case when sRANKL or the sRANKL/OPG ratio were entered as a dichotomized variable as well as a continuous and logarithmically transformed variable into the model (Table 3A, 3B), but OPG, whether dichotomized or undichotomized, did not show an independent prognostic effect in multivariable analysis (data not shown). As dichotomization of PSA levels as a variable might not sufficiently address this risk factor for BCR, models 1, 3 and 6 (Table 3A, 3B) were also performed using quartiles of PSA concentrations as a variable. Both sRANKL and sRANKL/OPG ratio were still significant risk factors using models 1, 3 and 6 with PSA concentration quartiles as variables (all P ≤ 0.02).

Table 3A. Multifactorial Cox proportional hazards analyses for BCR risk factors including (A) sRANKL and (B) sRANKL/OPG ratio.
VariableModel 1Model 2Model 3Model 4Model 5Model 6
HR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)P
  1. Six different models including different combinations of risk factors were applied. *HRs and P values of other variables in the model not given in the Table. Ln, log-normal distribution.
pT ≥3 vs <33.60 (1.72; 8.00)<0.0015.91; 2.89; 12.32<0.0016.59 (3.03; 14.47)<0.001      
Gleason score ≥7 vs <74.17 (1.40; 15.20)0.009    6.98 (2.68; 23.82)<0.0017.05 (2.65; 24.37)<0.001  
PSA ≥10 vs <10 μg/L  1.63 (0.79; 3.33)0.18  1.79 (0.88; 3.58)0.10  2.57 (1.27; 5.06)0.008
Resection margin status R1 vs R0    1.04 (0.46; 2.30)0.91  1.42 (0.67; 2.92)0.342.64; (1.26; 5.25)0.01
sRANKL, ≥ median vs < median2.65 (1.32; 5.59)0.0052.32 (1.15; 4.90)0.012.43 (1.21; 5.15)0.012.43 (1.21; 5.15)0.012.63 (1.31; 5.54)<0.0012.08 (1.04; 4.35)0.03
Ln sRANKL, continuous variable*1.14 (1.04; 1.26)0.0051.11 (1.01; 1.23)0.031.12 (1.02; 1.24)0.011.13 (1.03;1.22)0.0051.14 (1.04; 1.26)0.0041.10 (1.01; 1.20)0.05
Table 3B. Multifactorial Cox proportional hazards analyses for BCR risk factors including (A) sRANKL and (B) sRANKL/OPG ratio.
VariableModel 1Model 2Model 3Model 4Model 5Model 6
HR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)PHR (95% CI)P
  1. Six different models including different combinations of risk factors were applied. *HRs and P values of other variables in the model not given in the Table. Ln, log-normal distribution.
pT ≥3 vs <33.67 (1.75; 8.15)<0.0015.98 (2.93; 12.49)<0.0016.43 (3.00;13.98)<0.001      
Gleason ≥7 vs <74.14 (1.39; 15.08)0.009    6.95 (2.68; 23.72)<0.0017.01 (2.63; 24.22)<0.001  
PSA >10 vs ≤10 ng/L  1.64 (0.17; 3.35)0.17  1.84 (0.90; 3.67)0.08  2.59 (1.29; 5.12)0.008
R1 vs R0    1.16 (0.52; 2.50)0.70  1.44 (0.68; 2.96)0.342.66; (1.27; 5.30)0.01
Ratio Ln sRANKL/Ln OPG ≥ median vs < median2.84 (1.40; 6.08)0.0032.53 (1.25; 5.44)0.0092.64 (1.31; 5.65)0.0062.79 (1.38; 5.98)0.0032.78 (1.38; 5.96)0.0042.26; (1.12; 4.83)0.02
Ratio Ln sRANKL/Ln OPG (continuous variable)*1.78 (1.23; 2.62)0.0081.63 (1.11; 2.44)0.011.70 (1.17; 2.52)0.0041.78 (1.23; 2.61)0.0011.83 (1.26; 2.69)0.0011.53 (1.06; 2.25)0.02


Within the last decade, the RANKL pathway has been discovered to be a central regulator of bone absorption, osteoclast differentiation and cancer-induced bone disease [3]. In the context of prostate cancer, this pathway has become a therapeutic target for both delaying skeletal-related events in patients with bone metastases [10] and improving bone mineral density with fracture prevention in patients with cancer-treatment-induced bone loss [18]. Recently, a large phase III trial, which showed that inhibition of RANKL by the fully human monoclonal antibody denosumab can prevent the development of bone metastases in patients with CRPC, indicated that there was a benefit of targeting the RANKL pathway in earlier stages of the disease [11]. Several studies have identified the role of the RANKL pathway in the cancer biology of primary prostate cancer cells [7, 19, 20]. To date, only limited data exist on changes in the RANKL pathway in patients with clinically localized prostate cancer. We recently showed that changes in the RANKL pathway are present in serum and bone marrow samples of patients with localized prostate cancer [12], but the prognostic relevance of systemic changes in the RANKL system has not yet been assessed. In the present study, we aimed to investigate the prognostic impact of serum concentrations of sRANKL and OPG in a typical cohort of men undergoing radical prostatectomy.

Firstly, the results of the present study show that serum concentrations of neither sRANKL nor OPG are associated with the commonly used factors for aggressive disease, such as Gleason score, TNM stage or preoperative PSA levels. Secondly, the results clearly indicate that a higher activation status of this pathway in serum samples from patients undergoing radical prostatectomy is an independent risk factor for early BCR. Both higher sRANKL concentrations and a higher sRANKL/OPG ratio predicted BCR. This observation remains durable in both single-factor and multifactorial analysis, with both a dichotomized and an undichotomized predictor variable strengthening the prognostic relevance. The serum concentrations of sRANKL and OPG may therefore be used as risk assessment tools at the time of radical prostatectomy, serving as an additional biomarker for tumour aggressiveness. This may help us to further close the gap between the classic risk factors and the true outcomes. Moreover, inhibition of the RANKL pathway could be feasible in men with earlier stages of PC.

Several hypotheses about how RANKL changes might affect the biological behaviour of prostate cancer are possible. Firstly, some evidence exists that primary tumours are capable of expressing RANK [7, 20]: whereas Chen et al. [7] observed a significantly higher RANK expression in bone metastases compared with in primary tumours, Santini et al. [20] showed that 55% of prostate cancer samples express RANK with no difference between primary tumours and bone metastases. The latter authors were also able to prove the prognostic impact of RANK expression in other solid malignancies [21]. A greater binding of RANKL to RANK of prostate cancer cells might result in activation of downstream pathways, particularly the nuclear factor kappa B (NFkB) pathway. This pathway is known to be a promoter of tumour cell proliferation, angiogenesis, tumour cell migration and invasion [22-24]. Its activation status in prostate cancer cells has been shown to be associated with disease stage and time to BCR [25-27]; however, to evaluate a potential relationship between RANKL serum concentrations and status of NFkB signalling in prostate cancer, NFkB activation analysis in prostatectomy specimens is required. Although nuclear staining of the NFkB subunits p50 and p65 is often used as a surrogate measure of NFkB activation, DNA-binding assays are more specific for the quantification of NFkB activation but have not yet been evaluated in paraffin-embedded tissue [26, 28].

Another theory on how RANKL activation might influence the biological behaviour of prostate cancer cells is that increased activation of osteoclasts may cause the release of prostate-cancer-promoting cytokines and matrix proteins from the bone into the serum. These factors include TGFβ, platelet-derived growth factor, IGFs, fibroblast growth factors and bone morphogenic proteins, which have been shown to influence prostate cancer progression [29-31]. Although it has been shown that bone mineral density is lower in patients with prostate cancer, even before bone metastases have developed or androgen deprivation therapy is initiated [32], little is known about the role of greater bone turnover for patients with non-metastatic prostate cancer. Interleukin-6, a promoter of bone absorption, has been shown to have prognostic value in patients with localized prostate cancer [33, 34]. Preclinical studies have shown that the induction of osteoporosis in animal models by induced vitamin D deficiency or application of parathyroid hormone promotes disease progression and the development of metastases [35, 36]. As BCR often precedes the development of metastatic disease, BCR might also be attributable to micrometastases in the bone or disseminated tumour cells activated by increased activity of RANKL. Men featuring micrometastatic disease are precisely those for whom salvage radiation therapy fails; however, a longer follow-up would be required to evaluate the respective proportion of patients with BCR that would actually develop bone metastases.

The findings of the present study have several clinical implications. Firstly, although we do not know the molecular mechanism by which prostate cancer biology is affected, the present results promote further research in larger cohorts on serum sRANKL and OPG as a prognostic marker in prostate cancer. Many urologists use established nomograms to calculate the risk of recurrence in patients with prostate cancer undergoing radical prostatectomy. The most broadly used risk prediction tools such as the Kattan nomogram [37], the Stephenson nomogram [38] and the CAPRA or CAPRA-S score [39] include pathological and clinical risk factors known to be associated with BCR and have been externally validated [40]. Although the multivariate analysis used in the present study includes several factors which are also taken into account by currently used tools for risk prediction (such as T-stage, N-stage, resection margin status, Gleason score), we did not calculate whether these tools would perform better if sRANKL and OPG concentrations were included as risk factors. Before sRANKL and OPG concentrations can become broadly used as variables by which to calculate prognosis, studies including larger numbers of patients should directly compare the concordance indices of models that include the risk factors of established nomograms with models that include sRANKL and OPG values as parameters. This is considered to be a minimum requirement for a new biomarker before being used in clinical practice; however, even if the inclusion of sRANKL and OPG would not improve the performance of established risk calculation tools, these variables might be interesting for particular subsets of patients. In patients at high risk for the development of bone metastases in the course of disease, bone metabolism parameters might be helpful to identify patients who would potentially benefit from the application of antiresorptive drugs, but no tissue or serum variables for this purpose have yet been identified in prospective trials. The observations made in the present study encourage further research on the potential role of RANKL inhibition in patients with prostate cancer in a clinically non-metastatic setting. Although the US Food and Drug Administration declined the approval of denosumab for the prevention of bone metastases in patients with CRPC, subgroups of patients might exist with increased activity of RANKL proteins in the serum which might derive particular benefit from RANKL inhibition [41].

The present study has limitations inherent to all retrospective studies. In addition, the number of eligible patients was limited, and the observed number of events did not allow the use of more complex multifactorial models; therefore, we cannot be sure that the markers assessed will remain independent risk factors in larger models with a greater set of variables. As a result of the low number of patients who developed bone metastases, the potential prognostic effect of serum factors of the RANKL pathway for the development of bone metastases could not be reliably assessed.

In conclusion, this is the first study assessing the prognostic role of serum proteins of the RANKL pathway in patients with localized prostate cancer. Greater activity of the pathway, indicated by a high sRANKL concentration and a higher sRANKL/OPG ratio, is associated with a higher risk of BCR in patients with prostate cancer undergoing radical prostatectomy. The incorporation of sRANKL and OPG values as risk parameters for BCR might improve the diagnostic accuracy of nomograms currently used for outcome prediction in patients with clinically non-metastatic prostate cancer.

Conflicts of Interest

Tilman Todenhöfer and Arnulf Stenzl have acted as clinical consultants for Amgen and Novartis. The other authors have nothing to disclose.