Validation of quality indicators for radical prostatectomy

Abstract

The feasibility and validity of proposed radical prostatectomy quality indicators has not been well studied. We assessed indicator availability from treating charts. We tested the convergent construct validity of a modified subset that were available from this information source by correlating them to hospital prostatectomy volume, a variable repeatedly associated with the quality of surgical care. The study population consisted of a stratified random sample of prostate cancer patients who were: (i) diagnosed between 1990 and 1998 in Ontario and (ii) treated by radical prostatectomy with curative intent within 6 months of diagnosis (n = 645). Of the 9 candidate quality indicators assessed, 4 were missing for 25–56% of study subjects and were not analyzed further. We discuss the implications of this missing information on feasibility of their use. For blood transfusions of 3 units or greater, length of hospital stay and use of non-nerve-sparing surgical technique, worse outcomes were generally apparent with decreasing hospital volume. Acute complication rates and positive surgical margin rates did not increase with decreasing hospital volume. We were able to demonstrate convergent construct validity for 3 quality indicators. Upon further validation, this readily available information may be applied to aid providers and quality councils to more effectively identify problems and guide change in the management of early prostate cancer. © 2008 Wiley-Liss, Inc.

Prostate cancer is the most commonly diagnosed cancer and the third most common cause of cancer death among Canadian men.1, 2 The use of radical prostatectomy, the standard curative surgical treatment for patients with clinically localized prostate cancer, has increased with the advent of PSA testing and the resulting increase in the incidence of resectable disease. Despite the overall benefit to survival, the surgical removal of the prostate commonly leads to early and late morbidities that can significantly affect a patient's quality of life.3–6 Furthermore, ongoing controversy over many aspects of the management of prostate cancer7, 8 has led to regional and institutional variations in treatment7, 9–14 with some evidence that these variations can lead to differences in outcome.4, 15, 16 The timely evaluation of quality of care in the surgical treatment of prostate cancer has become a priority, although there is little known about which aspects of quality should be examined.17

Quality of care research evaluates the “degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.”18 Concerns over shortcomings in the quality of medical care in light of rising healthcare expenditures have led to consumer and payer driven efforts to better understand, measure and influence medical care quality. As a result, independent bodies with overall responsibility to monitor the quality of healthcare systems and to guide changes have emerged. However, the work of these groups have been impeded by a lack of reliable and valid data for measuring quality and a failure to control patient characteristics.19, 20

To our knowledge, 2 efforts have been made to address concerns about the quality of radical prostatectomy. Expert panels organized by the RAND group in California and by Cancer Care Ontario (CCO) in Canada have discussed and recommended quality indicators for prostate cancer surgery, based on the literature and/or consensus from the clinicians.7, 17 However, there is very little evidence on the feasibility, reliability and validity of many of the proposed indicators,17, 21–23 undermining their use for distinguishing between good and poor quality surgical care and their application to the quality-of-care process.18, 19

We investigated whether radical prostatectomy quality indicators are readily available from patient charts and whether those indicators that can be easily assessed are valid in distinguishing quality of care in a large population of patients. Because there is no reference standard for the determination of surgical quality, we sought instead to assess the convergent construct validity of these quality indicators,24, 25 an approach which requires the identification of a more established variable that is thought to measure the same construct as the variable(s) you are attempting to validate. In our case, we hypothesized that hospital prostatectomy volume would correlate with our candidate quality indicators because it is well established that higher volume hospitals have better post-surgical outcomes.26–29 We also investigated whether selected explanatory variables account for some of the associations we observed.

CCE, Division of Cancer Care and Epidemiology; CCO, Cancer Care Ontario; CCP, Canadian Classification of Procedures; CI, confidence interval; CIHI, Canadian Institute for Health Information; CIRS, Cumulative Illness Rating Scale; DAD, Discharge Abstract Database; ICD-9, International Classification of Diseases, 9th edition; OCR, Ontario Cancer Registry; OR, odds ratio; PSA, prostate-specific antigen; RCC, regional cancer center; RPP, radical perineal prostatectomy; RRP, radical retropubic prostatectomy; SAS, Statistical Analysis Software; SEER, Surveillance, Epidemiology and End Results Program; SES, socioeconomic status.

Material and methods

This was a retrospective cohort study that used chart-based data collected previously for a study whose objectives included an exploration of within-modality practice variations and their impact.

Study population

The study population was a stratified random sample of 1,703 patients chosen from a list of all prostate cancer patients treated for cure within 6 months of diagnosis with either external beam radiation or radical prostatectomy in Ontario between January 1, 1990 and December 31, 1998. Stratification was based on residence across the 8 Ontario Regional Cancer Center (RCC) catchments to ensure enough subjects from sparsely populated areas. Review of the medical charts confirmed that 661 of these patients had received curative surgical treatment. Two patients were excluded due to insufficient information and 14 were excluded because the surgery date exceeded 190 days following diagnosis for a final study population of 645.

Data sources

Subjects were identified through a cancer treatment database housed at the Division of Cancer Care & Epidemiology (CCE) at the Queen's Cancer Research Institute in Ontario, Canada. The CCE database links Ontario Cancer Registry (OCR) records with Canadian Institute for Health Information's (CIHI) Discharge Abstract Database (DAD), along with Cancer Care Ontario (CCO) radiotherapy treatment data and census area-level socioeconomic status (SES) indicators from Statistics Canada.30, 31 The ICD-9 diagnosis code 185 identified prostate cancer patients32 in the OCR and surgical procedure codes recorded in the DAD were used to identify the subset who had radical prostatectomy within 6 months of diagnosis.

Extensive data were collected on the study subjects during the larger study through retrospective chart review. Under the direction of the study coordinator, healthcare providers trained as data abstractors reviewed all treating hospital charts. Where available, cancer center charts were also reviewed to provide a greater depth of information. Other charts (including secondary hospital, urologist and general practitioner charts) were pursued if further initial information or post-treatment followup was required. The abstraction procedure was standardized using a computerized abstraction tool and comprehensive instruction manual. Logic and missing data checks were conducted weekly. The study coordinator conducted a mid-study in-field data validation for each abstractor. All information was entered directly into an electronic database using Medquest software.33

Study variables

Quality indicators

We selected a modified subset of quality indicators identified in the RAND and CCO-recommended lists that we had included in our chart abstraction and that were related to surgeon skill. We added 2 others, use of nerve-sparing surgical technique and positive surgical margins, because they had been of interest to the CCO panel (one of the authors, RS, was a member of that panel). The panel had rejected these indicators because they thought that variations in their use could not be interpreted without adjustment for patient and disease characteristics, but such adjustment was possible in our context. Our candidate list was: acute surgical complications, defined as any record of rectal injury, thromboembolism, or death within 90 days post-surgery, or myocardial infarction within 7 weeks post-surgery; total blood transfusions of 3 units or greater within 90 days post-surgery; length of hospital stay, defined in days after surgery; use of nerve-sparing surgical technique; positive surgical margins, which we categorized as yes-extensive, yes-minimal, no and unknown; undetectable PSA at first follow-up after surgery; urinary incontinence, arising post-surgery; erectile dysfunction, arising post-surgery; and lastly, biochemical disease-free survival, at 5 years after post-surgery.

Volume assignment

We used the DAD information in the CCE database to calculate each Ontario hospital's radical prostatectomy volume separately for each year of the study period. On the basis of a comparison of our chart-based surgical information to the procedures recorded in the DAD, we concluded that radical prostatectomy might have been coded incorrectly into the DAD for some patients, so in addition to the Canadian Classification of Procedures (CCP) code 72.4 (radical prostatectomy), we also included surgeries within 7 days prior and 6 months after diagnosis that were coded as 72.2 (suprapubic prostatectomy), 72.3 (retropubic prostatectomy), 72.52 (perineal prostatectomy) and 72.59 (other prostatectomy).34

We grouped hospital volume into 6 categories (<1, <2, <3, <4, <7 and ≥7) by average number of radical prostatectomies per month. Categories were chosen based on a roughly equal distribution of subjects in the lowest 4 categories and to distinguish between the 2 highest volume categories. The volume assignment was such that a given hospital's value could vary over time and treating hospital volume assignment to the patient data in our study was therefore linked to the year of treatment.

Explanatory variables

We examined how the associations between our surgical quality indicators and hospital volume were influenced by the following explanatory variables: age,2, 35–37 comorbidity expressed as the Cumulative Illness Rating Scale (CIRS) total score,36, 38–40 era of diagnosis (1990–1992, 1993–1995 and 1996–1998),3, 38, 41, 42 socioeconomic status via median household income by census enumeration area43 and disease severity (pretreatment PSA level, clinical T category and biopsy Gleason score).44–47

Statistical analysis

Data were analyzed using SAS (Statistical Analysis Software), version 8.2.48 The significance of the association between volume and each of the explanatory variables was tested using the Pearson χ² test for categorical covariates, and one-way ANOVA for continuous covariates. Explanatory variables must have been marginally associated with volume (p ≤ 0.20) for inclusion in the multivariate regression models.

Trends in the quality indicators with increasing hospital volume were assessed using the Cochran-Armitage linear test for trend where the quality indicator was dichotomous,49 and one-way ANOVA with linear contrast coefficients where the quality indicator was continuous. For nominal quality indicators, it was not possible to test for a linear trend and the Pearson χ² test for association was used instead. In the categorical data analyses, if more than 20% of the cells had counts less than five, the Monte Carlo simulation for the exact test was substituted.50 Two-sided p-values below 0.05 were considered statistically significant.

Logistic regression analysis or multivariate linear regression was performed to compute crude effect estimates for those quality indicators whose association with hospital volume revealed a statistically significant trend. We investigated whether these associations were modified by era (p < 0.05) to address a concern that association between hospital volume and our quality indicators may have varied over time. For quality indicators where there was no significant interaction, results were adjusted for those explanatory variables that caused at least a 10% change in at least one of the point estimates.51 Era was forced into all final models in order to account for variations in the volume distribution and because changes in other variables were expected over time.41, 42, 52

Results

Study subjects

The characteristics of the 645 members of the study population are presented in Table I. Most (89.5%) received radical retropubic prostatectomy (RRP) and were diagnosed in the later years of the study period. Socioeconomic status was fairly evenly distributed across the median household income quintiles, indicating that the income distribution of these patients was similar to that of the general population.

Table I. Description of the Study Population

	Mean	SD
Age at diagnosis (Years)	63.5	6.1
Comorbidity (CIRS total score)	5.1	3.3
Missing (N)	9
	N	%
Era of diagnosis
1990–1992	113	17.5
1993–1995	232	36.0
1996–1998	300	46.5
Socioeconomic status (Median household income)
Lowest quintile	106	16.6
2	135	21.2
3	136	21.4
4	150	23.6
Highest quintile	110	17.3
Missing	8
Pretreatment PSA (ng/mL)
<4	66	11.0
4–10	298	49.8
10–20	176	29.4
≥20	59	9.9
Missing	46
Clinical T category
T1a and T1b	39	6.2
T1c	215	34.3
T2a	225	35.9
T2b	139	22.2
T3 and above	9	1.4
Missing	18
Biopsy Gleason score
2–4	146	24.1
5	101	16.7
6	186	30.7
7	131	21.7
8–9	41	6.8
Missing	40

The study population generally fell within the appropriate age range for radical prostatectomy, with an interquartile range of 60–68 years. Almost all of the patients (98.3%) were younger than 75 years, the threshold stated in the CCO quality indicator recommendations.7 In terms of disease severity, 9.9% of patients had a pretreatment PSA level ≥20 ng/ml, 1.4% had a clinical stage of T3 or worse and 6.8% had biopsy Gleason scores of 8 or 9. According to the British Columbia Cancer Agency, these clinical disease characteristics represent high-risk disease for which radical prostatectomy is generally not recommended.53 The mean CIRS total score representing comorbid status was 5.1, with an interquartile range of 3–7, out of maximum possible 75 points, indicating that the study population did not have major comorbid illnesses that would contraindicate radical prostatectomy.

After data exploration and processing, we found that because of insufficient information captured in the charts, it was not feasible to assess the following quality indicators: undetectable PSA at first follow-up after surgery (25.9% missing), biochemical disease-free survival (29.2% missing), urinary incontinence (32.3% missing) and erectile dysfunction (55.7% missing). We had sufficient information on most patients (maximum of 7.1% missing) to assess the validity of our other candidate indicators: acute complications, blood loss, length of hospital stay, use of non-nerve-sparing surgical technique and positive surgical margins. Because the acute complications considered are all serious events requiring hospitalizations, we assumed that in the presence of adequate follow-up information from the time of surgery through the acute period of 90 days post-surgery, no documentation of such event meant that none occurred. Similarly, blood transfusions were well documented in surgical notes and were assumed not to have been given if it was not noted in the chart. Thus, no missing data were identified for these 2 quality indicators. On the other hand, 7.1% of patients were missing data on whether non-nerve sparing surgery was used, 2.8% of patients were missing length of hospital stay and 0.5% of patients were missing surgical margin status (distinct from a lab result with unknown margin status).

Acute complications and positive surgical margins

Only 29 subjects had any evidence of acute complications during and/or up to 90 days after surgery in their charts (Table II). Although acute complications most frequently occurred in the lowest volume category (6.2%) and least frequently in the highest (2.5%), this difference was not statistically significant (p = 0.41). Because of the small number of events and lack of statistical significance, we did not investigate this quality indicator further.

Table II. Acute Complications and Positive Surgical Margins by Hospital Volume

N	Hospital volume														p-Value
	All		<1/month		<2/month		<3/month		<4/month		<7/month		≥7/month
	645		113		164		122		106		59		81
	N	%	N	%	N	%	N	%	N	%	N	%	N	%
1 After excluding unknowns.
Acute complications, any	29	4.5	7	6.2	7	4.3	5	4.1	5	4.7	3	5.1	2	2.5	0.41
Positive surgical margins
Yes-extensive	27	4.2	4	3.6	9	5.5	4	3.3	4	3.8	2	3.4	4	4.9	0.10
Yes-minimal	221	34.3	30	27.0	58	35.4	49	40.2	30	28.6	22	37.3	32	39.5	0.531
No	320	49.6	56	50.5	77	47.0	52	42.6	65	61.9	30	50.9	40	49.4
Unknown	74	11.5	21	18.9	20	12.2	17	13.9	6	5.7	5	8.5	5	6.2
Missing	3		2		0		0		1		0		0

Positive surgical margins had a marginal statistically significant association with hospital volume (p = 0.10), and no linear trend by hospital volume was apparent among subjects for which surgical margin status was known (Table II). However, surgical margin status was more likely to be absent from the surgical pathology reports for men treated at the lower volume hospitals, indicating differences in the quality of reporting by hospital volume. Given the influence of unknown margin status on the results, it was not possible to proceed with further investigation of this quality indicator.

Length of hospital stay

The reported results for length of hospital stay are stratified by era (Table III) because the magnitude of the relationship between this indicator and hospital volume varied over time (p = 0.03). We did not examine the effects of explanatory variables because of the small sample size in the era-specific strata, especially in the earliest period (1990–1992). The length of stay generally decreased over the study period in each volume category. A trend of decreasing length of stay with increasing hospital volume was evident in the 2 most recent eras, although the results in 1993–1995 may be driven by the short length of stay in the <7 month category which is not maintained in the highest volume category.

Table III. Mean Length of Hospital Stay (In Days) by Hospital Volume, Stratified by ERA of Diagnosis

	Hospital volume												p-value1
	<1/month		<2/month		<3/month		<4/month		<7/month		≥7/month
	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD	Mean	SD
1 One-way ANOVA test with linear contrast coefficients.
All eras	12.0	5.9	8.7	4.7	9.0	4.5	6.8	3.7	6.3	3.3	7.5	3.4	<0.0001
1990–1992	14.2	6.6	11.6	4.4	10.4	3.3	7.7	0.6	11.0	0.9	11.5	2.5	0.13
1993–1995	11.5	5.1	10.2	5.2	9.4	3.2	9.3	4.5	5.6	3.0	10.2	2.5	0.003
1996–1998	7.6	2.6	7.1	3.7	7.6	6.2	5.8	2.7	6.0	3.1	5.7	2.3	0.007

Blood transfusions and non-nerve-sparing surgery

Table IV presents the results from the logistic regression modeling for blood transfusions of 3 units or greater and use of non-nerve-sparing surgical technique. The 2 lowest volume categories (<1 and <2 per month) had significantly higher odds of receiving blood transfusions of ≥3 units when compared to highest volume group. However, for the middle volume categories of <3, <4 and <7 per month, both the crude and adjusted odds ratios for blood transfusions demonstrated similar, but insignificantly, increased odds compared to the reference.

Table IV. Logistic Regression Analysis of Blood Transfusions and Non-Nerve-Sparing Surgery by Hospital Volume

	N	%	Crude		Adjusted
			OR	95% CI	OR	95% CI
1 Blood transfusions adjusted for: comorbidity, era of diagnosis, pretreatment PSA, clinical T category and biopsy Gleason score. 2 Non-nerve-sparing surgery adjusted for: era of diagnosis, pretreatment PSA and biopsy Gleason score.
Blood transfusions of 3 units or greater1
<1/month	83	77.6	7.78	3.43, 17.62	6.14	2.41, 15.63
<2/month	109	72.2	4.00	1.79, 8.92	3.63	1.49, 8.82
<3/month	84	71.2	2.12	0.90, 5.01	1.79	0.68, 4.69
<4/month	53	55.8	1.62	0.66, 4.00	1.45	0.52, 4.02
<7/month	38	76.0	1.64	0.59, 4.54	1.80	0.59, 5.45
≥7/month (Reference)	37	47.4	1.00	–	1.00	–
Use of non-nerve-sparing surgical technique2
<1/month	52	46.0	3.83	2.03, 7.24	4.20	1.99, 8.89
<2/month	50	30.5	2.88	1.63, 5.08	3.01	1.61, 5.62
<3/month	23	18.9	2.74	1.51, 4.97	2.43	1.24, 4.76
<4/month	16	15.1	1.40	0.77, 2.55	1.17	0.61, 2.24
<7/month	9	15.3	3.51	1.60, 7.71	4.72	1.92, 11.59
≥7/month (Reference)	8	9.9	1.00	–	1.00	–

The use of non-nerve-sparing surgery decreased as volume increased, with all volume categories except the <4/month group having significantly higher odds of receiving non-nerve-sparing surgery when compared with highest volume reference group. An exception occurred in the <7/month group, with higher (and statistically significant) ORs than the next lowest category.

Discussion

We tested the convergent construct validity of prostatectomy quality indicators by testing them against hospital prostatectomy volume, a surrogate measure of surgeon experience and an accepted structural indicator of the quality of surgical care. We demonstrated convergent construct validity for number of units of blood transfused, length of hospital stay and use of a nerve-sparing surgical technique providing evidence that these easily assessed quality indicators may be valid surgical indicators for quality assessment purposes.

In addition to the investigation of construct validity, we were able to assess feasibility of our candidate list of quality indicators. Feasibility was not demonstrated for: undetectable PSA at first post-surgical follow-up, biochemical disease-free survival and urinary incontinence and erectile dysfunction post-treatment.

This was the first Canadian study examining the feasibility and validity of quality indicators in localized prostate cancer surgery and it was the first study in the field to examine the relationship between quality indicators and volume while considering the effect of key explanatory variables. Most other work in this area has used U.S. administrative data, which have little information on covariates.26–29, 54–56 Both the OCR and the DAD, the key sources of the CCE database for this study, have high capture rates and are thus excellent sources for identifying study subjects.57–59 Over-sampling of subjects from the smallest Ontario Regional Cancer Centers increased our ability to study low volume hospitals. We captured information on 97.6% of patients confirmed to have received radical prostatectomy as curative treatment in our sample. Also, we used high-quality, medical chart data, often accessing more than 1 chart to maximize data capture. Chart abstractors were unbiased because they were unaware of the study objectives.

The advantages of using the 3 validated quality indicators to track prostatectomy-related quality of care are that they are measurable at the time of the prostatectomy admission and their capture does not necessarily require medical chart review. These quality indicators add information about how care can be improved, which contrasts with the report of volume effects that suggest centralization of care only rather than addressing underlying quality issues. We caution that due to the stratified random sampling strategy utilized, our volume-related results do not necessarily describe the average overall Ontario experience.

We could not assess incontinence and erectile dysfunction as quality indicators because of missing or inadequate chart information. This finding was consistent with the RAND followup studies, which reported that patient satisfaction with their urinary and erectile status after surgery could be assessed using validated patient questionnaires, but not through administrative documents or charts.21, 22 According to Miller et al., the systematic use of validated survey instruments for preoperative and postoperative urinary, sexual and bowel function are necessary to truly measure and compare the occurrence of these important quality-of-life outcomes following radical prostatectomy.21 Such an approach would address the data quality issues accounting for the wide variability in late morbidity rates in the literature.3, 4, 35, 38, 60–62

It was feasible to assess acute surgical complications in our chart review, supporting the conclusion reached by Miller et al.21 Although there was no statistically significant linear trend, this may have been due to a lack of study power rather than the absence of a real trend by volume, as we had 40% power to detect a difference.63 Because the rates were too low (2.5–6.2%) for us to conduct multivariate analysis, it is possible that differences in comorbidity, disease severity and other covariates might explain some or all of this volume trend and thus, it was not possible to determine the construct validity of acute surgical complications. Cancer Care Ontario is currently reporting 2002–2004 30-day mortality rates across geographic areas that range from 0–0.95% and do not vary statistically.64 With such low rates and without control for covariates, it is questionable whether 30-day mortality and acute surgical complication rates are very useful indicators of differences in the quality of surgical care. This is especially the case for men currently receiving radical prostatectomy, for whom these rates are likely even lower than the men in this study, as surgical quality should have improved over the past decade since this data was collected.

We were able to access pathology reports for almost all of the patients in our study population. It was not possible however, to determine the construct validity of positive surgical margins using these reports because of the high proportion that did not mention margin status. It is interesting to note that lower volume hospitals were more likely to have missing documentation of margin status, which is a pathology reporting quality indicator currently reported by Cancer Care Ontario.23, 64

Our finding that blood transfusion is a valid quality indicator is consistent with research conducted in the U.S. and in Europe. In a multi-institution study conducted over a similar time period of 1994–2000, for 1123 consecutive RRP cases treated at the University of Michigan,65 surgeons who performed ≤15 surgeries per year had an 8.6 times increased odds of their patients requiring any blood transfusion compared to those who performed >5 surgeries. Investigators in a European study66, 67 also found that blood transfusions of ≥3 units were given more frequently by lower volume surgeons. However, their reported rates were all lower than ours. For example, they reported that 17.4% of patients treated by surgeons performing less than 12 RRPs annually required transfusion compared to 46.0% in our corresponding <1/month group. This difference might be explained by the voluntary participation of surgeons in that study and the self-report of their performance, which created opportunities for selection and response bias. In addition, the European study did not adjust for covariates although differences were noted, and the nature of the participating institutions was unclear (i.e., academic centers vs. community hospitals).

The average length of hospital stay decreased over the 3 eras of the study, in the meantime becoming more reflective of surgical quality. The phenomenon of declining length of stay is consistent with expectations, as economic pressures for early discharge in the cost-cutting climate of the 1990s led to evolutions in perioperative management that were successful at reducing length of stay without increasing acute complication and mortality rates. For example, practice changes included assessment in pre-admission clinics, admitting patients directly into the operating room on the day of surgery, and earlier post-operative ambulation.42, 68 Furthermore, the introduction of more advanced and less morbid procedures may have also contributed to the decline.68 All of these changes would decrease the likelihood of a long stay for uneventful surgeries.69 So with the clearest volume trend seen in the last era of our study (1996–1998), it seems that length of stay has become more reflective of the patient's post-surgical condition and thus a valid and useful indicator of surgical quality.

Our findings are consistent with U.S. studies that examined the volume–length of stay relationship using the Medicare and SEER databases.29, 54 These investigators found that length of stay decreased over time and they observed longer stays in lower hospital as well as surgeon volume settings. Length of stay in these studies was generally shorter than our findings in similar eras, perhaps because the pressure to reduce costs began earlier in the American system. Although these studies did not adjust for covariates in their assessment of hospital stay, we were unable to improve upon this limitation, because we did not have enough statistical power to examine the effect of explanatory variables after stratifying by era to account for the interaction effect.

The association between the use of non-nerve-sparing surgery and hospital surgical volume became stronger with adjustment for explanatory variables, indicating that the distributions of the explanatory variables were masking some of the variation. Because treatment decisions are based on severity of disease,53 we recommend that the use of nerve-sparing surgery as a quality indicator should only occur when adjustment can be made for patient and disease characteristics.

Our evidence on the validity of nerve-sparing surgery as a quality indicator represents new knowledge. Both expert panels considered but ultimately rejected its use during their quality indicator development processes due to concerns about the need for case mix adjustment (ref.17 and Siemens DR, personal communication) and there are no studies on its use in the quality indicator and volume literature. Our findings suggest that following further investigation, the use of nerve-sparing surgery could become a useful surrogate quality indicator for evaluating the presence of late morbidities because the use of this technique has been shown to increase the recovery of continence and erectile function after radical prostatectomy.2, 23, 35, 60, 61, 70–73

In this study, we did not analyze individual surgeon volume, as that data was unavailable to us. However, most studies examining adverse outcomes following cancer surgery have focused on hospital volume, as a proxy for surgeon volume, although high hospital volume could very well be spread among a number of low volume surgeons.26 It has been shown that while surgeon volume rather than hospital volume is the most important predictor of short-term radical prostatectomy outcomes, hospital volume can still provide a good approximation of surgeon volume if surgeon-specific data are unavailable.29 A review article on urological volume literature found that improvements in acute surgical complication rates following radical prostatectomy were more significant for high surgeon volume as opposed to high hospital volume. The authors concluded that while both are independent predictors of outcome, models that include both surgeon and hospital volume suggest that high volume hospitals have better outcomes in part because of the effect of surgeon volume, and vice versa.27

Some hospital volume misclassification is likely in our data due to hospital amalgamations that occurred during the study period and the unusually high use of non-nerve sparing surgery in the <7/month volume category may be due to this misclassification. Almost half of the patients in this volume category were treated at a particular group of hospitals that amalgamated during the study period. It is possible that these amalgamated hospitals (which would be in a higher volume category than the constituent hospitals previously were) continued to operate as separate institutions that did not use this surgical technique.

Our study sample may have excluded some patients who were incorrectly coded in the DAD as having lymph node dissection only rather than radical prostatectomy. However, based on the surgical coding accuracy we observed between the DAD and our chart review, this number is likely to have been very small, and the strength of our results and the volume trend observed make it unlikely that this minor selection bias could account for all of the variations seen.

We advise further study on undetectable PSA after surgery and positive surgical margins as potential prostatectomy-related quality indicators. These measures are important because they describe the effect of surgical quality on cancer control and risk of recurrence. A PSA test performed consistently after surgery would provide a quality indicator within a short time frame after treatment. Cancer Care Ontario reports that margin status is now being consistently reported,64 so it is possible that surgical margin status is now a feasible quality indicator and it should be assessed again for its validity.

Furthermore, because incontinence and erectile dysfunction are considered very important outcomes of prostate surgery and reflect quality of surgical care,2, 35, 60, 61, 70–73 we strongly advocate that bodies responsible for cancer care quality consider moving towards widespread application of a validated survey instrument to evaluate these indicators in a proactive and prospective manner. With adequate control for confounders, this could provide relatively timely information for incorporation into quality improvement processes. This will be especially useful now, given the dramatic changes in the surgical care of prostate cancer in recent years, with the advent of less invasive surgical procedures.4, 74

Ultimately, the goal of the entire quality of care process is to improve patient outcomes following radical prostatectomy through feedback to surgeons, heads of surgical departments, hospital administrators and quality councils. Identifying groups receiving poor quality care using readily available, validated measures is a first step towards finding ways that surgical care can be improved for all men undergoing radical prostatectomy for prostate cancer.

Acknowledgements

We thank Ms. Susan Rohland for insight on the parent study. This study used data from the Ontario Cancer Registry, Cancer Care Ontario and the Canadian Institutes of Health Information's Discharge Abstract Database.