The authors assessed adherence with the College of American Pathologists (CAP) radical prostatectomy (RP) practice protocol in a national sample of men who underwent RP for early-stage prostate cancer.
The authors assessed adherence with the College of American Pathologists (CAP) radical prostatectomy (RP) practice protocol in a national sample of men who underwent RP for early-stage prostate cancer.
Using the National Cancer Data Base, the authors identified a nationally representative sample of 1240 men (unweighted) who underwent RP. For each patient, local cancer registrars performed an explicit medical record review to assess patient-level compliance with surgical pathology report documentation of 7 morphologic criteria (ie, quality indicators). Applying the CAP prognostic factor classification framework, composite measures and all-or-none measures of quality indicator compliance were calculated for the following analytic categories: 1) a strict subset of CAP category I prognostic factors (3 indicators), 2) a broad subset of CAP category I factors (6 indicators), and 3) the full set of 7 indicators.
Among a weighted sample of 24,420 patients who underwent RP, compliance with documentation of the CAP category I factors varied from 54% (95% confidence interval [95% CI], 50–58%) for pathologic tumor, lymph node, metastases classification (according to the American Joint Committee on Cancer staging system) to 97% (95% CI, 96–99%) for Gleason score. In composite, RP pathology reports contained 83% (95% CI, 81–84%), 85% (95% CI, 84–87%), and 79% (95% CI, 78–80%) of the recommended data elements measured by the strict CAP category I subset, the broad CAP category I subset, and the full set of 7 indicators, respectively. In contrast to the generally higher composite scores, only 52% (95% CI, 48–56%) and 41% (95% CI, 37–45%) of men who underwent RP had complete documentation in their pathology reports for the strict and broad CAP category I subsets, respectively.
RP surgical pathology reports contained most of the recommended data elements; however, the frequent absence of pathologic stage provides an opportunity for quality improvement. Cancer 2007. © 2007 American Cancer Society.
Recognizing the need for meticulous clinical communication,1 the College of American Pathologists (CAP) endorsed a consensus statement that classifies the prognostic parameters (eg, Gleason score, margin status) derived from radical prostatectomy (RP) specimens.2–4 Concurrently, the regularly updated CAP prostate cancer protocol seeks to facilitate systematic, clear, and unambiguous reporting of prognostically significant pathologic findings from individual RP specimens.2–4
In addition to its relevance for individual patients, assessment and optimization of the quality of pathologic care for men undergoing RP is recognized today as an important population-level cancer-control initiative.1, 5 The Institute of Medicine's (IOM) Committee on Assessing Improvements in Cancer Care recently identified the adequacy of RP pathology reports as a useful metric for the quality of diagnostic prostate cancer care.6 That committee concurrently highlighted the paucity of existing data regarding the quality of pathology reporting for RP and recommended the American College of Surgeons' (ACoS) Commission on Cancer (CoC)-sponsored studies as a potential source of benchmark data.6
Coincident with these activities, investigators at RAND developed a set of quality indicators for early-stage prostate cancer care.7–9 The RAND indicators included adherence to the College of American Pathologists Cancer Committee's practice protocol for the management of pathology specimens as a valid and feasible quality indicator for men undergoing RP for early-stage prostate cancer.7, 9
In an effort to build on this complementary work, the ACoS CoC undertook a special study with the broad goal of using a subset of the RAND indicators to perform the first nationwide assessment of the quality of care for men with localized prostate cancer.10 One study objective was to assess pathologic quality indicator compliance in the context of the CAP RP protocol. This goal was significant, because it provides contemporary, nationally representative pathologic benchmarks that are relevant to ongoing state and national cancer control endeavors and simultaneously evaluates the feasibility of using the existing CoC infrastructure to monitor the quality of pathology reports for oncologic surgical specimens. In this context, we set out to determine the quality of pathology reporting for RP specimens in the U.S.
The National Cancer Database (NCDB) is a project of the ACoS, and receives funding for operational support from the American Cancer Society. The NCDB maintains data on cancer diagnosis, management, and outcomes among patients diagnosed at CoC-approved programs in the U.S.11 NCDB data are collected from hospital-based cancer registries using a standardized, electronic data abstraction format.11 Demographic characteristics of patients with prostate cancer reported to the NCDB are similar to those in the population-based sample maintained by the Surveillance, Epidemiology, and End Results (SEER) registry.12 From 2000 to 2001, the NCDB collected data for nearly 70% of incident prostate cancer cases in the U.S.
For the current study, we sampled existing cases from the NCDB based on the following a priori inclusion criteria: 1) black or white men diagnosed with adenocarcinoma of the prostate in 2000 or 2001, and 2) American Joint Committee on Cancer stage I or II tumors (ie, early-stage or localized disease). Using these criteria, a de-identified file of 117,953 men with localized prostate cancer diagnosed during 2000 and 2001 was extracted from the NCDB. From this population, we selected a 5% stratified, random sample of cases that comprised equal-sized cells based on race (2 levels: black and white), U.S. Census division (9 levels: New England, Middle Atlantic, South Atlantic, East North Central, East South Central, West North Central, West South Central, Mountain, and Pacific), and CoC categories of approval13 for participating facilities (3 levels: teaching-research hospitals, comprehensive cancer centers, and community cancer centers). Based on this design, we developed patient-level sampling weights that represented each patient's inverse probability of inclusion in the overall (ie, eligible) study sample (n = 5655 men). We applied the sampling weights in all subsequent analyses to produce nationally representative estimates.
This sampling scheme yielded 5655 eligible men, and these were submitted to participating facilities for explicit chart abstraction, including assessment of quality indicator compliance. The number of patients selected from any 1 facility ranged from 1 to 30; eligible patients must have received all or part of their first course of therapy at the reporting facility. The subsequent case-level response rate was 92.5%, resulting in an unweighted sample of 5230 men with early-stage prostate cancer. Among this sample, we used explicitly collected variables that described cancer-specific surgery and surgical approach to identify men who underwent RP.
Given the necessarily large number of individuals performing data abstraction, we used a pilot-tested, study-specific chart abstraction instrument to guide local registrars in their assessment of indicator compliance. In preliminary studies, inter-rater reliability with a similar chart abstraction tool exceeded 95%.14 We also developed a manual that contained uniform and explicit instructions for verifying compliance (or lack thereof) with individual quality indicators. For all data elements that were not reported previously to the NCDB, we instructed data abstractors to perform an explicit medical record review that included recollection of certain previously abstracted variables (eg, treatment type, treatment dates). For an additional quality control measure, we requested that a designated physician review the data items in each report for completeness and validity.
Developed in 19944 and subsequently updated in 19993 and 2005,2, 15 the CAP protocol assists pathologists in the provision of essential clinical information when reporting results for RP specimens. The protocol distinguishes 3 categories of prognostic factors from the RP surgical pathology report (Table 1).2, 15 Category I prognostic factors (Gleason score; pathologic tumor, lymph node, and metastases [TNM] stage according to the American Joint Committee on Cancer [AJCC] classification; and surgical margin status) are those for which the prognostic value and relevance to patient management are supported well by the literature. Category II factors (eg, tumor volume, histologic subtype) comprise pathologic findings that show significant promise as prognostic variables but require additional validation studies prior to routine clinical use. Category III factors are histologic findings (eg, perineural and lymphovascular invasion) for which there are insufficient data to support prognostic value.
|Morphologic criteria||CoC study definition||CAP prognostic category*||Analytic indicator sets|
|Strict CAP category I cubset||Broad CAP category I subset||Full indicator set|
|Pathologic Gleason score||Documentation of Gleason score in surgical pathology report||I||X†||X||X|
|Pathologic TNM stage||Documentation of pathologic TNM stage in surgical pathology report||I||X||X||X|
|Seminal vesicle involvement||Documentation of seminal vesicle involvement in surgical pathology report||I‡||X||X|
|Capsular invasion||Documentation of capsular invasion in surgical pathology report||I‡||X||X|
|Tumor location||Documentation of tumor location in surgical pathology report||I‡||X||X|
|Margin status||Documentation of surgical margin status in surgical pathology report||I||X||X||X|
|Tumor size||Documentation of tumor size in surgical pathology report||II||X|
In the current study, we assessed adherence to the CAP RP protocol by evaluating surgical pathology report documentation of the following 7 morphologic-based criteria: 1) Gleason score, 2) pathologic stage (TNM), 3) surgical margin status, 4) presence or absence of seminal vesicle invasion, 5) presence or absence of capsular invasion, 6) tumor size, and 7) tumor location. We refer to each of these morphologic criteria as quality indicators.
To guide abstractors' assessments, we provided the following written instructions: “This item describes the documentation appearing on the surgical pathology report following radical retropubic or perineal prostatectomy. Indicate whether each of the following items (Gleason score, pathologic stage, status of surgical margins, status of seminal vesicles, status of capsular invasion, location of tumor, size of tumor) was documented on the surgical pathology report” (emphasis present in instruction manual). Consistent with established methods for indicator assessment, failure to document findings (positive or negative) for an indicator was considered noncompliance.7, 9, 16, 17
The primary outcome for this study was subject-level indicator compliance. For analytic purposes, we defined 3 distinct (but not mutually exclusive) sets of pathologic quality indicators (Table 1). The first analytic set comprises the 3 explicitly defined CAP category I prognostic factors (Gleason score, pathologic TNM stage, and surgical margin status) (Table 1).2 We refer to this group as the strict CAP category I subset. Because tumor location, seminal vesicle status, and capsular invasion also make essential contributions to the accurate assignment of pathologic stage, we combined those criteria with the 3 category I criteria to define a second analytic indicator set (Table 1). We refer to this group of 6 indicators as the broad CAP category I subset. The final analytic group comprises the full indicator set of 7 pathologic criteria assessed in the CoC special study (Table 1).
Nolan and Berwick described 3 different approaches to measuring compliance with multiple, discrete measures for the same clinical condition (as in the current study).18 The first is item-by-item measurement, in which compliance with each measure is reported separately. For this approach, the individual quality measure is the unit of analysis; the denominator is the number of patients in the sample who are eligible for assessment, and the numerator is the number of patients with documented compliance.18 The second approach, which is referred to generally as composite measurement, specifies the entire study sample as the unit of analysis. Using this approach, composite performance on multiple elements of care (eg, reporting of multiple pathologic data elements) is determined by computing a percentage across all patients and quality indicators.18 The composite measurement approach is used by the Centers for Medicare and Medicaid Services (CMS) in its Hospital Quality Demonstration Project.18, 19 The third approach is all-or-none measurement, which uses the individual patient as the unit of analysis.18 Using this methodology, a compliance percentage is calculated by specifying an all-or-none rule (eg, a pathology report must contain all of the recommended data elements to be compliant) at the patient level. The all-or-none measurement approach is favored now by the IOM and CMS, because it better represents the needs of individual patients.18–20 In the current study, we used all 3 approaches to assess the quality of pathology reporting for RP specimens.
In our first analytic step, we calculated the item-by-item mean percent compliance (with 95% confidence intervals [95% CIs]) for each of the 7 measured pathology indicators (Table 1). For each item, the numerator is the number of pathology reports that contained the relevant data element, and the denominator is the total number of cases (ie, pathology reports) evaluated.
Next, we combined data across individual patients and indicators to calculate composite measures of quality indicator compliance. Specifically, we divided all instances in which review of a pathology report confirmed adherence with documentation for an individual indicator (ie, the numerator) by the total number of eligible indicators across all patients (ie, the denominator). We calculated composite compliance proportions for the strict and broad CAP category I analytic sets and for the full analytic set of 7 indicators.
We also used individual patients as the unit of analysis to calculate all-or-none measurements of quality indicator compliance. Of primary interest, we determined the proportion of men whose pathology reports achieved complete compliance with indicator documentation. We defined complete compliance as documented adherence with all of the indicators in a particular analytic set. The numerator, therefore, is the number of men who were compliant with each of the indicators (for a given analytic indicator set). The denominator for this calculation is the total number of patients evaluated. For example, the complete compliance proportion for the strict CAP category I subset was calculated by dividing the total number of men whose pathology reports contained documentation for Gleason score, pathologic stage, and surgical margin status (ie, the numerator) by the total number of patients evaluated (ie, the denominator). We also determined the proportion of patients who achieved compliance with all but 1 indicator in a particular analytic subset. We performed the all-or-none analyses for the strict and broad CAP category I subsets and for the full set of 7 indicators.
All results are presented as proportions (with 95% CIs), theoretically ranging in value from 0% to 100%. To obtain national estimates of adherence, we applied sample weights and the strata variable for all analyses using SAS software (version 9.1; SAS Institute, Cary, NC). We perform no hypothesis testing; therefore, our analyses do not account for potential clustering of outcomes within hospitals.
Among the national sample of 5230 men (92.5% case-level response rate) who received early-stage prostate cancer care at 984 CoC-approved facilities in 2000 or 2001, we identified 1390 men (from 542 facilities) who underwent initial RP. Eliminating 150 men who were noncompliant with all 7 pathology indicators (whose surgical pathology reports presumably could not be identified during medical record review) yielded an unweighted analytic sample size of 1240 men. Application of the sample weights to this unweighted analytic cohort yielded a weighted sample of 24,420 patients who underwent surgery. Table 2 summarizes the demographic and cancer-specific characteristics of the RP cohort.
|Characteristic||No. of patients (%)|
|Total no. of men||24,420|
|Pretreatment PSA, ng/mL‡|
|Clinical T classification|
|Biopsy Gleason score¶|
|Use of neoadjuvant hormone therapy|
|No. of comorbidities#|
|Private insurance||5758 (23.6)|
|Managed care||8381 (34.3)|
|Not insured||328 (1.3)|
|Comprehensive community cancer center||9567 (39.2)|
|Community cancer center||9933 (40.7)|
|U.S. Census division|
|New England||1723 (7)|
|Middle Atlantic||3713 (15.2)|
|South Atlantic||5118 (21)|
|East North Central||3797 (15.6)|
|East South Central||1960 (8)|
|West North Central||1594 (6.5)|
|West South Central||2294 (9.4)|
Table 3 presents item-by-item compliance for the individual pathology indicators. The inclusion of individual measures in RP pathology reports varied from 42% (95% CI, 38–46%) for tumor size to 97% (95% CI, 96–99%) for Gleason score. Among the CAP category I factors, adherence was lowest for the documentation of pathologic TNM stage (54%; 95% CI, 50–58%).
|Pathology quality indicators||Weighted % compliance (95% CI)*|
|Gleason score||97.3 (95.9–98.8)|
|Pathologic TNM stage||54.3 (50.3–58.4)|
|Surgical margin status||95.7 (94–97.4)|
|Seminal vesicle status||92.8 (90.6–95.1)|
|Capsular invasion status||77.4 (74–80.7)|
|Tumor location||94.1 (92.3–96)|
|Tumor size||41.9 (38–45.8)|
Table 4 presents the results for the composite measurement approach. In this table, we specify the number of indicators that were included in the composite score for each analytic indicator set, the weighted number of men who were eligible for compliance assessment within each analytic set, the corresponding weighted number of eligible events (ie, the denominator), and the weighted number of times that indicator compliance was documented. In composite, RP pathology reports contained 83% (95% CI, 81–84%) and 85% (95% CI, 84–87%) of the recommended data elements measured by the strict and broad CAP category I subsets, respectively (Table 2). For the full set of 7 pathology indicators, the composite compliance proportion was slightly lower at 79% (95% CI, 78–80%).
|Pathology quality indicators||No. of indicators||Weighted no. of eligible men||Weighted no. of eligible events (Denominator)*||Weighted no. of times indicator compliance documented||Weighted % compliance (95% CI)|
|Strict CAP category I indicators†||3||24,420||73,260||60,414||82.5 (80.6–84.3)|
|Broad CAP category I indicators‡||6||24,420||146,520||124,975||85.3 (84.1–86.5)|
|Full indicator set||7||24,420||170,940||135,199||79.1 (77.8–80.4)|
Table 5 summarizes the results for the all-or-none measurement approach. In contrast to the generally higher composite scores, only 52% (95% CI, 48–56%) and 41% (95% CI, 37–45%) of men who underwent RP had complete documentation (ie, complete compliance) in their pathology reports for the strict and broad CAP category I subsets, respectively (Table 5). When we considered the full indicator set (7 measures), the complete compliance proportion decreased to 21% (95% CI, 18–25%) (Table 5). The pathology reports for 96% of patients contained documentation for at least 2 of the 3 criteria in the strict CAP category I subset.
|Pathology quality indicators||No. of indicators||Median no. of indicators with documented compliance (range)||Weighted % complete compliance (95% CI)†||Weighted % complete compliance or compliant with all but 1 quality indicator (95% CI)‡|
|Strict CAP category I indicators§||3||3 (1–3)||52 (47.9–56.1)||96.2 (94.6–97.9)|
|Broad CAP category I indicators‖||6||5 (1–6)||41.3 (37.3–45.3)||79.4 (76–82.8)|
|Full indicator set||7||6 (1–7)||21.4 (18.2–24.6)||55.6 (51.7–59.5)|
In this report, we have provided a contemporary description of the quality of surgical pathology care reporting for men in the U.S. who undergo RP for early-stage prostate cancer. Among the 7 morphologic indicators that we assessed in this study, compliance ranged from 42% for documentation of tumor size to 97% for documentation of the Gleason score. For the entire sample, the pathology reports contained 83% of the data elements specified by the strict CAP category I subset and 79% of recommended data as measured by the full set of 7 pathologic indicators. At the patient level, only 52% and 21% of men who underwent RP had pathology reports that achieved complete compliance with documentation for the strict CAP category I subset and the full set of 7 pathology indicators, respectively.
In general, our findings are consistent with the limited existing literature that evaluates the quality of pathologic assessment and reporting for RP and other surgical oncology specimens.16, 17, 21–23 In a study of Medicare beneficiaries, Imperato et al. reported similarly high item-by-item levels of compliance for both surgical margin status (96%) and Gleason score (97%). Unlike the current study, however, those authors did not evaluate compliance with the assignment of pathologic stage.16, 17, 22 Furthermore, all-or-none measurements (with individual patients as the unit of analysis) of pathology indicator compliance have not been reported previously; the less favorable performance on this metric for the strictly defined CAP category I factors (52%), for instance, generally reflects the absence of partial credit18 for cases with documentation of Gleason score and margin status but not pathologic TNM stage. Taken together, these data suggest that most men receive high-quality assessment and communication of the pathologic findings in their RP specimens. Recognizing, however, that completeness of pathology reports for both individual items and individual patients is the objective,1 a second principal finding is that opportunities exist to improve surgical pathology care for men who undergo RP.
Specifically, despite outstanding performance with respect to both Gleason score (97%) and surgical margin status (96%), only 54% of pathology reports contained explicit documentation of the pathologic TNM stage (the third category I prognostic factor). The less frequent compliance with documentation of pathologic stage may not be surprising, because the assignment of a formal TNM stage requires both the presence and integration of several data elements, including tumor location, extraprostatic extension, seminal vesicle invasion, and lymph node status (which may be unknown in the increasingly common scenario in which concurrent pelvic lymphadenectomy is not performed24). Moreover, we acknowledge that many of the reports without explicit documentation for pathologic TNM stage contained sufficient data (ie, the remaining elements of the broad subset of CAP category I factors) to ascertain the pathologic T classification.
At the same time, however, such caveats do not necessarily justify the omission of pathologic stage from RP specimen pathology reports.25, 26 To be sure, routine and accurate synthesis and reporting of pathologic TNM stage (and its component data elements) guide evidence-based recommendations for adjuvant and salvage therapies,2, 22, 27 precise assessment of eligibility for clinical trials of emerging therapeutic protocols,28 communication among clinicians from different specialties and institutions,1, 2 and prognostic group assignment by cancer registrars.25 Moreover, the AJCC guidelines29 specify very few situations in which a specific T classification, N classification, or M classification cannot be assigned (even if assignment relies on some combination of clinical judgment and relevant imaging studies).25 Accordingly, the frequent absence of pathologic stage highlights an opportunity to improve the quality of pathology reports for individual patients who undergo RP.25
Beyond their clinical implications, our findings are consequential for current population-level cancer control initiatives.6 For instance, in 2004, the ACoS CoC modified its accreditation process by requiring that pathology laboratories at CoC-certified facilities explicitly report the following scientifically validated CAP measures for RP specimens: histologic type, Gleason score, pathologic stage (TNM), surgical margin status, extraprostatic extension, and seminal vesicle invasion.30 Currently, the CoC is working to institute a complementary national audit and feedback program with the specific objectives of evaluating and improving the proportion of RP (and other surgical oncology specimen) pathology reports that include all of the CAP-recommended data elements. Data from the current study may provide useful points of reference for evaluating the success of this nascent intervention.
It is noteworthy that a Medicare Peer Review Organization previously demonstrated the feasibility of using audit and feedback to improve the quality of RP pathology reports.16, 17, 22 In a study that was performed in New York, Imperato et al used a cooperative educational program, which included a performance audit with feedback to hospitals and pathology laboratory directors, to facilitate improvements in pathology report documentation for 10 quality indicators. After the intervention, compliance improved for 9 of the 10 measures (range of improvement, 1.4–23.9%).16, 17, 22 Despite its success, that program had several limitations, including its temporary nature, limited geographic scope, and inclusion of certain indicators (eg, frozen section submission) with limited clinical validity. Ideally, future interventions in this area will employ a sustainable, national infrastructure and will maintain a primary focus on achieving universal compliance with documentation of pathologic TNM stage and the other CAP category I prognostic factors.1, 2
Finally, these data may motivate surgeons, pathologists, laboratories, and hospitals further to achieve standardization of the basic content of surgical pathology reports.1, 2, 31–33 Standardized reporting has the potential to improve the reliability of pathologic data and, in turn, both the quality of clinical care and the validity of clinical and epidemiologic research in prostate cancer and other malignancies.1 Directly relevant to this effort, the CAP protocol for RP specimens is published in checklist form and provides a standardized, universally available medium for recording and reporting essential pathologic information.2, 15
The current study has several limitations. First, although use of the NCDB yields a nationally representative sample, the fact that our sampling frame was limited to CoC-approved hospitals introduces potential selection bias. That is, unlike nonaccredited programs, CoC-approved facilities have demonstrated attainment of a baseline quality threshold with respect to the provision or availability of basic clinical and supportive oncology services.13 Accordingly, it is possible that the quality of pathology reporting also systematically differs between CoC-approved and nonapproved facilities.
Next, our reliance on medical record abstraction to assess levels of indicator compliance raises legitimate concerns regarding the distinction between deficits in quality versus deficits in documentation.34–36 Despite this concern, our methodology was based on the a priori assumption that poor or absent documentation itself is an indicator of poor quality.7
An additional limitation stems from the largely consensus-based foundation for several of the pathology quality measures. In the absence of a clear linkage with specific, favorable outcomes, the observed variation in compliance with individual indicators simply may reflect differential interpretations of the imperfect evidence base supporting the value of reporting a particular pathologic finding.
Beyond this general concern, the individual quality indicators have several specific limitations. First, an important premise of this study is that universal pathology indicator compliance is both feasible and desirable for all men who undergo RP. However, there are noteworthy exceptions to this assumption, including the consensus that accurate Gleason score assignment is not possible for patients who receive neoadjuvant hormone therapy (9.5% of the entire RP sample, 25% of patients noncompliant with the Gleason score indicator).2, 37 Although it may be pertinent to the current study, this concern has limited applicability to quality assessments in more contemporary RP patients (among whom neoadjuvant hormone therapy has no established therapeutic benefit and, thus, its use should be rare).38 Second, compliance with the surgical margin indicator required only that the margin status be documented in the surgical pathology report. In contrast, full compliance with the CAP protocol requires additional documentation of the location and extent of positive margins.2, 15 A third limitation of the individual indicators is that there is neither a standard method for measuring tumor volume in RP specimens nor a consensus regarding the prognostic value of this information.2
Despite these limitations, the current report provides national data describing the quality of surgical pathology reports for men who undergo RP for early-stage prostate cancer. Although, at a population-level, RP surgical pathology reports contain most of the recommended data elements, the average patient has a 50% chance of receiving a pathology report that is missing clinically important data (ie, a CAP category I prognostic factor). Our findings suggest that enhanced communication and documentation of the pathologic TNM stage may be fruitful targets for quality-improvement endeavors.