The authors have no conflicts of interest to report.
Validating posttransplant hepatocellular carcinoma recurrence data in the united network for organ sharing database
Article first published online: 21 OCT 2013
© 2013 American Association for the Study of Liver Diseases
Volume 19, Issue 12, pages 1318–1323, December 2013
How to Cite
Samoylova, M. L., Dodge, J. L., Vittinghoff, E., Yao, F. Y. and Roberts, J. P. (2013), Validating posttransplant hepatocellular carcinoma recurrence data in the united network for organ sharing database. Liver Transpl, 19: 1318–1323. doi: 10.1002/lt.23735
This work was supported in part by the Biostatistics Service of the University of California San Francisco Liver Center (P30 DK026743).
- Issue published online: 27 NOV 2013
- Article first published online: 21 OCT 2013
- Accepted manuscript online: 27 AUG 2013 09:54AM EST
- Manuscript Accepted: 7 AUG 2013
- Manuscript Received: 4 JUL 2013
The Organ Procurement and Transplantation Network (OPTN)/United Network for Organ Sharing (UNOS) database is the most comprehensive collection of liver transplantation data, but the quality of these data with respect to hepatocellular carcinoma (HCC) recurrence has not been well assessed. In this study, we compared observed HCC recurrence rates in the UNOS database to expected rates calculated with a hierarchical model for recurrence adjusted for recipient and tumor characteristics. We used the UNOS Standard Transplant Analysis and Research data set for adult transplant patients with an initial exception for an HCC diagnosis granted between January 1, 2006 and September 30, 2010 who underwent transplantation within the same time window. We developed a risk-adjusted Poisson model with patients as the unit of analysis, random effects for transplant centers, and years of follow-up as an offset to predict expected recurrences for each center. To further investigate the possibility of underreporting, we imputed expected recurrences for non-HCC deaths. In all, 5034 HCC liver transplant recipients were identified, and 6.8% experienced recurrence at a median of 1 year after transplantation. The covariate-adjusted shrinkage estimates of the observed/expected HCC recurrence ratios by transplant center ranged from 0.6 to 1.76 (median = 0.97). The 95% confidence intervals for the shrinkage ratios included unity for every center, and this indicated that none could be unambiguously identified as having lower or higher than expected HCC recurrence rates. Imputing outcomes for patients potentially experiencing unreported recurrence changed the center-specific shrinkage ratios to 0.72 to 1.39 (median = 0.98), with no centers having a shrinkage ratio significantly different from 1. The observed HCC recurrence rate was not significantly lower than the expected rate at any center, and this suggests that no systematic underreporting has occurred. This study validates the OPTN HCC recurrence data and supports their potential for further analysis. Liver Transpl 19:1318–1323, 2013. © 2013 AASLD.
Adult-to-Adult Living Donor Liver Transplantation Cohort Study
donor risk index
incidence rate ratio
Model for End-Stage Liver Disease
Organ Procurement and Transplantation Network
Scientific Registry of Transplant Recipients
United Network for Organ Sharing
The Organ Procurement and Transplantation Network (OPTN)/United Network for Organ Sharing (UNOS) database contains information on all transplants occurring in the United States since 1986, and it is currently the most comprehensive collection of liver transplantation data available for analysis in the United States and arguably in the world because few data on a center-specific level are available in other countries. Transplantation centers must submit forms detailing yearly organ recipient follow-up, including posttransplant outcomes such as malignancies and causes of death, with the requirement that these forms be completed in a timely manner. The content of these forms, however, is not validated except by individual comparison with the Social Security Death Master File. Additionally, although the most recent UNOS policy puts forth stringent guidelines for reporting information about a recipient's explanted hepatocellular carcinoma (HCC), it contains no mandated follow-up protocols. Thus, there is no formal incentive for centers to report recurrence data accurately and completely, and this leads to the question whether some centers are underreporting and, therefore, exhibiting recurrence rates lower than the actual rates.
The past few years have seen efforts to assess the quality of data in OPTN/UNOS databases. In particular, a 2007 comparison with private insurance claim data sets demonstrated some reporting discrepancies in patient characteristics. Claim comparisons, however, are confounded by the fragmented nature of the private insurance sector: it is difficult to get a data set representative of the patients in the UNOS database.
A more recent analysis compared the OPTN/Scientific Registry of Transplant Recipients (SRTR) database to the Adult-to-Adult Living Donor Liver Transplant Cohort Study (A2ALL) database, which is an independent collection of source data from 9 major transplant centers that has been validated by comparison with national outcomes. Several parameters were found to be significantly different between the 2 data sets; perhaps the most striking was the cold ischemia time. HCC recurrence, however, was not found to be significantly misreported. For patients who received transplants for a diagnosis of HCC (n = 111), 9.0% of the recurrence data were missing from the OPTN database but were present in the A2ALL database, 3.6% were missing from the A2ALL database but not from the OPTN database, and 2.7% were missing from both. When values were present in both data sets, they were inconsistent 8.1% of the time: 7% were recorded as yes in the A2ALL database and as no in the OPTN/SRTR database, and 1% were recorded as no in the A2ALL database and as yes in the OPTN/SRTR database. The study concluded that the discrepancy was symmetric between the OPTN/SRTR and A2ALL databases and suggested that although OPTN data were less carefully collected, HCC recurrence was not systematically underreported.
Because the OPTN database is the best source of data available for analyzing current transplantation practice on a multicenter basis, it is important to establish the validity of the outcome data found within it. In this study, we evaluated the reliability of posttransplant HCC recurrence data in the OPTN/UNOS database by comparing observed recurrence rates to expected rates calculated with a risk-adjusted Poisson model.
PATIENTS AND METHODS
Adults who were listed for primary liver transplantation with an initial exception granted for an HCC diagnosis meeting policy 126.96.36.199 criteria (stage T2) between January 1, 2006 and September 30, 2010 and who underwent transplantation within the same time period were identified from the UNOS Standard Transplant Analysis and Research files (created on March 2, 2012). Patients who died after transplantation because of cholangiocarcinoma were excluded from the analysis because HCC was likely misdiagnosed during the initial evaluation (n = 4).
HCC recurrence, defined as a posttransplant HCC-related death or a diagnosis of HCC recurrence, was determined by a physician's review (J.P.R.) of primary and contributory causes of death or an indication of recurrence in the malignancy follow-up data. The follow-up time after liver transplantation was defined as the number of years from liver transplantation to the first of HCC recurrence, death, or last follow-up. For patients subsequently receiving a second or third liver transplant, the follow-up time was evaluated from the date of first transplantation to the first event after retransplantation (HCC recurrence, death, or last follow-up after retransplantation). The posttransplant vital status and follow-up date were updated when valid data from the Social Security Death Master File were available.
Hepatoma was designated as the primary diagnosis for 34% of the patients. To identify the underlying liver disease for patients with a primary diagnosis of hepatoma, the secondary diagnosis at listing and the diagnosis at transplant (when a secondary diagnosis was unavailable or was also hepatoma) were evaluated. Patients with only a diagnosis of hepatoma and evidence of viral hepatitis (seropositive for hepatitis C virus or positive for hepatitis B virus surface antigen) were categorized by their viral hepatitis diagnosis.
Frequency distributions and medians and interquartile ranges (IQRs) for recipient, donor, and tumor characteristics were described for the total population and by outcome (HCC recurrence versus no reported recurrence). We calculated the tumor volume in cubic centimeters as the volume of a sphere (4/3 × π × tumor radius3; the tumor radius was half of the reported tumor size). For patients with multiple tumors, we summed the volumes of all tumors. The donor risk index (DRI) was calculated in accordance with Feng et al.
To predict the expected number of HCC recurrences by transplant center, we developed a risk-adjusted Poisson model with patients as the unit of analysis, random effects for transplant centers, and years of follow-up as an offset. To select risk adjustment variables, we first estimated single predictor Poisson models for the effects of recipient and tumor characteristics and DRIs on HCC recurrence. Variables with P < 0.1 were evaluated in the multivariate model. All variables remained statistically significant (P < 0.05) except for recipient sex, which was removed from the model. Incidence rate ratios (IRRs) and 95% confidence intervals (CIs) for each covariate were calculated from the multivariate model coefficients. Next, the ratio of observed HCC recurrences to expected recurrences for each transplant center was estimated with the best linear, unbiased prediction of its random effect. This approach differentially shrank the ratio for small centers toward 1, which reflected imprecision due to small numbers. The exponentiated random effect comprised a shrinkage estimate of the ratio of observed recurrences to expected recurrences for each center. We computed both 95% and 90% CIs for the shrinkage observed/expected ratios. Centers with upper confidence limits less than 1 would be identified as potentially underreporting HCC recurrence.
To further investigate the possibility of HCC recurrence underreporting, we imputed the expected value of the outcome for observations with non-HCC deaths on the basis of the first run of the model. This identified patients likely to have experienced unreported HCC recurrence and reclassified the expected proportion of these patients at each center to the HCC recurrence outcome group. We then reran the Poisson model with these imputed values, and we rechecked for centers with shrinkage observed/expected ratios significantly less than 1.
The data manipulation and analysis were completed with SAS 9.3 (SAS Institute, Inc., Cary, NC). Poisson regression modeling was completed with Stata/IC 11.1 (StataCorp, College Station, TX). This study received approval from the committee on human research at the University of California San Francisco.
HCC liver transplant recipients (n = 5034) were primarily male, white, and nondiabetic. Hepatitis C virus was the most common diagnosis (62.1%). At transplant, the patients had a median age of 57 years (IQR = 53-62 years) with a laboratory Model for End-Stage Liver Disease (MELD) score of 12 (IQR = 9-16; Table 1). When patients were granted an HCC exception, the median tumor volume for all tumors combined was 9.2 cm3 (IQR = 5.6-17.2 cm3), with 43.3% of the patients undergoing ablative therapy and 5.9% having an alpha-fetoprotein (AFP) level greater than 500 ng/mL. Multiple tumors were identified in 34.8% of the patients (Table 2).
|Characteristic||Total Population (n = 5034)||No HCC Recurrence (n = 4691)||Recurrence/ Death From HCC (n = 343)|
|Sex: male [n (%)]||3899 (77.5)||3625 (77.3)||274 (79.9)|
|Ethnicity [n (%)]|
|White||3397 (67.5)||3154 (67.2)||243 (70.8)|
|Black||449 (8.9)||427 (9.1)||22 (6.4)|
|Hispanic/Latino||687 (13.6)||650 (13.9)||37 (10.8)|
|Asian||437 (8.7)||399 (8.5)||38 (11.1)|
|Other/multiracial||64 (1.3)||61 (1.3)||3 (0.9)|
|Diabetes [n (%)]||1407 (27.9)||1326 (28.3)||81 (23.6)|
|Intensive care unit at transplant [n (%)]||73 (1.5)||68 (1.4)||5 (1.5)|
|Dialysis in the week before transplant [n (%)]||91 (1.8)||86 (1.8)||5 (1.5)|
|Total assistance at transplant [n (%)]||200 (4.0)||188 (4.0)||12 (3.5)|
|Diagnosis [n (%)]|
|Hepatitis C virus||3128 (62.1)||2905 (61.9)||223 (65.0)|
|Alcoholic cirrhosis||426 (8.5)||403 (8.6)||23 (6.7)|
|Noncholestatic cirrhosis||289 (5.7)||278 (5.9)||11 (3.2)|
|Hepatitis B virus||292 (5.8)||264 (5.6)||28 (8.2)|
|Nonalcoholic steatohepatitis||215 (4.3)||201 (4.3)||14 (4.1)|
|Other||684 (13.6)||640 (13.6)||44 (12.8)|
|Age at transplant (years)a||57 (53-62)||57 (53-62)||57 (53-62)|
|Laboratory MELD score at transplanta||12 (9-16)||12 (9-16)||12 (8-16)|
|Ethnicity [n (%)]|
|White||3320 (66.0)||3107 (66.2)||213 (62.1)|
|Black||835 (16.6)||770 (16.4)||65 (19.0)|
|Hispanic/Latino||686 (13.6)||633 (13.5)||53 (15.5)|
|Asian||151 (3.0)||143 (3.0)||8 (2.3)|
|Other/multiracial||42 (0.8)||38 (0.8)||4 (1.2)|
|Partial or split liver [n (%)]||123 (2.4)||108 (2.3)||15 (4.4)|
|Cause of death: stroke [n (%)]||2100 (41.7)||1940 (41.4)||160 (46.6)|
|Age (years)a||43 (26-55)||43 (26-55)||45 (29-58)|
|DRIa||1.37 (1.13-1.68)||1.37 (1.13-1.67)||1.48 (1.19-1.78)|
|Tumor Characteristic||Total Population (n = 5034)||No HCC Recurrence (n = 4691)||Recurrence/ Death From HCC (n = 343)|
|Tumor number and size [n (%)]|
|>1 tumor, all < 2 cm||638 (12.7)||610 (13.0)||28 (8.2)|
|At least 1 tumor ≥ 2 cm||4099 (81.4)||3806 (81.1)||293 (85.4)|
|2-3 tumors ≥ 2 cm||297 (5.9)||275 (5.9)||22 (6.4)|
|Number of tumors on the wait list [n (%)]|
|1||3283 (65.2)||3042 (64.8)||241 (70.3)|
|2||1229 (24.4)||1158 (24.7)||71 (20.7)|
|3||522 (10.4)||491 (10.5)||31 (9.0)|
|Milan criteria at exception [n (%)]||4978 (98.9)||4644 (99.0)||334 (97.4)|
|Ablative therapy at exception [n (%)]||2179 (43.3)||2004 (42.7)||175 (51.0)|
|AFP > 500 ng/mL at exception [n (%)]||298 (5.9)||247 (5.3)||51 (14.9)|
|Total tumor volume at exception (cm3)a||9.2 (5.6-17.2)||9.2 (5.2-17.2)||12.6 (7.2-22.4)|
|Time from exception to transplant (days)a||77 (21-158)||78 (27-160)||68 (26-134)|
|Posttransplant follow-up (years)a||2.1 (1.0-3.6)||2.1 (1.1-3.8)||1.0 (0.5-1.8)|
Patients were followed for a median of 2.1 years (IQR = 1.0-3.6 years) after liver transplantation. Death due to HCC recurrence or a diagnosis of HCC recurrence was identified for 6.8% of the patients at a median of 1 year (IQR = 0.5-1.8 years) after transplantation (Table 2).
After adjustments for covariates, the IRRs for HCC recurrence were increased for liver transplant recipients with 1 tumor ≥ 2 cm (IRR = 1.63, 95% CI = 1.10-2.41, P = 0.02) or 2 to 3 tumors ≥ 2 cm (IRR = 1.82, 95% CI = 1.04-3.20, P = 0.04) versus recipients with >1 tumor, all < 2 cm; with ablative therapy versus none (IRR = 1.44, 95% CI = 1.15-1.79, P = 0.001); with an AFP level > 500 ng/mL versus an AFP level ≤ 500 (IRR = 3.00, 95% CI = 2.21-4.09, P < 0.001); and with an increasing DRI (IRR = 1.97, 95% CI = 1.52-2.57, P < 0.001). HCC recurrence IRRs were decreased for black race versus white race (IRR = 0.61, 95% CI = 0.39-0.95, P = 0.03) and for noncholestatic cirrhosis versus hepatitis C virus (IRR = 0.48, 95% CI = 0.26-0.89, P = 0.02; Table 3).
|Characteristic||IRR (95% CI)||P Value|
|Tumor number and size|
|>1 tumor, all < 2 cm||1.00|
|1 tumor ≥ 2 cm||1.63 (1.10-2.41)||0.02|
|2-3 tumors ≥ 2 cm||1.82 (1.04-3.20)||0.04|
|Ablative therapy at exception||1.44 (1.15-1.79)||0.001|
|AFP > 500 ng/mL at exception||3.00 (2.21-4.09)||<0.001|
|Hepatitis C virus||1.00|
|Alcoholic cirrhosis||0.71 (0.46-1.09)||0.12|
|Noncholestatic cirrhosis||0.48 (0.26-0.89)||0.02|
|Hepatitis B virus||1.04 (0.65-1.67)||0.86|
|Nonalcoholic steatohepatitis||0.93 (0.54-1.60)||0.78|
The shrinkage estimates of the ratio of observed HCC recurrences to expected HCC recurrences by center ranged from 0.6 to 1.76 (median = 0.97; Fig. 1). Although the model provided strong evidence (P = 0.001) that adjusted recurrence rates varied by center, no transplant center had a shrinkage observed/expected ratio significantly less than 1 on the basis of 95% CIs. Additionally, no transplant center was identified with a significantly higher than expected HCC recurrence rate. Using narrower 90% CIs did not change these results. After we imputed outcomes for patients potentially experiencing unreported HCC recurrence, the center-specific shrinkage ratios ranged from 0.72 to 1.39 (median = 0.98), and no center had shrinkage ratios significantly different from 1.
The evaluation of center-level effects was limited by the small numbers of patients at some centers. In this case, the expected number of recurrences was low, so that even when no recurrences were reported, the CIs for the shrinkage observed/expected ratios did not exclude 1.0.
As the transplant community continues to improve the liver transplantation guidelines for HCC, it is vital to identify and appropriately analyze the data on HCC recurrence. Single-center studies are generally limited by small sample sizes. The UNOS/OPTN database is the largest collection of US liver transplant data available. It has been demonstrated, however, that these data are often incomplete and sometimes inaccurate; a systemic misreporting of HCC recurrence would heavily skew any analysis of recurrence based on this data set. Publications on HCC using data from the OPTN/SRTR database have, therefore, exclusively relied on overall survival as the only outcome measure.[7-11] In fact, overall mortality has been used as a surrogate for tumor recurrence in evaluating the total tumor volume, AFP,[7, 9-11] and pretransplant local regional therapy as prognostic factors. Our study clearly shows that recurrence rates vary by center, even after we account for recipient, donor, and tumor characteristics. However, no centers could be conclusively singled out as having lower or higher than expected recurrence rates.
Shrinkage estimators are used in health quality research to examine center outcomes because they provide more stable and conservative center rankings than traditional alternatives. In brief, the shrinkage rate estimators are weighted averages of the center-specific observed/expected ratios and the ratio for all centers (by construction, 1.0), with weights primarily determined by the sample size at each center. Although the relatively precise estimates for large centers are almost unchanged by this averaging, estimates for smaller centers increasingly shrink toward 1. This decreases the effect of random variation due to small sample sizes on small centers' ratios and prevents them from being unfairly singled out. Mukamel et al. recently provided a cogent discussion of shrinkage estimators in health policy.
Although this analysis demonstrates unexplained variation among centers in recurrence rates, no particular center could be clearly identified as reporting fewer than the expected number of recurrences. Using data for a longer period would increase center sample sizes, but this might also obscure changes in reporting practices over time. Factors other than inaccurate reporting, including mismeasurement or the omission of important risk adjustment variables, could underlie the unexplained variation, although A2ALL comparison data demonstrate that the variables used by us are highly concordant. The absolute accuracy of center HCC reporting is not known and would require a comparison of the OPTN database with one of known quality with respect to HCC recurrence and mortality in the same patient data set. However, the A2ALL comparison demonstrates that even this approach is limited by variations in reporting between data sets.
A limitation is that our primary analysis provides no information about the cause of the differences between observed and expected recurrences reflected in the shrinkage ratios. In particular, it cannot distinguish underreporting from other unmeasured factors that might decrease HCC recurrence, such as earlier transplantation or different ablative techniques. However, our sensitivity analysis imputing recurrences among patients who died without recurrence may more directly capture underreporting. Specifically, the shrinkage relative risk (RR) for a center that underreports recurrences should increase numerically with respect to the estimate without imputation if our supposition is correct that recurrences are primarily underreported among patients who have died. This analysis would be most informative if we had identified centers with RRs significantly below 1.0 in the main analysis and found that their RRs moved toward or even above 1.0 in the sensitivity analysis.
Our inability to identify center underreporting or overreporting opens up the OPTN database to further research. We suggest further research efforts to evaluate whether the quality of reporting remains stable over time. Repeating this type of analysis every few years may be necessary to ensure the reliability of research using the UNOS/OPTN database.
- 1Organ Procurement and Transplantation Network.OPTN policy 7.0. Data submission requirements. http://optn.transplant.hrsa.gov/PoliciesandBylaws2/policies/pdfs/policy_23.pdf. Accessed June 2013.
- 5for Adult-to-Adult Living Donor Liver Transplantation Cohort Study Group. Outcomes of adult living donor liver transplantation: comparison of the Adult-to-Adult Living Donor Liver Transplantation Cohort Study and the national experience. Liver Transpl 2011;17:789-797., , , , , , ;